2024-05-04T02:51:42.200 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-04T02:51:42.200 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} --owner scheduled_pdonnell@teuthology --name pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/orig.config.yaml 2024-05-04T02:51:42.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-04T02:51:42.347 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612 branch: wip-pdonnell-testing-20240503.163550-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7688612' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240503.163550-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240503.163550-debug sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1842 sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 sleep_before_teardown: 0 subset: 478/512 suite: fs suite_branch: wip-pdonnell-testing-20240503.163550-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bd4e00a39d766328b458e4c953f4d722fed96a31/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP1RVUDZNYC3nk0rGOZv8d49lTsGAosjH6I+t56Z9JwdrdqWn+BlFIqtIL4EEDZ5Va2zXUGUgCOsj0AvjK/OYtE= smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJeNqzlzWeGP8RpvgpxrGi7JBVwC7twrTXAoLxxc6X4ruyow/ki3+RU68ym9PtZt44CRdwV1yYrKWGqWo8Fdb7Y= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFzM9Cq1Jfrj1meqnQo/DF3oi4xjYjlpW70OR9xZ+4eRxy30saMbu/rW093g4Oj6OjoeeotRcTHD9O8323o4fDA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-03_22:48:16 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-04T02:51:42.348 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bd4e00a39d766328b458e4c953f4d722fed96a31/qa; will attempt to use it 2024-05-04T02:51:42.348 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bd4e00a39d766328b458e4c953f4d722fed96a31/qa/tasks 2024-05-04T02:51:42.348 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-04T02:51:42.350 INFO:teuthology.task.internal:Checking packages... 2024-05-04T02:51:42.374 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bd4e00a39d766328b458e4c953f4d722fed96a31' 2024-05-04T02:51:42.375 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-04T02:51:42.375 INFO:teuthology.packaging:ref: None 2024-05-04T02:51:42.375 INFO:teuthology.packaging:tag: None 2024-05-04T02:51:42.375 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240503.163550-debug 2024-05-04T02:51:42.375 INFO:teuthology.packaging:sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:51:42.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240503.163550-debug 2024-05-04T02:51:42.595 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3522.gbd4e00a3 2024-05-04T02:51:42.597 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-04T02:51:42.603 INFO:teuthology.task.internal:no buildpackages task found 2024-05-04T02:51:42.603 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-04T02:51:42.637 INFO:teuthology.task.internal:Saving configuration 2024-05-04T02:51:42.657 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-04T02:51:42.703 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-04T02:51:42.729 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 02:45:47.366076', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP1RVUDZNYC3nk0rGOZv8d49lTsGAosjH6I+t56Z9JwdrdqWn+BlFIqtIL4EEDZ5Va2zXUGUgCOsj0AvjK/OYtE='} 2024-05-04T02:51:42.752 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 02:45:47.365082', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJeNqzlzWeGP8RpvgpxrGi7JBVwC7twrTXAoLxxc6X4ruyow/ki3+RU68ym9PtZt44CRdwV1yYrKWGqWo8Fdb7Y='} 2024-05-04T02:51:42.776 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 02:45:47.363864', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFzM9Cq1Jfrj1meqnQo/DF3oi4xjYjlpW70OR9xZ+4eRxy30saMbu/rW093g4Oj6OjoeeotRcTHD9O8323o4fDA='} 2024-05-04T02:51:42.776 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-04T02:51:42.782 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-04T02:51:42.782 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-04T02:51:42.782 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-04T02:51:42.782 INFO:teuthology.run_tasks:Running task console_log... 2024-05-04T02:51:42.924 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f746600f0d0>, signals=[15]) 2024-05-04T02:51:42.925 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-04T02:51:42.930 INFO:teuthology.task.internal:Opening connections... 2024-05-04T02:51:42.930 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T02:51:42.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:51:43.007 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-05-04T02:51:43.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:51:43.078 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-05-04T02:51:43.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:51:43.154 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-04T02:51:43.160 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-05-04T02:51:43.179 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-05-04T02:51:43.179 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-05-04T02:51:43.236 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-05-04T02:51:43.236 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-04T02:51:43.237 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-04T02:51:43.238 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-05-04T02:51:43.257 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-05-04T02:51:43.275 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-05-04T02:51:43.275 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-05-04T02:51:43.333 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-05-04T02:51:43.333 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-05-04T02:51:43.333 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-05-04T02:51:43.333 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-05-04T02:51:43.333 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-05-04T02:51:43.333 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-05-04T02:51:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-04T02:51:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-05-04T02:51:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-05-04T02:51:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-04T02:51:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-05-04T02:51:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-04T02:51:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-04T02:51:43.334 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-04T02:51:43.334 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-05-04T02:51:43.357 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-05-04T02:51:43.375 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-05-04T02:51:43.375 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-04T02:51:43.433 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-04T02:51:43.434 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-04T02:51:43.434 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-05-04T02:51:43.456 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-04T02:51:43.466 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-04T02:51:43.496 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-04T02:51:43.496 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-05-04T02:51:43.498 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-05-04T02:51:43.501 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-05-04T02:51:43.516 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-04T02:51:43.528 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-04T02:51:43.529 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-05-04T02:51:43.555 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-05-04T02:51:43.558 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-05-04T02:51:43.654 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-04T02:51:43.736 INFO:teuthology.run_tasks:Running task kernel... 2024-05-04T02:51:43.751 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-04T02:51:43.751 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-04T02:51:43.751 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-04T02:51:43.752 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-04T02:51:43.752 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-04T02:51:43.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:51:43.769 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-05-04T02:51:43.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:51:43.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:51:43.771 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-05-04T02:51:43.772 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-05-04T02:51:43.827 INFO:teuthology.orchestra.run.smithi167.stdout:5.14.0-437.el9.x86_64 2024-05-04T02:51:43.828 INFO:teuthology.task.kernel:Running kernel on smithi167: 5.14.0-437.el9.x86_64 2024-05-04T02:51:43.828 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum install -y kernel 2024-05-04T02:51:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-437.el9.x86_64 2024-05-04T02:51:43.830 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-437.el9.x86_64 2024-05-04T02:51:43.830 INFO:teuthology.task.kernel:Running kernel on smithi039: 5.14.0-437.el9.x86_64 2024-05-04T02:51:43.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum install -y kernel 2024-05-04T02:51:43.830 INFO:teuthology.task.kernel:Running kernel on smithi066: 5.14.0-437.el9.x86_64 2024-05-04T02:51:43.830 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum install -y kernel 2024-05-04T02:51:47.055 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-05-04T02:51:47.363 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-05-04T02:51:48.583 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.0 MB 00:03 2024-05-04T02:51:49.197 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-04T02:51:50.569 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-05-04T02:51:50.922 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-05-04T02:51:53.281 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-05-04T02:51:54.684 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-04T02:51:54.815 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-05-04T02:51:55.875 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-04T02:51:56.310 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-04T02:51:57.373 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.2 MB 00:02 2024-05-04T02:51:57.947 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-04T02:51:58.998 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-04T02:51:59.883 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-05-04T02:52:01.108 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-05-04T02:52:03.056 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-04T02:52:03.057 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-04T02:52:03.145 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-04T02:52:03.146 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-05-04T02:52:03.146 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-05-04T02:52:03.228 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-05-04T02:52:03.239 DEBUG:teuthology.orchestra.run.smithi039:> echo no | sudo yum reinstall kernel || true 2024-05-04T02:52:03.762 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:02 ago on Sat 04 May 2024 02:52:01 AM UTC. 2024-05-04T02:52:03.886 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-04T02:52:03.902 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalling: 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 255 k 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 0 2024-05-04T02:52:03.903 INFO:teuthology.orchestra.run.smithi039.stderr:Operation aborted. 2024-05-04T02:52:03.961 INFO:teuthology.orchestra.run.smithi039.stdout:Is this ok [y/N]: 2024-05-04T02:52:03.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum reinstall -y kernel || true 2024-05-04T02:52:04.480 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:03 ago on Sat 04 May 2024 02:52:01 AM UTC. 2024-05-04T02:52:04.600 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-04T02:52:04.601 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-04T02:52:04.601 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-05-04T02:52:04.601 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-04T02:52:04.601 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalling: 2024-05-04T02:52:04.601 INFO:teuthology.orchestra.run.smithi039.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-04T02:52:04.601 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:52:04.602 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-05-04T02:52:04.602 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-05-04T02:52:04.602 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:52:04.602 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 255 k 2024-05-04T02:52:04.602 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 0 2024-05-04T02:52:04.602 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-05-04T02:52:04.877 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-05-04T02:52:04.878 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-05-04T02:52:04.878 INFO:teuthology.orchestra.run.smithi039.stdout:Total 925 kB/s | 255 kB 00:00 2024-05-04T02:52:04.895 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-05-04T02:52:05.016 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-04T02:52:05.078 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-05-04T02:52:05.078 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-05-04T02:52:05.083 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-05-04T02:52:05.084 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-05-04T02:52:05.126 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-04T02:52:05.126 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-04T02:52:05.207 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-05-04T02:52:05.208 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2024-05-04T02:52:05.208 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-05-04T02:52:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-05-04T02:52:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-04T02:52:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:05.317 DEBUG:teuthology.orchestra.run.smithi066:> echo no | sudo yum reinstall kernel || true 2024-05-04T02:52:05.841 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:02 ago on Sat 04 May 2024 02:52:03 AM UTC. 2024-05-04T02:52:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-04T02:52:05.972 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalling: 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 255 k 2024-05-04T02:52:05.982 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 0 2024-05-04T02:52:05.983 INFO:teuthology.orchestra.run.smithi066.stderr:Operation aborted. 2024-05-04T02:52:06.036 INFO:teuthology.orchestra.run.smithi066.stdout:Is this ok [y/N]: 2024-05-04T02:52:06.036 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum reinstall -y kernel || true 2024-05-04T02:52:06.551 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:03 ago on Sat 04 May 2024 02:52:03 AM UTC. 2024-05-04T02:52:06.623 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:06.623 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:52:06.623 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalled: 2024-05-04T02:52:06.624 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-04T02:52:06.624 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:52:06.624 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-05-04T02:52:06.672 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalling: 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-05-04T02:52:06.673 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:52:06.674 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 255 k 2024-05-04T02:52:06.674 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 0 2024-05-04T02:52:06.674 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-05-04T02:52:06.791 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel | sort -rV | head -n 1 2024-05-04T02:52:06.839 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-04T02:52:06.840 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-04T02:52:06.840 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-04T02:52:06.840 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-04T02:52:06.840 DEBUG:teuthology.orchestra.run.smithi039:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-04T02:52:06.884 INFO:teuthology.orchestra.run.smithi039.stdout:ttyS1 2024-05-04T02:52:06.905 DEBUG:teuthology.parallel:result is None 2024-05-04T02:52:06.940 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-04T02:52:06.940 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-04T02:52:07.026 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-05-04T02:52:07.027 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-05-04T02:52:07.027 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-05-04T02:52:07.162 DEBUG:teuthology.orchestra.run.smithi167:> echo no | sudo yum reinstall kernel || true 2024-05-04T02:52:07.330 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-437.el9.x86_64.rpm 430 kB/s | 255 kB 00:00 2024-05-04T02:52:07.330 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-05-04T02:52:07.330 INFO:teuthology.orchestra.run.smithi066.stdout:Total 388 kB/s | 255 kB 00:00 2024-05-04T02:52:07.352 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-05-04T02:52:07.543 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-05-04T02:52:07.544 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-05-04T02:52:07.549 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-05-04T02:52:07.549 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-05-04T02:52:07.664 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:02 ago on Sat 04 May 2024 02:52:05 AM UTC. 2024-05-04T02:52:07.670 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-05-04T02:52:07.748 INFO:teuthology.orchestra.run.smithi066.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-04T02:52:07.748 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:07.787 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-05-04T02:52:07.796 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-04T02:52:07.796 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-05-04T02:52:07.796 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stdout:Reinstalling: 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 255 k 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 0 2024-05-04T02:52:07.797 INFO:teuthology.orchestra.run.smithi167.stderr:Operation aborted. 2024-05-04T02:52:07.858 INFO:teuthology.orchestra.run.smithi167.stdout:Is this ok [y/N]: 2024-05-04T02:52:07.858 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum reinstall -y kernel || true 2024-05-04T02:52:08.354 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:03 ago on Sat 04 May 2024 02:52:05 AM UTC. 2024-05-04T02:52:08.473 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-05-04T02:52:08.473 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-04T02:52:08.473 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-05-04T02:52:08.473 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-04T02:52:08.473 INFO:teuthology.orchestra.run.smithi167.stdout:Reinstalling: 2024-05-04T02:52:08.473 INFO:teuthology.orchestra.run.smithi167.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-04T02:52:08.474 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:52:08.474 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-05-04T02:52:08.474 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-04T02:52:08.474 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:52:08.474 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 255 k 2024-05-04T02:52:08.474 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 0 2024-05-04T02:52:08.474 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-05-04T02:52:08.488 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:08.488 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-04T02:52:08.744 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.2 MB/s | 255 kB 00:00 2024-05-04T02:52:08.744 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-05-04T02:52:08.744 INFO:teuthology.orchestra.run.smithi167.stdout:Total 945 kB/s | 255 kB 00:00 2024-05-04T02:52:08.759 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-05-04T02:52:08.970 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-05-04T02:52:08.970 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-05-04T02:52:08.976 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-05-04T02:52:08.976 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-05-04T02:52:09.096 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-05-04T02:52:09.141 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:09.141 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:52:09.141 INFO:teuthology.orchestra.run.smithi066.stdout:Reinstalled: 2024-05-04T02:52:09.141 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-04T02:52:09.141 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:52:09.141 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-05-04T02:52:09.178 INFO:teuthology.orchestra.run.smithi167.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-04T02:52:09.178 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:09.310 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-05-04T02:52:09.357 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-04T02:52:09.358 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-04T02:52:09.358 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-04T02:52:09.358 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-04T02:52:09.358 DEBUG:teuthology.orchestra.run.smithi066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-04T02:52:09.390 INFO:teuthology.orchestra.run.smithi066.stdout:ttyS1 2024-05-04T02:52:09.410 DEBUG:teuthology.parallel:result is None 2024-05-04T02:52:09.897 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:09.897 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-04T02:52:10.575 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:52:10.575 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:52:10.575 INFO:teuthology.orchestra.run.smithi167.stdout:Reinstalled: 2024-05-04T02:52:10.576 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-04T02:52:10.576 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:52:10.576 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-05-04T02:52:10.800 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel | sort -rV | head -n 1 2024-05-04T02:52:10.847 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-04T02:52:10.847 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-04T02:52:10.847 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-04T02:52:10.847 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-04T02:52:10.848 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-04T02:52:10.876 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2024-05-04T02:52:10.897 DEBUG:teuthology.parallel:result is None 2024-05-04T02:52:10.897 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-04T02:52:10.905 INFO:teuthology.task.internal:Creating test directory... 2024-05-04T02:52:10.905 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-04T02:52:10.908 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-04T02:52:10.910 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-04T02:52:10.952 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-04T02:52:11.015 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-04T02:52:11.042 INFO:teuthology.task.internal:Creating archive directory... 2024-05-04T02:52:11.043 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-04T02:52:11.045 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-04T02:52:11.048 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-04T02:52:11.090 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-04T02:52:11.096 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-04T02:52:11.097 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-04T02:52:11.110 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-04T02:52:11.126 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-04T02:52:11.152 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:52:11.163 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:52:11.164 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:52:11.173 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:52:11.178 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:52:11.189 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:52:11.190 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-04T02:52:11.198 INFO:teuthology.task.internal:Configuring sudo... 2024-05-04T02:52:11.198 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-04T02:52:11.200 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-04T02:52:11.202 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-04T02:52:11.238 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-04T02:52:11.248 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-04T02:52:11.249 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-04T02:52:11.268 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-04T02:52:11.271 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-04T02:52:11.293 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:52:11.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:52:11.416 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:52:11.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:52:11.545 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T02:52:11.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-04T02:52:11.611 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:52:11.652 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:52:11.708 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:52:11.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:52:11.837 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T02:52:11.838 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-04T02:52:11.905 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:52:11.944 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:52:11.993 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:52:12.072 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:52:12.114 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T02:52:12.114 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-04T02:52:12.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-05-04T02:52:12.183 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-05-04T02:52:12.186 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-05-04T02:52:12.221 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T02:52:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T02:52:12.263 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T02:52:12.726 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-04T02:52:12.734 INFO:teuthology.task.internal:Starting timer... 2024-05-04T02:52:12.734 INFO:teuthology.run_tasks:Running task pcp... 2024-05-04T02:52:12.771 INFO:teuthology.run_tasks:Running task selinux... 2024-05-04T02:52:12.817 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-04T02:52:12.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-05-04T02:52:12.881 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-05-04T02:52:12.882 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-05-04T02:52:12.942 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-05-04T02:52:12.944 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-05-04T02:52:12.992 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-05-04T02:52:12.994 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-04T02:52:12.994 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-05-04T02:52:13.019 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-05-04T02:52:13.019 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-05-04T02:52:13.042 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-05-04T02:52:13.042 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-05-04T02:52:13.077 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-05-04T02:52:13.077 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-05-04T02:52:13.077 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T02:52:13.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:52:13.103 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T02:52:13.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:52:13.130 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T02:52:13.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:52:13.154 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-04T02:52:13.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-05-04T02:52:13.179 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-05-04T02:52:13.206 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-05-04T02:52:13.229 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-04T02:52:13.238 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-04T02:52:13.255 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-04T02:52:13.257 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi066.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-04T02:56:07.837 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-05-04T02:56:07.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-05-04T02:56:07.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:56:07.916 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-05-04T02:56:07.994 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-05-04T02:56:07.995 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-05-04T02:56:07.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:56:08.077 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-05-04T02:56:08.165 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-05-04T02:56:08.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-04T02:56:08.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:56:08.241 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-05-04T02:56:08.314 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-04T02:56:08.314 INFO:teuthology.run_tasks:Running task clock... 2024-05-04T02:56:08.325 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-04T02:56:08.325 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-04T02:56:08.326 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T02:56:08.329 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-04T02:56:08.329 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T02:56:08.331 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-04T02:56:08.331 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T02:56:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-04T02:56:08.358 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-04T02:56:08.373 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-04T02:56:08.373 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-04T02:56:08.383 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-04T02:56:08.396 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-05-04T02:56:08.397 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-04T02:56:08.397 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-05-04T02:56:08.407 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-05-04T02:56:08.408 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-05-04T02:56:08.420 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-05-04T02:56:08.420 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-04T02:56:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-04T02:56:08.431 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-05-04T02:56:08.434 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-04T02:56:08.436 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-04T02:56:08.444 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-04T02:56:08.457 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-04T02:56:08.491 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-05-04T02:56:08.495 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T02:56:08.495 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-05-04T02:56:08.495 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.495 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.495 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.495 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.501 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-05-04T02:56:08.504 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T02:56:08.504 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-05-04T02:56:08.504 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.504 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.504 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.504 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.510 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-05-04T02:56:08.513 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T02:56:08.513 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-05-04T02:56:08.513 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.513 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.513 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.513 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T02:56:08.514 INFO:teuthology.run_tasks:Running task install... 2024-05-04T02:56:08.522 DEBUG:teuthology.task.install:project ceph 2024-05-04T02:56:08.522 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31'}} 2024-05-04T02:56:08.522 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31'} 2024-05-04T02:56:08.522 INFO:teuthology.task.install:Using flavor: default 2024-05-04T02:56:08.530 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-04T02:56:08.530 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-04T02:56:08.530 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-04T02:56:08.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:56:08.533 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-04T02:56:08.533 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:56:08.534 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-04T02:56:08.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:56:08.752 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-04T02:56:08.752 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-04T02:56:08.765 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-04T02:56:08.765 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-04T02:56:08.769 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-04T02:56:08.769 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-04T02:56:08.876 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-04T02:56:08.876 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T02:56:08.877 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-04T02:56:08.883 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-04T02:56:08.883 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T02:56:08.883 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-04T02:56:08.901 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-04T02:56:08.901 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-04T02:56:08.910 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-04T02:56:08.910 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-04T02:56:08.932 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-04T02:56:08.932 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T02:56:08.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-04T02:56:08.959 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-04T02:56:08.959 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-04T02:56:08.965 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-04T02:56:08.975 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-04T02:56:09.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-04T02:56:09.039 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-04T02:56:09.049 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-04T02:56:09.100 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-04T02:56:09.102 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-05-04T02:56:09.103 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-05-04T02:56:09.112 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-05-04T02:56:09.114 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-05-04T02:56:09.165 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-05-04T02:56:09.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-05-04T02:56:09.328 INFO:teuthology.orchestra.run.smithi167.stdout:50 files removed 2024-05-04T02:56:09.348 INFO:teuthology.orchestra.run.smithi066.stdout:50 files removed 2024-05-04T02:56:09.355 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-04T02:56:09.375 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-04T02:56:09.402 INFO:teuthology.orchestra.run.smithi039.stdout:50 files removed 2024-05-04T02:56:09.430 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-04T02:56:10.020 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 206 kB/s | 84 kB 00:00 2024-05-04T02:56:10.062 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 204 kB/s | 84 kB 00:00 2024-05-04T02:56:10.078 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 222 kB/s | 84 kB 00:00 2024-05-04T02:56:10.454 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-04T02:56:10.462 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-04T02:56:10.496 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-04T02:56:10.821 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-05-04T02:56:10.863 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-04T02:56:10.913 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-04T02:56:11.372 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-05-04T02:56:11.463 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-05-04T02:56:11.621 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-05-04T02:56:13.531 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-04T02:56:13.758 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-05-04T02:56:18.150 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.2 MB 00:00 2024-05-04T02:56:19.399 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-05-04T02:56:20.549 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-05-04T02:56:20.991 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-05-04T02:56:22.520 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-04T02:56:22.643 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-04T02:56:27.742 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-04T02:56:27.859 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-05-04T02:56:28.529 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-05-04T02:56:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T02:56:29.945 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T02:56:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-04T02:56:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-04T02:56:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-04T02:56:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-04T02:56:29.960 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-04T02:56:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-04T02:56:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-04T02:56:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-05-04T02:56:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-05-04T02:56:30.036 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-05-04T02:56:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-05-04T02:56:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-05-04T02:56:30.036 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-04T02:56:30.037 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-04T02:56:30.038 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-04T02:56:30.039 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-04T02:56:30.040 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-04T02:56:30.041 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-04T02:56:30.042 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-04T02:56:30.043 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-04T02:56:30.044 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-04T02:56:30.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-04T02:56:30.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-04T02:56:30.047 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:Installing weak dependencies: 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:Install 148 Packages 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 4 Packages 2024-05-04T02:56:30.048 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:56:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 214 M 2024-05-04T02:56:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-05-04T02:56:30.061 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T02:56:30.062 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T02:56:30.073 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-04T02:56:30.074 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-04T02:56:30.075 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-04T02:56:30.075 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-04T02:56:30.077 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-04T02:56:30.079 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-04T02:56:30.080 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-04T02:56:30.148 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-05-04T02:56:30.157 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-05-04T02:56:30.157 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-05-04T02:56:30.157 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-04T02:56:30.158 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-04T02:56:30.159 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-05-04T02:56:30.160 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-04T02:56:30.161 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-04T02:56:30.162 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-04T02:56:30.163 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-04T02:56:30.164 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-04T02:56:30.165 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-04T02:56:30.166 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-04T02:56:30.167 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-04T02:56:30.168 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-05-04T02:56:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-04T02:56:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:56:30.169 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-05-04T02:56:30.169 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-05-04T02:56:30.169 INFO:teuthology.orchestra.run.smithi066.stdout:Install 148 Packages 2024-05-04T02:56:30.169 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 4 Packages 2024-05-04T02:56:30.169 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:56:30.169 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 214 M 2024-05-04T02:56:30.170 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-05-04T02:56:30.603 INFO:teuthology.orchestra.run.smithi066.stdout:(1/152): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-04T02:56:30.619 INFO:teuthology.orchestra.run.smithi039.stdout:(1/152): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-04T02:56:30.870 INFO:teuthology.orchestra.run.smithi066.stdout:(2/152): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 3.2 MB/s | 863 kB 00:00 2024-05-04T02:56:30.894 INFO:teuthology.orchestra.run.smithi039.stdout:(2/152): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 3.1 MB/s | 863 kB 00:00 2024-05-04T02:56:30.936 INFO:teuthology.orchestra.run.smithi066.stdout:(3/152): ceph-immutable-object-cache-19.0.0-352 2.2 MB/s | 147 kB 00:00 2024-05-04T02:56:30.961 INFO:teuthology.orchestra.run.smithi039.stdout:(3/152): ceph-immutable-object-cache-19.0.0-352 2.2 MB/s | 147 kB 00:00 2024-05-04T02:56:31.428 INFO:teuthology.orchestra.run.smithi066.stdout:(4/152): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 4.6 MB/s | 2.3 MB 00:00 2024-05-04T02:56:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:(4/152): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 4.4 MB/s | 2.3 MB 00:00 2024-05-04T02:56:31.520 INFO:teuthology.orchestra.run.smithi066.stdout:(5/152): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 5.0 MB/s | 5.4 MB 00:01 2024-05-04T02:56:31.796 INFO:teuthology.orchestra.run.smithi066.stdout:(6/152): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-05-04T02:56:31.803 INFO:teuthology.orchestra.run.smithi039.stdout:(5/152): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 4.6 MB/s | 1.5 MB 00:00 2024-05-04T02:56:32.330 INFO:teuthology.orchestra.run.smithi066.stdout:(7/152): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 6.0 MB/s | 4.9 MB 00:00 2024-05-04T02:56:32.774 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-05-04T02:56:32.978 INFO:teuthology.orchestra.run.smithi039.stdout:(6/152): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-05-04T02:56:33.053 INFO:teuthology.orchestra.run.smithi039.stdout:(7/152): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 3.9 MB/s | 4.9 MB 00:01 2024-05-04T02:56:34.046 INFO:teuthology.orchestra.run.smithi066.stdout:(8/152): ceph-common-19.0.0-3522.gbd4e00a3.el9. 5.4 MB/s | 20 MB 00:03 2024-05-04T02:56:34.154 INFO:teuthology.orchestra.run.smithi066.stdout:(9/152): ceph-radosgw-19.0.0-3522.gbd4e00a3.el9 5.5 MB/s | 10 MB 00:01 2024-05-04T02:56:34.180 INFO:teuthology.orchestra.run.smithi066.stdout:(10/152): ceph-selinux-19.0.0-3522.gbd4e00a3.el 188 kB/s | 25 kB 00:00 2024-05-04T02:56:34.271 INFO:teuthology.orchestra.run.smithi066.stdout:(11/152): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 2.4 MB/s | 224 kB 00:00 2024-05-04T02:56:34.439 INFO:teuthology.orchestra.run.smithi066.stdout:(12/152): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x8 6.7 MB/s | 18 MB 00:02 2024-05-04T02:56:34.464 INFO:teuthology.orchestra.run.smithi066.stdout:(13/152): cephfs-shell-19.0.0-3522.gbd4e00a3.el 141 kB/s | 27 kB 00:00 2024-05-04T02:56:34.497 INFO:teuthology.orchestra.run.smithi066.stdout:(14/152): libcephfs-devel-19.0.0-3522.gbd4e00a3 544 kB/s | 32 kB 00:00 2024-05-04T02:56:34.590 INFO:teuthology.orchestra.run.smithi066.stdout:(15/152): libcephsqlite-19.0.0-3522.gbd4e00a3.e 1.7 MB/s | 158 kB 00:00 2024-05-04T02:56:34.665 INFO:teuthology.orchestra.run.smithi066.stdout:(16/152): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 3.7 MB/s | 756 kB 00:00 2024-05-04T02:56:34.698 INFO:teuthology.orchestra.run.smithi066.stdout:(17/152): librados-devel-19.0.0-3522.gbd4e00a3. 1.1 MB/s | 127 kB 00:00 2024-05-04T02:56:34.728 INFO:teuthology.orchestra.run.smithi039.stdout:(8/152): ceph-radosgw-19.0.0-3522.gbd4e00a3.el9 6.0 MB/s | 10 MB 00:01 2024-05-04T02:56:34.757 INFO:teuthology.orchestra.run.smithi066.stdout:(18/152): libradosstriper1-19.0.0-3522.gbd4e00a 5.3 MB/s | 497 kB 00:00 2024-05-04T02:56:34.804 INFO:teuthology.orchestra.run.smithi039.stdout:(9/152): ceph-selinux-19.0.0-3522.gbd4e00a3.el9 335 kB/s | 25 kB 00:00 2024-05-04T02:56:34.815 INFO:teuthology.orchestra.run.smithi066.stdout:(19/152): python3-ceph-argparse-19.0.0-3522.gbd 775 kB/s | 45 kB 00:00 2024-05-04T02:56:34.908 INFO:teuthology.orchestra.run.smithi066.stdout:(20/152): python3-ceph-common-19.0.0-3522.gbd4e 1.4 MB/s | 135 kB 00:00 2024-05-04T02:56:34.991 INFO:teuthology.orchestra.run.smithi066.stdout:(21/152): python3-cephfs-19.0.0-3522.gbd4e00a3. 1.9 MB/s | 162 kB 00:00 2024-05-04T02:56:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-04T02:56:35.046 INFO:teuthology.orchestra.run.smithi039.stdout:(10/152): ceph-common-19.0.0-3522.gbd4e00a3.el9 4.3 MB/s | 20 MB 00:04 2024-05-04T02:56:35.083 INFO:teuthology.orchestra.run.smithi066.stdout:(22/152): python3-rados-19.0.0-3522.gbd4e00a3.e 3.4 MB/s | 321 kB 00:00 2024-05-04T02:56:35.154 INFO:teuthology.orchestra.run.smithi039.stdout:(11/152): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 2.0 MB/s | 224 kB 00:00 2024-05-04T02:56:35.175 INFO:teuthology.orchestra.run.smithi066.stdout:(23/152): python3-rbd-19.0.0-3522.gbd4e00a3.el9 3.2 MB/s | 298 kB 00:00 2024-05-04T02:56:35.238 INFO:teuthology.orchestra.run.smithi039.stdout:(12/152): cephfs-shell-19.0.0-3522.gbd4e00a3.el 323 kB/s | 27 kB 00:00 2024-05-04T02:56:35.251 INFO:teuthology.orchestra.run.smithi066.stdout:(24/152): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 9.4 MB/s | 5.2 MB 00:00 2024-05-04T02:56:35.276 INFO:teuthology.orchestra.run.smithi066.stdout:(25/152): python3-rgw-19.0.0-3522.gbd4e00a3.el9 992 kB/s | 100 kB 00:00 2024-05-04T02:56:35.301 INFO:teuthology.orchestra.run.smithi066.stdout:(26/152): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-04T02:56:35.305 INFO:teuthology.orchestra.run.smithi039.stdout:(13/152): libcephfs-devel-19.0.0-3522.gbd4e00a3 476 kB/s | 32 kB 00:00 2024-05-04T02:56:35.360 INFO:teuthology.orchestra.run.smithi066.stdout:(27/152): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 2.8 MB/s | 170 kB 00:00 2024-05-04T02:56:35.435 INFO:teuthology.orchestra.run.smithi066.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3522.g 456 kB/s | 34 kB 00:00 2024-05-04T02:56:35.472 INFO:teuthology.orchestra.run.smithi039.stdout:(14/152): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 4.4 MB/s | 756 kB 00:00 2024-05-04T02:56:35.510 INFO:teuthology.orchestra.run.smithi066.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 1.9 MB/s | 142 kB 00:00 2024-05-04T02:56:35.572 INFO:teuthology.orchestra.run.smithi039.stdout:(15/152): libcephsqlite-19.0.0-3522.gbd4e00a3.e 1.5 MB/s | 158 kB 00:00 2024-05-04T02:56:35.673 INFO:teuthology.orchestra.run.smithi039.stdout:(16/152): librados-devel-19.0.0-3522.gbd4e00a3. 1.2 MB/s | 127 kB 00:00 2024-05-04T02:56:35.856 INFO:teuthology.orchestra.run.smithi039.stdout:(17/152): libradosstriper1-19.0.0-3522.gbd4e00a 2.7 MB/s | 497 kB 00:00 2024-05-04T02:56:35.911 INFO:teuthology.orchestra.run.smithi066.stdout:(30/152): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 4.8 MB/s | 3.0 MB 00:00 2024-05-04T02:56:36.019 INFO:teuthology.orchestra.run.smithi066.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 7.0 MB/s | 3.5 MB 00:00 2024-05-04T02:56:36.103 INFO:teuthology.orchestra.run.smithi066.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3522.gbd 3.2 MB/s | 274 kB 00:00 2024-05-04T02:56:36.178 INFO:teuthology.orchestra.run.smithi066.stdout:(33/152): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 657 kB/s | 49 kB 00:00 2024-05-04T02:56:36.253 INFO:teuthology.orchestra.run.smithi066.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3522.gb 220 kB/s | 16 kB 00:00 2024-05-04T02:56:36.345 INFO:teuthology.orchestra.run.smithi066.stdout:(35/152): ceph-volume-19.0.0-3522.gbd4e00a3.el9 2.9 MB/s | 273 kB 00:00 2024-05-04T02:56:36.512 INFO:teuthology.orchestra.run.smithi066.stdout:(36/152): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 4.5 MB/s | 763 kB 00:00 2024-05-04T02:56:36.577 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-04T02:56:36.587 INFO:teuthology.orchestra.run.smithi066.stdout:(37/152): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 244 kB/s | 18 kB 00:00 2024-05-04T02:56:36.698 INFO:teuthology.orchestra.run.smithi039.stdout:(18/152): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 6.1 MB/s | 5.2 MB 00:00 2024-05-04T02:56:36.782 INFO:teuthology.orchestra.run.smithi039.stdout:(19/152): python3-ceph-argparse-19.0.0-3522.gbd 542 kB/s | 45 kB 00:00 2024-05-04T02:56:36.796 INFO:teuthology.orchestra.run.smithi066.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 386 kB/s | 80 kB 00:00 2024-05-04T02:56:36.857 INFO:teuthology.orchestra.run.smithi039.stdout:(20/152): python3-ceph-common-19.0.0-3522.gbd4e 1.8 MB/s | 135 kB 00:00 2024-05-04T02:56:36.913 INFO:teuthology.orchestra.run.smithi066.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-05-04T02:56:36.941 INFO:teuthology.orchestra.run.smithi039.stdout:(21/152): python3-cephfs-19.0.0-3522.gbd4e00a3. 1.9 MB/s | 162 kB 00:00 2024-05-04T02:56:36.972 INFO:teuthology.orchestra.run.smithi066.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-04T02:56:37.024 INFO:teuthology.orchestra.run.smithi039.stdout:(22/152): python3-rados-19.0.0-3522.gbd4e00a3.e 3.8 MB/s | 321 kB 00:00 2024-05-04T02:56:37.030 INFO:teuthology.orchestra.run.smithi066.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-05-04T02:56:37.099 INFO:teuthology.orchestra.run.smithi039.stdout:(23/152): python3-rbd-19.0.0-3522.gbd4e00a3.el9 3.9 MB/s | 298 kB 00:00 2024-05-04T02:56:37.114 INFO:teuthology.orchestra.run.smithi066.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-05-04T02:56:37.173 INFO:teuthology.orchestra.run.smithi066.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-04T02:56:37.175 INFO:teuthology.orchestra.run.smithi039.stdout:(24/152): python3-rgw-19.0.0-3522.gbd4e00a3.el9 1.3 MB/s | 100 kB 00:00 2024-05-04T02:56:37.242 INFO:teuthology.orchestra.run.smithi066.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-04T02:56:37.250 INFO:teuthology.orchestra.run.smithi039.stdout:(25/152): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-04T02:56:37.307 INFO:teuthology.orchestra.run.smithi066.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-04T02:56:37.365 INFO:teuthology.orchestra.run.smithi066.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-04T02:56:37.499 INFO:teuthology.orchestra.run.smithi066.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-05-04T02:56:37.641 INFO:teuthology.orchestra.run.smithi066.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-05-04T02:56:37.683 INFO:teuthology.orchestra.run.smithi066.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-04T02:56:37.709 INFO:teuthology.orchestra.run.smithi039.stdout:(26/152): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 6.6 MB/s | 3.0 MB 00:00 2024-05-04T02:56:37.733 INFO:teuthology.orchestra.run.smithi066.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-05-04T02:56:37.775 INFO:teuthology.orchestra.run.smithi066.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-05-04T02:56:37.784 INFO:teuthology.orchestra.run.smithi039.stdout:(27/152): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-04T02:56:37.800 INFO:teuthology.orchestra.run.smithi066.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-04T02:56:37.826 INFO:teuthology.orchestra.run.smithi066.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-05-04T02:56:37.853 INFO:teuthology.orchestra.run.smithi039.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3522.g 512 kB/s | 34 kB 00:00 2024-05-04T02:56:37.876 INFO:teuthology.orchestra.run.smithi066.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-04T02:56:37.926 INFO:teuthology.orchestra.run.smithi039.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 1.9 MB/s | 142 kB 00:00 2024-05-04T02:56:37.935 INFO:teuthology.orchestra.run.smithi066.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-05-04T02:56:37.960 INFO:teuthology.orchestra.run.smithi066.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-04T02:56:37.985 INFO:teuthology.orchestra.run.smithi066.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-04T02:56:38.010 INFO:teuthology.orchestra.run.smithi066.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-05-04T02:56:38.044 INFO:teuthology.orchestra.run.smithi066.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-04T02:56:38.078 INFO:teuthology.orchestra.run.smithi066.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-04T02:56:38.103 INFO:teuthology.orchestra.run.smithi066.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-05-04T02:56:38.128 INFO:teuthology.orchestra.run.smithi066.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-04T02:56:38.154 INFO:teuthology.orchestra.run.smithi066.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-04T02:56:38.179 INFO:teuthology.orchestra.run.smithi066.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-04T02:56:38.204 INFO:teuthology.orchestra.run.smithi066.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-05-04T02:56:38.355 INFO:teuthology.orchestra.run.smithi066.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-05-04T02:56:38.421 INFO:teuthology.orchestra.run.smithi066.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-04T02:56:38.426 INFO:teuthology.orchestra.run.smithi039.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 7.1 MB/s | 3.5 MB 00:00 2024-05-04T02:56:38.463 INFO:teuthology.orchestra.run.smithi066.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-04T02:56:38.497 INFO:teuthology.orchestra.run.smithi066.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-04T02:56:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-04T02:56:38.598 INFO:teuthology.orchestra.run.smithi066.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-05-04T02:56:38.681 INFO:teuthology.orchestra.run.smithi066.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-05-04T02:56:38.707 INFO:teuthology.orchestra.run.smithi066.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 803 kB/s | 87 kB 00:00 2024-05-04T02:56:38.765 INFO:teuthology.orchestra.run.smithi066.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-05-04T02:56:38.849 INFO:teuthology.orchestra.run.smithi066.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-05-04T02:56:38.874 INFO:teuthology.orchestra.run.smithi066.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-04T02:56:38.891 INFO:teuthology.orchestra.run.smithi066.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-05-04T02:56:38.917 INFO:teuthology.orchestra.run.smithi066.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 455 kB/s | 19 kB 00:00 2024-05-04T02:56:38.942 INFO:teuthology.orchestra.run.smithi066.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-04T02:56:38.967 INFO:teuthology.orchestra.run.smithi066.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-05-04T02:56:39.034 INFO:teuthology.orchestra.run.smithi066.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-05-04T02:56:39.101 INFO:teuthology.orchestra.run.smithi066.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-05-04T02:56:39.126 INFO:teuthology.orchestra.run.smithi066.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-04T02:56:39.152 INFO:teuthology.orchestra.run.smithi066.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-04T02:56:39.210 INFO:teuthology.orchestra.run.smithi066.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 42 kB 00:00 2024-05-04T02:56:39.236 INFO:teuthology.orchestra.run.smithi066.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-05-04T02:56:39.261 INFO:teuthology.orchestra.run.smithi066.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-04T02:56:39.286 INFO:teuthology.orchestra.run.smithi066.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-04T02:56:39.311 INFO:teuthology.orchestra.run.smithi066.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-04T02:56:39.336 INFO:teuthology.orchestra.run.smithi066.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-04T02:56:39.402 INFO:teuthology.orchestra.run.smithi039.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-05-04T02:56:39.485 INFO:teuthology.orchestra.run.smithi039.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3522.gbd 3.2 MB/s | 274 kB 00:00 2024-05-04T02:56:39.552 INFO:teuthology.orchestra.run.smithi039.stdout:(33/152): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 741 kB/s | 49 kB 00:00 2024-05-04T02:56:39.611 INFO:teuthology.orchestra.run.smithi039.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3522.gb 282 kB/s | 16 kB 00:00 2024-05-04T02:56:39.686 INFO:teuthology.orchestra.run.smithi039.stdout:(35/152): ceph-volume-19.0.0-3522.gbd4e00a3.el9 3.6 MB/s | 273 kB 00:00 2024-05-04T02:56:39.737 INFO:teuthology.orchestra.run.smithi066.stdout:(91/152): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 7.7 MB/s | 43 MB 00:05 2024-05-04T02:56:39.762 INFO:teuthology.orchestra.run.smithi066.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 98 kB/s | 42 kB 00:00 2024-05-04T02:56:39.812 INFO:teuthology.orchestra.run.smithi066.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-04T02:56:39.854 INFO:teuthology.orchestra.run.smithi066.stdout:(94/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-05-04T02:56:39.880 INFO:teuthology.orchestra.run.smithi066.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-05-04T02:56:39.906 INFO:teuthology.orchestra.run.smithi066.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-04T02:56:39.930 INFO:teuthology.orchestra.run.smithi066.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-04T02:56:39.955 INFO:teuthology.orchestra.run.smithi066.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-04T02:56:39.981 INFO:teuthology.orchestra.run.smithi066.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-04T02:56:40.006 INFO:teuthology.orchestra.run.smithi066.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-04T02:56:40.031 INFO:teuthology.orchestra.run.smithi066.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-05-04T02:56:40.078 INFO:teuthology.orchestra.run.smithi039.stdout:(36/152): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 8.2 MB/s | 43 MB 00:05 2024-05-04T02:56:40.082 INFO:teuthology.orchestra.run.smithi066.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-05-04T02:56:40.120 INFO:teuthology.orchestra.run.smithi039.stdout:(37/152): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 1.7 MB/s | 763 kB 00:00 2024-05-04T02:56:40.145 INFO:teuthology.orchestra.run.smithi039.stdout:(38/152): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 274 kB/s | 18 kB 00:00 2024-05-04T02:56:40.266 INFO:teuthology.orchestra.run.smithi066.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-05-04T02:56:40.304 INFO:teuthology.orchestra.run.smithi039.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 437 kB/s | 80 kB 00:00 2024-05-04T02:56:40.307 INFO:teuthology.orchestra.run.smithi066.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-05-04T02:56:40.398 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] gperftools-libs-2.9.1-2.el9.x86_64.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/g/gperftools-libs-2.9.1-2.el9.x86_64.rpm [SSL certificate problem: certificate has expired] 2024-05-04T02:56:40.404 INFO:teuthology.orchestra.run.smithi039.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-05-04T02:56:40.423 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] abseil-cpp-20211102.0-3.el9.x86_64.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/a/abseil-cpp-20211102.0-3.el9.x86_64.rpm [SSL certificate problem: certificate has expired] 2024-05-04T02:56:40.448 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] grpc-data-1.46.7-10.el9.noarch.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/g/grpc-data-1.46.7-10.el9.noarch.rpm [SSL certificate problem: certificate has expired] 2024-05-04T02:56:40.488 INFO:teuthology.orchestra.run.smithi039.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-04T02:56:40.522 INFO:teuthology.orchestra.run.smithi039.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-05-04T02:56:40.589 INFO:teuthology.orchestra.run.smithi039.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-04T02:56:40.633 INFO:teuthology.orchestra.run.smithi066.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 60 kB/s | 19 kB 00:00 2024-05-04T02:56:40.664 INFO:teuthology.orchestra.run.smithi039.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-04T02:56:40.698 INFO:teuthology.orchestra.run.smithi039.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-05-04T02:56:40.731 INFO:teuthology.orchestra.run.smithi039.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-04T02:56:40.756 INFO:teuthology.orchestra.run.smithi039.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-04T02:56:40.783 INFO:teuthology.orchestra.run.smithi066.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 597 kB/s | 309 kB 00:00 2024-05-04T02:56:40.867 INFO:teuthology.orchestra.run.smithi066.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-05-04T02:56:40.892 INFO:teuthology.orchestra.run.smithi066.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 677 kB/s | 548 kB 00:00 2024-05-04T02:56:40.951 INFO:teuthology.orchestra.run.smithi066.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-05-04T02:56:40.976 INFO:teuthology.orchestra.run.smithi066.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 808 kB/s | 67 kB 00:00 2024-05-04T02:56:41.043 INFO:teuthology.orchestra.run.smithi066.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-04T02:56:41.151 INFO:teuthology.orchestra.run.smithi066.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-05-04T02:56:41.218 INFO:teuthology.orchestra.run.smithi066.stdout:(113/152): libarrow-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2024-05-04T02:56:41.244 INFO:teuthology.orchestra.run.smithi066.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-05-04T02:56:41.269 INFO:teuthology.orchestra.run.smithi066.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-05-04T02:56:41.290 INFO:teuthology.orchestra.run.smithi039.stdout:(48/152): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x8 2.1 MB/s | 18 MB 00:08 2024-05-04T02:56:41.294 INFO:teuthology.orchestra.run.smithi066.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 427 kB/s | 32 kB 00:00 2024-05-04T02:56:41.319 INFO:teuthology.orchestra.run.smithi066.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-04T02:56:41.353 INFO:teuthology.orchestra.run.smithi066.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-04T02:56:41.386 INFO:teuthology.orchestra.run.smithi066.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-05-04T02:56:41.412 INFO:teuthology.orchestra.run.smithi066.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-04T02:56:41.437 INFO:teuthology.orchestra.run.smithi066.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-05-04T02:56:41.471 INFO:teuthology.orchestra.run.smithi066.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-04T02:56:41.530 INFO:teuthology.orchestra.run.smithi066.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-05-04T02:56:41.554 INFO:teuthology.orchestra.run.smithi066.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-05-04T02:56:41.580 INFO:teuthology.orchestra.run.smithi066.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-05-04T02:56:41.583 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-04T02:56:41.605 INFO:teuthology.orchestra.run.smithi066.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-04T02:56:41.630 INFO:teuthology.orchestra.run.smithi066.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-05-04T02:56:41.656 INFO:teuthology.orchestra.run.smithi066.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-04T02:56:41.681 INFO:teuthology.orchestra.run.smithi066.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-04T02:56:41.727 INFO:teuthology.orchestra.run.smithi066.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-05-04T02:56:41.748 INFO:teuthology.orchestra.run.smithi066.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-05-04T02:56:41.774 INFO:teuthology.orchestra.run.smithi066.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 852 kB/s | 79 kB 00:00 2024-05-04T02:56:41.799 INFO:teuthology.orchestra.run.smithi066.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 803 kB/s | 58 kB 00:00 2024-05-04T02:56:41.832 INFO:teuthology.orchestra.run.smithi039.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 626 kB/s | 689 kB 00:01 2024-05-04T02:56:41.833 INFO:teuthology.orchestra.run.smithi066.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-04T02:56:41.858 INFO:teuthology.orchestra.run.smithi066.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-05-04T02:56:41.866 INFO:teuthology.orchestra.run.smithi039.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 850 kB/s | 943 kB 00:01 2024-05-04T02:56:41.883 INFO:teuthology.orchestra.run.smithi066.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-05-04T02:56:41.908 INFO:teuthology.orchestra.run.smithi066.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-04T02:56:41.933 INFO:teuthology.orchestra.run.smithi066.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-04T02:56:41.959 INFO:teuthology.orchestra.run.smithi066.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-04T02:56:41.984 INFO:teuthology.orchestra.run.smithi066.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-05-04T02:56:42.033 INFO:teuthology.orchestra.run.smithi039.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 144 kB/s | 107 kB 00:00 2024-05-04T02:56:42.034 INFO:teuthology.orchestra.run.smithi066.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-05-04T02:56:42.261 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] python3-tempora-5.0.0-2.el9.noarch.rpm: Status code: 404 for http://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/p/python3-tempora-5.0.0-2.el9.noarch.rpm (IP: 173.208.208.139) 2024-05-04T02:56:42.296 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] python3-webob-1.8.7-6.el9.noarch.rpm: Status code: 404 for http://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/p/python3-webob-1.8.7-6.el9.noarch.rpm (IP: 173.208.208.139) 2024-05-04T02:56:42.375 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] python3-websocket-client-1.2.3-2.el9.noarch.rpm: Status code: 404 for http://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/p/python3-websocket-client-1.2.3-2.el9.noarch.rpm (IP: 173.208.208.139) 2024-05-04T02:56:42.393 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] python3-tempora-5.0.0-2.el9.noarch.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/p/python3-tempora-5.0.0-2.el9.noarch.rpm [SSL certificate problem: certificate has expired] 2024-05-04T02:56:42.424 INFO:teuthology.orchestra.run.smithi039.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 36 kB/s | 21 kB 00:00 2024-05-04T02:56:42.426 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] python3-webob-1.8.7-6.el9.noarch.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/p/python3-webob-1.8.7-6.el9.noarch.rpm [SSL certificate problem: certificate has expired] 2024-05-04T02:56:42.476 INFO:teuthology.orchestra.run.smithi066.stdout:(142/152): python3-tempora-5.0.0-2.el9.noarch.r 66 kB/s | 36 kB 00:00 2024-05-04T02:56:42.509 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] python3-websocket-client-1.2.3-2.el9.noarch.rpm: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/p/python3-websocket-client-1.2.3-2.el9.noarch.rpm [SSL certificate problem: certificate has expired] 2024-05-04T02:56:42.585 INFO:teuthology.orchestra.run.smithi066.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 382 kB/s | 230 kB 00:00 2024-05-04T02:56:42.610 INFO:teuthology.orchestra.run.smithi066.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 156 kB/s | 90 kB 00:00 2024-05-04T02:56:42.635 INFO:teuthology.orchestra.run.smithi066.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-05-04T02:56:42.649 INFO:teuthology.orchestra.run.smithi039.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 53 kB/s | 33 kB 00:00 2024-05-04T02:56:42.775 INFO:teuthology.orchestra.run.smithi039.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 345 kB/s | 313 kB 00:00 2024-05-04T02:56:42.966 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] thrift-0.15.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/t/thrift-0.15.0-2.el9.x86_64.rpm (IP: 173.208.208.139) 2024-05-04T02:56:43.192 INFO:teuthology.orchestra.run.smithi039.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-05-04T02:56:43.325 INFO:teuthology.orchestra.run.smithi039.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 20 kB/s | 11 kB 00:00 2024-05-04T02:56:43.336 INFO:teuthology.orchestra.run.smithi066.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-05-04T02:56:43.353 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] re2-20211101-3.el9.x86_64.rpm: Status code: 404 for http://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/r/re2-20211101-3.el9.x86_64.rpm (IP: 173.208.208.139) 2024-05-04T02:56:43.436 INFO:teuthology.orchestra.run.smithi066.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 232 kB/s | 192 kB 00:00 2024-05-04T02:56:43.576 INFO:teuthology.orchestra.run.smithi066.stdout:[MIRROR] python3-zc-lockfile-2.0-10.el9.noarch.rpm: Status code: 404 for http://mirror-mci.yuki.net.uk/fedora-epel/9/Everything/x86_64/Packages/p/python3-zc-lockfile-2.0-10.el9.noarch.rpm (IP: 173.208.208.139) 2024-05-04T02:56:43.611 INFO:teuthology.orchestra.run.smithi066.stdout:(148/152): python3-zc-lockfile-2.0-10.el9.noarc 19 kB/s | 20 kB 00:01 2024-05-04T02:56:43.634 INFO:teuthology.orchestra.run.smithi039.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-05-04T02:56:43.662 INFO:teuthology.orchestra.run.smithi066.stdout:(149/152): librados2-19.0.0-3522.gbd4e00a3.el9. 10 MB/s | 3.3 MB 00:00 2024-05-04T02:56:43.701 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T02:56:43.701 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T02:56:43.711 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-04T02:56:43.712 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-04T02:56:43.713 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-04T02:56:43.714 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-04T02:56:43.715 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-04T02:56:43.717 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-04T02:56:43.718 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-04T02:56:43.779 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-05-04T02:56:43.788 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-04T02:56:43.788 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-05-04T02:56:43.789 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-04T02:56:43.789 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-05-04T02:56:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-04T02:56:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-04T02:56:43.789 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-04T02:56:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-04T02:56:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-04T02:56:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-04T02:56:43.790 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-04T02:56:43.793 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-04T02:56:43.794 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-04T02:56:43.794 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-04T02:56:43.795 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-04T02:56:43.795 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-04T02:56:43.795 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-04T02:56:43.795 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-04T02:56:43.796 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-04T02:56:43.796 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-04T02:56:43.796 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-04T02:56:43.796 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-04T02:56:43.796 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-04T02:56:43.796 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-04T02:56:43.797 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-04T02:56:43.798 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-04T02:56:43.799 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-04T02:56:43.800 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-04T02:56:43.801 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-04T02:56:43.802 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-04T02:56:43.803 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-04T02:56:43.804 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-04T02:56:43.805 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-04T02:56:43.806 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:56:43.807 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-05-04T02:56:43.807 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-04T02:56:43.807 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-05-04T02:56:43.807 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 4 Packages 2024-05-04T02:56:43.807 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:56:43.807 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 214 M 2024-05-04T02:56:43.808 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-05-04T02:56:43.870 INFO:teuthology.orchestra.run.smithi066.stdout:(150/152): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-05-04T02:56:43.979 INFO:teuthology.orchestra.run.smithi066.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 3.7 MB/s | 1.3 MB 00:00 2024-05-04T02:56:43.992 INFO:teuthology.orchestra.run.smithi039.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-05-04T02:56:44.018 INFO:teuthology.orchestra.run.smithi039.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 194 kB/s | 160 kB 00:00 2024-05-04T02:56:44.037 INFO:teuthology.orchestra.run.smithi066.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.4 MB/s | 2.4 MB 00:00 2024-05-04T02:56:44.042 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-05-04T02:56:44.042 INFO:teuthology.orchestra.run.smithi066.stdout:Total 15 MB/s | 214 MB 00:13 2024-05-04T02:56:44.551 INFO:teuthology.orchestra.run.smithi167.stdout:(1/152): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-05-04T02:56:44.626 INFO:teuthology.orchestra.run.smithi039.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 668 kB/s | 662 kB 00:00 2024-05-04T02:56:44.652 INFO:teuthology.orchestra.run.smithi039.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 40 kB/s | 25 kB 00:00 2024-05-04T02:56:44.751 INFO:teuthology.orchestra.run.smithi167.stdout:(2/152): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 4.2 MB/s | 863 kB 00:00 2024-05-04T02:56:44.810 INFO:teuthology.orchestra.run.smithi167.stdout:(3/152): ceph-immutable-object-cache-19.0.0-352 2.5 MB/s | 147 kB 00:00 2024-05-04T02:56:44.943 INFO:teuthology.orchestra.run.smithi039.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 613 kB/s | 582 kB 00:00 2024-05-04T02:56:45.118 INFO:teuthology.orchestra.run.smithi167.stdout:(4/152): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 7.3 MB/s | 2.3 MB 00:00 2024-05-04T02:56:45.239 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-05-04T02:56:45.286 INFO:teuthology.orchestra.run.smithi039.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-05-04T02:56:45.327 INFO:teuthology.orchestra.run.smithi167.stdout:(5/152): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-05-04T02:56:45.398 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-05-04T02:56:45.398 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-05-04T02:56:45.511 INFO:teuthology.orchestra.run.smithi039.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 340 kB/s | 292 kB 00:00 2024-05-04T02:56:45.602 INFO:teuthology.orchestra.run.smithi167.stdout:(6/152): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 4.4 MB/s | 5.4 MB 00:01 2024-05-04T02:56:45.794 INFO:teuthology.orchestra.run.smithi039.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 222 kB/s | 188 kB 00:00 2024-05-04T02:56:45.919 INFO:teuthology.orchestra.run.smithi039.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 56 kB/s | 35 kB 00:00 2024-05-04T02:56:45.936 INFO:teuthology.orchestra.run.smithi167.stdout:(7/152): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 8.0 MB/s | 4.9 MB 00:00 2024-05-04T02:56:46.553 INFO:teuthology.orchestra.run.smithi167.stdout:(8/152): ceph-common-19.0.0-3522.gbd4e00a3.el9. 9.0 MB/s | 20 MB 00:02 2024-05-04T02:56:46.695 INFO:teuthology.orchestra.run.smithi167.stdout:(9/152): ceph-radosgw-19.0.0-3522.gbd4e00a3.el9 13 MB/s | 10 MB 00:00 2024-05-04T02:56:46.720 INFO:teuthology.orchestra.run.smithi167.stdout:(10/152): ceph-selinux-19.0.0-3522.gbd4e00a3.el 151 kB/s | 25 kB 00:00 2024-05-04T02:56:46.728 INFO:teuthology.orchestra.run.smithi039.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.9 MB/s | 4.7 MB 00:01 2024-05-04T02:56:46.812 INFO:teuthology.orchestra.run.smithi167.stdout:(11/152): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 2.4 MB/s | 224 kB 00:00 2024-05-04T02:56:46.887 INFO:teuthology.orchestra.run.smithi167.stdout:(12/152): cephfs-shell-19.0.0-3522.gbd4e00a3.el 361 kB/s | 27 kB 00:00 2024-05-04T02:56:46.962 INFO:teuthology.orchestra.run.smithi167.stdout:(13/152): libcephfs-devel-19.0.0-3522.gbd4e00a3 423 kB/s | 32 kB 00:00 2024-05-04T02:56:47.045 INFO:teuthology.orchestra.run.smithi039.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-04T02:56:47.080 INFO:teuthology.orchestra.run.smithi167.stdout:(14/152): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 6.3 MB/s | 756 kB 00:00 2024-05-04T02:56:47.163 INFO:teuthology.orchestra.run.smithi167.stdout:(15/152): libcephsqlite-19.0.0-3522.gbd4e00a3.e 1.9 MB/s | 158 kB 00:00 2024-05-04T02:56:47.232 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-05-04T02:56:47.233 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-05-04T02:56:47.247 INFO:teuthology.orchestra.run.smithi167.stdout:(16/152): librados-devel-19.0.0-3522.gbd4e00a3. 1.5 MB/s | 127 kB 00:00 2024-05-04T02:56:47.262 INFO:teuthology.orchestra.run.smithi039.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2024-05-04T02:56:47.423 INFO:teuthology.orchestra.run.smithi167.stdout:(17/152): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x8 9.7 MB/s | 18 MB 00:01 2024-05-04T02:56:47.456 INFO:teuthology.orchestra.run.smithi167.stdout:(18/152): libradosstriper1-19.0.0-3522.gbd4e00a 2.3 MB/s | 497 kB 00:00 2024-05-04T02:56:47.532 INFO:teuthology.orchestra.run.smithi167.stdout:(19/152): python3-ceph-argparse-19.0.0-3522.gbd 601 kB/s | 45 kB 00:00 2024-05-04T02:56:47.615 INFO:teuthology.orchestra.run.smithi167.stdout:(20/152): python3-ceph-common-19.0.0-3522.gbd4e 1.6 MB/s | 135 kB 00:00 2024-05-04T02:56:47.707 INFO:teuthology.orchestra.run.smithi167.stdout:(21/152): python3-cephfs-19.0.0-3522.gbd4e00a3. 1.7 MB/s | 162 kB 00:00 2024-05-04T02:56:47.754 INFO:teuthology.orchestra.run.smithi039.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 889 kB/s | 911 kB 00:01 2024-05-04T02:56:47.799 INFO:teuthology.orchestra.run.smithi167.stdout:(22/152): python3-rados-19.0.0-3522.gbd4e00a3.e 3.4 MB/s | 321 kB 00:00 2024-05-04T02:56:47.821 INFO:teuthology.orchestra.run.smithi039.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 172 kB/s | 133 kB 00:00 2024-05-04T02:56:47.899 INFO:teuthology.orchestra.run.smithi167.stdout:(23/152): python3-rbd-19.0.0-3522.gbd4e00a3.el9 2.9 MB/s | 298 kB 00:00 2024-05-04T02:56:47.966 INFO:teuthology.orchestra.run.smithi167.stdout:(24/152): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 9.5 MB/s | 5.2 MB 00:00 2024-05-04T02:56:47.992 INFO:teuthology.orchestra.run.smithi167.stdout:(25/152): python3-rgw-19.0.0-3522.gbd4e00a3.el9 1.1 MB/s | 100 kB 00:00 2024-05-04T02:56:48.042 INFO:teuthology.orchestra.run.smithi167.stdout:(26/152): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-04T02:56:48.151 INFO:teuthology.orchestra.run.smithi167.stdout:(27/152): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 1.5 MB/s | 170 kB 00:00 2024-05-04T02:56:48.259 INFO:teuthology.orchestra.run.smithi167.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3522.g 316 kB/s | 34 kB 00:00 2024-05-04T02:56:48.318 INFO:teuthology.orchestra.run.smithi167.stdout:(29/152): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 9.3 MB/s | 3.0 MB 00:00 2024-05-04T02:56:48.343 INFO:teuthology.orchestra.run.smithi167.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 1.7 MB/s | 142 kB 00:00 2024-05-04T02:56:48.613 INFO:teuthology.orchestra.run.smithi039.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 4.3 MB/s | 5.8 MB 00:01 2024-05-04T02:56:48.638 INFO:teuthology.orchestra.run.smithi039.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 107 kB/s | 87 kB 00:00 2024-05-04T02:56:48.718 INFO:teuthology.orchestra.run.smithi167.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 8.8 MB/s | 3.5 MB 00:00 2024-05-04T02:56:48.780 INFO:teuthology.orchestra.run.smithi039.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-05-04T02:56:48.802 INFO:teuthology.orchestra.run.smithi167.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3522.gbd 3.2 MB/s | 274 kB 00:00 2024-05-04T02:56:48.861 INFO:teuthology.orchestra.run.smithi167.stdout:(33/152): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 848 kB/s | 49 kB 00:00 2024-05-04T02:56:48.911 INFO:teuthology.orchestra.run.smithi167.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3522.gb 330 kB/s | 16 kB 00:00 2024-05-04T02:56:48.986 INFO:teuthology.orchestra.run.smithi167.stdout:(35/152): ceph-volume-19.0.0-3522.gbd4e00a3.el9 3.6 MB/s | 273 kB 00:00 2024-05-04T02:56:49.128 INFO:teuthology.orchestra.run.smithi167.stdout:(36/152): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 5.3 MB/s | 763 kB 00:00 2024-05-04T02:56:49.187 INFO:teuthology.orchestra.run.smithi167.stdout:(37/152): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 313 kB/s | 18 kB 00:00 2024-05-04T02:56:49.522 INFO:teuthology.orchestra.run.smithi039.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 276 kB/s | 244 kB 00:00 2024-05-04T02:56:49.680 INFO:teuthology.orchestra.run.smithi039.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 275 kB/s | 248 kB 00:00 2024-05-04T02:56:49.687 INFO:teuthology.orchestra.run.smithi167.stdout:(38/152): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 14 MB/s | 43 MB 00:02 2024-05-04T02:56:49.713 INFO:teuthology.orchestra.run.smithi167.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 153 kB/s | 80 kB 00:00 2024-05-04T02:56:49.804 INFO:teuthology.orchestra.run.smithi167.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-05-04T02:56:49.906 INFO:teuthology.orchestra.run.smithi039.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 4.6 MB/s | 6.0 MB 00:01 2024-05-04T02:56:49.922 INFO:teuthology.orchestra.run.smithi167.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-05-04T02:56:50.014 INFO:teuthology.orchestra.run.smithi167.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.2 MB 00:00 2024-05-04T02:56:50.039 INFO:teuthology.orchestra.run.smithi167.stdout:(43/152): libquadmath-11.4.1-3.el9.x86_64.rpm 599 kB/s | 195 kB 00:00 2024-05-04T02:56:50.064 INFO:teuthology.orchestra.run.smithi167.stdout:(44/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 977 kB/s | 253 kB 00:00 2024-05-04T02:56:50.089 INFO:teuthology.orchestra.run.smithi167.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-04T02:56:50.115 INFO:teuthology.orchestra.run.smithi167.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-04T02:56:50.140 INFO:teuthology.orchestra.run.smithi167.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-05-04T02:56:50.165 INFO:teuthology.orchestra.run.smithi167.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-04T02:56:50.197 INFO:teuthology.orchestra.run.smithi039.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 38 kB/s | 26 kB 00:00 2024-05-04T02:56:50.299 INFO:teuthology.orchestra.run.smithi167.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-05-04T02:56:50.324 INFO:teuthology.orchestra.run.smithi167.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 673 kB/s | 107 kB 00:00 2024-05-04T02:56:50.349 INFO:teuthology.orchestra.run.smithi167.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-05-04T02:56:50.400 INFO:teuthology.orchestra.run.smithi167.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-04T02:56:50.425 INFO:teuthology.orchestra.run.smithi167.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-05-04T02:56:50.459 INFO:teuthology.orchestra.run.smithi167.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-05-04T02:56:50.484 INFO:teuthology.orchestra.run.smithi167.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-05-04T02:56:50.490 INFO:teuthology.orchestra.run.smithi039.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 24 kB/s | 19 kB 00:00 2024-05-04T02:56:50.509 INFO:teuthology.orchestra.run.smithi167.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-04T02:56:50.535 INFO:teuthology.orchestra.run.smithi167.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-04T02:56:50.568 INFO:teuthology.orchestra.run.smithi167.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-05-04T02:56:50.602 INFO:teuthology.orchestra.run.smithi167.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-05-04T02:56:50.636 INFO:teuthology.orchestra.run.smithi167.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-04T02:56:50.661 INFO:teuthology.orchestra.run.smithi167.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-05-04T02:56:50.686 INFO:teuthology.orchestra.run.smithi167.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-04T02:56:50.711 INFO:teuthology.orchestra.run.smithi167.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-04T02:56:50.737 INFO:teuthology.orchestra.run.smithi167.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-04T02:56:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 839 kB/s | 35 kB 00:00 2024-05-04T02:56:50.804 INFO:teuthology.orchestra.run.smithi167.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-05-04T02:56:50.906 INFO:teuthology.orchestra.run.smithi039.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 172 kB/s | 172 kB 00:00 2024-05-04T02:56:50.938 INFO:teuthology.orchestra.run.smithi167.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-05-04T02:56:51.005 INFO:teuthology.orchestra.run.smithi167.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-05-04T02:56:51.015 INFO:teuthology.orchestra.run.smithi039.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 42 kB/s | 35 kB 00:00 2024-05-04T02:56:51.064 INFO:teuthology.orchestra.run.smithi167.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-04T02:56:51.105 INFO:teuthology.orchestra.run.smithi167.stdout:(70/152): postgresql-13.14-1.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-05-04T02:56:51.131 INFO:teuthology.orchestra.run.smithi167.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-04T02:56:51.181 INFO:teuthology.orchestra.run.smithi167.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-04T02:56:51.223 INFO:teuthology.orchestra.run.smithi167.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-05-04T02:56:51.298 INFO:teuthology.orchestra.run.smithi167.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-05-04T02:56:51.323 INFO:teuthology.orchestra.run.smithi167.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-04T02:56:51.349 INFO:teuthology.orchestra.run.smithi167.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-04T02:56:51.374 INFO:teuthology.orchestra.run.smithi167.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-04T02:56:51.399 INFO:teuthology.orchestra.run.smithi167.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-04T02:56:51.425 INFO:teuthology.orchestra.run.smithi167.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-04T02:56:51.450 INFO:teuthology.orchestra.run.smithi167.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-05-04T02:56:51.525 INFO:teuthology.orchestra.run.smithi167.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-04T02:56:51.601 INFO:teuthology.orchestra.run.smithi167.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-04T02:56:51.626 INFO:teuthology.orchestra.run.smithi167.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-04T02:56:51.651 INFO:teuthology.orchestra.run.smithi167.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-04T02:56:51.676 INFO:teuthology.orchestra.run.smithi167.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-05-04T02:56:51.702 INFO:teuthology.orchestra.run.smithi167.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-04T02:56:51.727 INFO:teuthology.orchestra.run.smithi167.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-04T02:56:51.786 INFO:teuthology.orchestra.run.smithi167.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-05-04T02:56:51.811 INFO:teuthology.orchestra.run.smithi167.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-05-04T02:56:51.840 INFO:teuthology.orchestra.run.smithi039.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.8 MB/s | 5.1 MB 00:01 2024-05-04T02:56:51.844 INFO:teuthology.orchestra.run.smithi167.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-05-04T02:56:51.865 INFO:teuthology.orchestra.run.smithi039.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 261 kB/s | 250 kB 00:00 2024-05-04T02:56:51.870 INFO:teuthology.orchestra.run.smithi167.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 640 kB/s | 54 kB 00:00 2024-05-04T02:56:51.895 INFO:teuthology.orchestra.run.smithi167.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-05-04T02:56:51.915 INFO:teuthology.orchestra.run.smithi039.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 247 kB/s | 222 kB 00:00 2024-05-04T02:56:51.920 INFO:teuthology.orchestra.run.smithi167.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 876 kB/s | 44 kB 00:00 2024-05-04T02:56:51.946 INFO:teuthology.orchestra.run.smithi167.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-04T02:56:51.971 INFO:teuthology.orchestra.run.smithi167.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-05-04T02:56:51.997 INFO:teuthology.orchestra.run.smithi167.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 937 kB/s | 47 kB 00:00 2024-05-04T02:56:52.022 INFO:teuthology.orchestra.run.smithi167.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-04T02:56:52.039 INFO:teuthology.orchestra.run.smithi167.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-05-04T02:56:52.073 INFO:teuthology.orchestra.run.smithi167.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-04T02:56:52.089 INFO:teuthology.orchestra.run.smithi167.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-04T02:56:52.115 INFO:teuthology.orchestra.run.smithi167.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-05-04T02:56:52.132 INFO:teuthology.orchestra.run.smithi167.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 529 kB/s | 22 kB 00:00 2024-05-04T02:56:52.186 INFO:teuthology.orchestra.run.smithi167.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-05-04T02:56:52.441 INFO:teuthology.orchestra.run.smithi167.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-04T02:56:52.524 INFO:teuthology.orchestra.run.smithi039.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 64 kB/s | 42 kB 00:00 2024-05-04T02:56:52.566 INFO:teuthology.orchestra.run.smithi039.stdout:(86/152): python3-packaging-20.9-5.el9.noarch.r 106 kB/s | 77 kB 00:00 2024-05-04T02:56:52.574 INFO:teuthology.orchestra.run.smithi167.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-05-04T02:56:52.625 INFO:teuthology.orchestra.run.smithi167.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 742 kB/s | 309 kB 00:00 2024-05-04T02:56:52.666 INFO:teuthology.orchestra.run.smithi167.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-04T02:56:52.692 INFO:teuthology.orchestra.run.smithi167.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-05-04T02:56:52.717 INFO:teuthology.orchestra.run.smithi167.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 964 kB/s | 49 kB 00:00 2024-05-04T02:56:52.751 INFO:teuthology.orchestra.run.smithi167.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-04T02:56:52.766 INFO:teuthology.orchestra.run.smithi039.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 317 kB/s | 269 kB 00:00 2024-05-04T02:56:52.809 INFO:teuthology.orchestra.run.smithi167.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-04T02:56:53.035 INFO:teuthology.orchestra.run.smithi167.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-05-04T02:56:53.077 INFO:teuthology.orchestra.run.smithi167.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-04T02:56:53.119 INFO:teuthology.orchestra.run.smithi167.stdout:(114/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.2 MB/s | 839 kB 00:00 2024-05-04T02:56:53.144 INFO:teuthology.orchestra.run.smithi167.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-05-04T02:56:53.169 INFO:teuthology.orchestra.run.smithi167.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-05-04T02:56:53.253 INFO:teuthology.orchestra.run.smithi167.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-04T02:56:53.341 INFO:teuthology.orchestra.run.smithi039.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 194 kB/s | 159 kB 00:00 2024-05-04T02:56:53.353 INFO:teuthology.orchestra.run.smithi167.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-05-04T02:56:53.395 INFO:teuthology.orchestra.run.smithi167.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 853 kB/s | 35 kB 00:00 2024-05-04T02:56:53.417 INFO:teuthology.orchestra.run.smithi039.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 328 kB/s | 279 kB 00:00 2024-05-04T02:56:53.420 INFO:teuthology.orchestra.run.smithi167.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-05-04T02:56:53.467 INFO:teuthology.orchestra.run.smithi039.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 76 kB/s | 51 kB 00:00 2024-05-04T02:56:53.513 INFO:teuthology.orchestra.run.smithi167.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-04T02:56:53.538 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-05-04T02:56:53.579 INFO:teuthology.orchestra.run.smithi167.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-04T02:56:53.622 INFO:teuthology.orchestra.run.smithi167.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 254 kB/s | 11 kB 00:00 2024-05-04T02:56:53.625 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-04T02:56:53.663 INFO:teuthology.orchestra.run.smithi167.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 425 kB/s | 18 kB 00:00 2024-05-04T02:56:53.664 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-04T02:56:53.705 INFO:teuthology.orchestra.run.smithi167.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 556 kB/s | 23 kB 00:00 2024-05-04T02:56:53.718 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-04T02:56:53.748 INFO:teuthology.orchestra.run.smithi167.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 464 kB/s | 19 kB 00:00 2024-05-04T02:56:53.761 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-04T02:56:53.785 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-04T02:56:53.790 INFO:teuthology.orchestra.run.smithi167.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-05-04T02:56:53.829 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-04T02:56:54.009 INFO:teuthology.orchestra.run.smithi039.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 80 kB/s | 54 kB 00:00 2024-05-04T02:56:54.093 INFO:teuthology.orchestra.run.smithi039.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-05-04T02:56:54.148 INFO:teuthology.orchestra.run.smithi167.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-05-04T02:56:54.189 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-04T02:56:54.190 INFO:teuthology.orchestra.run.smithi167.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-04T02:56:54.232 INFO:teuthology.orchestra.run.smithi167.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-05-04T02:56:54.274 INFO:teuthology.orchestra.run.smithi167.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 943 kB/s | 79 kB 00:00 2024-05-04T02:56:54.299 INFO:teuthology.orchestra.run.smithi167.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-05-04T02:56:54.311 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:56:54.341 INFO:teuthology.orchestra.run.smithi167.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 395 kB/s | 16 kB 00:00 2024-05-04T02:56:54.400 INFO:teuthology.orchestra.run.smithi167.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-04T02:56:54.439 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:56:54.442 INFO:teuthology.orchestra.run.smithi167.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-05-04T02:56:54.459 INFO:teuthology.orchestra.run.smithi167.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-05-04T02:56:54.462 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:56:54.476 INFO:teuthology.orchestra.run.smithi167.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 914 kB/s | 31 kB 00:00 2024-05-04T02:56:54.546 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:56:54.559 INFO:teuthology.orchestra.run.smithi167.stdout:(138/152): libarrow-9.0.0-11.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:01 2024-05-04T02:56:54.575 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T02:56:54.585 INFO:teuthology.orchestra.run.smithi167.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-05-04T02:56:54.610 INFO:teuthology.orchestra.run.smithi167.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-04T02:56:54.660 INFO:teuthology.orchestra.run.smithi039.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 68 kB/s | 44 kB 00:00 2024-05-04T02:56:54.660 INFO:teuthology.orchestra.run.smithi167.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-05-04T02:56:54.686 INFO:teuthology.orchestra.run.smithi167.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 832 kB/s | 188 kB 00:00 2024-05-04T02:56:54.732 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-04T02:56:54.744 INFO:teuthology.orchestra.run.smithi167.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-05-04T02:56:54.761 INFO:teuthology.orchestra.run.smithi167.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-05-04T02:56:54.787 INFO:teuthology.orchestra.run.smithi167.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-05-04T02:56:54.789 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-04T02:56:54.816 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:56:54.912 INFO:teuthology.orchestra.run.smithi167.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-05-04T02:56:54.923 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:56:54.950 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T02:56:55.013 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T02:56:55.020 INFO:teuthology.orchestra.run.smithi167.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.2 MB/s | 427 kB 00:00 2024-05-04T02:56:55.035 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T02:56:55.056 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T02:56:55.059 INFO:teuthology.orchestra.run.smithi039.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.9 MB/s | 16 MB 00:01 2024-05-04T02:56:55.118 INFO:teuthology.orchestra.run.smithi039.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 215 kB/s | 221 kB 00:01 2024-05-04T02:56:55.213 INFO:teuthology.orchestra.run.smithi167.stdout:(148/152): librados2-19.0.0-3522.gbd4e00a3.el9. 11 MB/s | 3.3 MB 00:00 2024-05-04T02:56:55.255 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T02:56:55.284 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T02:56:55.396 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-04T02:56:55.443 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-04T02:56:55.471 INFO:teuthology.orchestra.run.smithi167.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-05-04T02:56:55.484 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/156 2024-05-04T02:56:55.510 INFO:teuthology.orchestra.run.smithi039.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 42 kB/s | 36 kB 00:00 2024-05-04T02:56:55.521 INFO:teuthology.orchestra.run.smithi167.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.4 MB/s | 1.3 MB 00:00 2024-05-04T02:56:55.561 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/156 2024-05-04T02:56:55.572 INFO:teuthology.orchestra.run.smithi167.stdout:(151/152): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-05-04T02:56:55.609 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/156 2024-05-04T02:56:55.639 INFO:teuthology.orchestra.run.smithi167.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-04T02:56:55.646 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-05-04T02:56:55.646 INFO:teuthology.orchestra.run.smithi167.stdout:Total 18 MB/s | 214 MB 00:11 2024-05-04T02:56:55.652 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/156 2024-05-04T02:56:55.685 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/156 2024-05-04T02:56:55.713 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/156 2024-05-04T02:56:55.752 INFO:teuthology.orchestra.run.smithi039.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 71 kB/s | 47 kB 00:00 2024-05-04T02:56:55.786 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/156 2024-05-04T02:56:55.845 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/156 2024-05-04T02:56:55.874 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/156 2024-05-04T02:56:55.909 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/156 2024-05-04T02:56:55.978 INFO:teuthology.orchestra.run.smithi039.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 355 kB/s | 305 kB 00:00 2024-05-04T02:56:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/156 2024-05-04T02:56:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 30/156 2024-05-04T02:56:56.064 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 31/156 2024-05-04T02:56:56.092 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 32/156 2024-05-04T02:56:56.165 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/156 2024-05-04T02:56:56.215 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/156 2024-05-04T02:56:56.220 INFO:teuthology.orchestra.run.smithi039.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 82 kB/s | 58 kB 00:00 2024-05-04T02:56:56.294 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/156 2024-05-04T02:56:56.439 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/156 2024-05-04T02:56:56.494 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/156 2024-05-04T02:56:56.536 INFO:teuthology.orchestra.run.smithi039.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 124 kB/s | 94 kB 00:00 2024-05-04T02:56:56.628 INFO:teuthology.orchestra.run.smithi039.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-04T02:56:56.679 INFO:teuthology.orchestra.run.smithi039.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 91 kB/s | 64 kB 00:00 2024-05-04T02:56:56.758 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-05-04T02:56:56.906 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-05-04T02:56:56.907 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-05-04T02:56:56.912 INFO:teuthology.orchestra.run.smithi039.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-05-04T02:56:56.954 INFO:teuthology.orchestra.run.smithi039.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-05-04T02:56:57.054 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/156 2024-05-04T02:56:57.142 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/156 2024-05-04T02:56:57.175 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/156 2024-05-04T02:56:57.218 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/156 2024-05-04T02:56:57.256 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 42/156 2024-05-04T02:56:57.298 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/156 2024-05-04T02:56:57.312 INFO:teuthology.orchestra.run.smithi039.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 54 kB/s | 19 kB 00:00 2024-05-04T02:56:57.430 INFO:teuthology.orchestra.run.smithi039.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 599 kB/s | 309 kB 00:00 2024-05-04T02:56:57.457 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/156 2024-05-04T02:56:57.500 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 45/156 2024-05-04T02:56:57.521 INFO:teuthology.orchestra.run.smithi039.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 651 kB/s | 548 kB 00:00 2024-05-04T02:56:57.596 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 46/156 2024-05-04T02:56:57.597 INFO:teuthology.orchestra.run.smithi039.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-05-04T02:56:57.666 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 47/156 2024-05-04T02:56:57.739 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-04T02:56:57.776 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-04T02:56:57.813 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-04T02:56:57.847 INFO:teuthology.orchestra.run.smithi039.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 150 kB/s | 49 kB 00:00 2024-05-04T02:56:57.861 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-04T02:56:57.895 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-04T02:56:57.939 INFO:teuthology.orchestra.run.smithi039.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-05-04T02:56:57.940 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-04T02:56:57.956 INFO:teuthology.orchestra.run.smithi039.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 188 kB/s | 67 kB 00:00 2024-05-04T02:56:57.981 INFO:teuthology.orchestra.run.smithi039.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-04T02:56:57.981 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-04T02:56:58.073 INFO:teuthology.orchestra.run.smithi039.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-04T02:56:58.098 INFO:teuthology.orchestra.run.smithi039.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-05-04T02:56:58.123 INFO:teuthology.orchestra.run.smithi039.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 306 kB/s | 43 kB 00:00 2024-05-04T02:56:58.126 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-04T02:56:58.178 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-04T02:56:58.182 INFO:teuthology.orchestra.run.smithi039.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 300 kB/s | 32 kB 00:00 2024-05-04T02:56:58.208 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-04T02:56:58.250 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-04T02:56:58.291 INFO:teuthology.orchestra.run.smithi039.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 90 kB/s | 15 kB 00:00 2024-05-04T02:56:58.316 INFO:teuthology.orchestra.run.smithi039.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 894 kB/s | 172 kB 00:00 2024-05-04T02:56:58.341 INFO:teuthology.orchestra.run.smithi039.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-05-04T02:56:58.349 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-04T02:56:58.382 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-04T02:56:58.408 INFO:teuthology.orchestra.run.smithi039.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.6 MB/s | 303 kB 00:00 2024-05-04T02:56:58.422 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-04T02:56:58.448 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-04T02:56:58.450 INFO:teuthology.orchestra.run.smithi039.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-05-04T02:56:58.467 INFO:teuthology.orchestra.run.smithi039.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-05-04T02:56:58.472 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-04T02:56:58.551 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-04T02:56:58.567 INFO:teuthology.orchestra.run.smithi039.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-05-04T02:56:58.609 INFO:teuthology.orchestra.run.smithi039.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 912 kB/s | 144 kB 00:00 2024-05-04T02:56:58.634 INFO:teuthology.orchestra.run.smithi039.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-05-04T02:56:58.668 INFO:teuthology.orchestra.run.smithi039.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-04T02:56:58.687 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-04T02:56:58.743 INFO:teuthology.orchestra.run.smithi039.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-05-04T02:56:58.768 INFO:teuthology.orchestra.run.smithi039.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2024-05-04T02:56:58.794 INFO:teuthology.orchestra.run.smithi039.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-05-04T02:56:58.838 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-05-04T02:56:58.839 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-05-04T02:56:58.886 INFO:teuthology.orchestra.run.smithi039.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-04T02:56:58.911 INFO:teuthology.orchestra.run.smithi039.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 325 kB/s | 46 kB 00:00 2024-05-04T02:56:58.936 INFO:teuthology.orchestra.run.smithi039.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 554 kB/s | 79 kB 00:00 2024-05-04T02:56:58.960 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-04T02:56:58.995 INFO:teuthology.orchestra.run.smithi039.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 532 kB/s | 58 kB 00:00 2024-05-04T02:56:59.063 INFO:teuthology.orchestra.run.smithi039.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-04T02:56:59.088 INFO:teuthology.orchestra.run.smithi039.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-05-04T02:56:59.112 INFO:teuthology.orchestra.run.smithi039.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 768 kB/s | 90 kB 00:00 2024-05-04T02:56:59.179 INFO:teuthology.orchestra.run.smithi039.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-05-04T02:56:59.202 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-04T02:56:59.213 INFO:teuthology.orchestra.run.smithi039.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-05-04T02:56:59.236 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-04T02:56:59.255 INFO:teuthology.orchestra.run.smithi039.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-05-04T02:56:59.284 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-04T02:56:59.288 INFO:teuthology.orchestra.run.smithi039.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-05-04T02:56:59.314 INFO:teuthology.orchestra.run.smithi039.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-05-04T02:56:59.324 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-04T02:56:59.352 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-04T02:56:59.364 INFO:teuthology.orchestra.run.smithi039.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-05-04T02:56:59.387 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-04T02:56:59.397 INFO:teuthology.orchestra.run.smithi039.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-04T02:56:59.420 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-04T02:56:59.423 INFO:teuthology.orchestra.run.smithi039.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-05-04T02:56:59.464 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-04T02:56:59.490 INFO:teuthology.orchestra.run.smithi039.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-04T02:56:59.496 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-04T02:56:59.592 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-04T02:56:59.632 INFO:teuthology.orchestra.run.smithi039.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 85 kB/s | 20 kB 00:00 2024-05-04T02:56:59.653 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-04T02:56:59.657 INFO:teuthology.orchestra.run.smithi039.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 820 kB/s | 192 kB 00:00 2024-05-04T02:56:59.691 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-04T02:56:59.699 INFO:teuthology.orchestra.run.smithi039.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-05-04T02:56:59.740 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-04T02:56:59.851 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-04T02:56:59.901 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-04T02:56:59.958 INFO:teuthology.orchestra.run.smithi039.stdout:(149/152): librados2-19.0.0-3522.gbd4e00a3.el9. 10 MB/s | 3.3 MB 00:00 2024-05-04T02:57:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.3 MB/s | 1.3 MB 00:00 2024-05-04T02:57:00.549 INFO:teuthology.orchestra.run.smithi039.stdout:(151/152): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-05-04T02:57:00.667 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-04T02:57:00.714 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-04T02:57:00.756 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-04T02:57:00.791 INFO:teuthology.orchestra.run.smithi039.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.9 MB/s | 2.4 MB 00:00 2024-05-04T02:57:00.796 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-05-04T02:57:00.796 INFO:teuthology.orchestra.run.smithi039.stdout:Total 7.0 MB/s | 214 MB 00:30 2024-05-04T02:57:00.859 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-04T02:57:00.915 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-04T02:57:01.005 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-04T02:57:01.041 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-04T02:57:01.226 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-04T02:57:01.255 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-04T02:57:01.335 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-04T02:57:01.892 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-05-04T02:57:02.020 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-04T02:57:02.050 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-05-04T02:57:02.050 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-05-04T02:57:02.141 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-04T02:57:03.444 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-04T02:57:03.517 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-04T02:57:03.546 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-04T02:57:03.578 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-04T02:57:03.890 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-05-04T02:57:03.891 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-05-04T02:57:03.913 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-04T02:57:03.958 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T02:57:04.049 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T02:57:04.079 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 100/156 2024-05-04T02:57:04.116 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-04T02:57:04.633 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-04T02:57:04.661 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T02:57:04.732 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T02:57:04.762 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 104/156 2024-05-04T02:57:04.764 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-05-04T02:57:04.893 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-04T02:57:05.100 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-04T02:57:05.153 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-04T02:57:05.194 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-04T02:57:05.220 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-04T02:57:05.263 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-04T02:57:05.609 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-04T02:57:05.738 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:57:05.864 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:57:05.897 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:57:05.981 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:57:06.018 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T02:57:06.169 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-04T02:57:06.215 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-04T02:57:06.243 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:57:06.347 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:57:06.369 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T02:57:06.432 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T02:57:06.453 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T02:57:06.474 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T02:57:06.653 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T02:57:06.687 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T02:57:06.796 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-04T02:57:06.826 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-04T02:57:06.861 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/156 2024-05-04T02:57:06.925 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/156 2024-05-04T02:57:06.961 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/156 2024-05-04T02:57:07.004 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/156 2024-05-04T02:57:07.037 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/156 2024-05-04T02:57:07.055 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:07.065 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/156 2024-05-04T02:57:07.092 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:07.133 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/156 2024-05-04T02:57:07.195 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/156 2024-05-04T02:57:07.235 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/156 2024-05-04T02:57:07.270 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/156 2024-05-04T02:57:07.345 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/156 2024-05-04T02:57:07.380 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 30/156 2024-05-04T02:57:07.424 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 31/156 2024-05-04T02:57:07.452 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 32/156 2024-05-04T02:57:07.523 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/156 2024-05-04T02:57:07.568 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/156 2024-05-04T02:57:07.642 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/156 2024-05-04T02:57:07.683 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:07.720 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T02:57:07.773 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/156 2024-05-04T02:57:07.810 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T02:57:07.810 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-04T02:57:07.810 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-04T02:57:07.810 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:07.847 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/156 2024-05-04T02:57:07.863 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T02:57:08.368 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/156 2024-05-04T02:57:08.458 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/156 2024-05-04T02:57:08.486 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/156 2024-05-04T02:57:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/156 2024-05-04T02:57:08.567 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 42/156 2024-05-04T02:57:08.608 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/156 2024-05-04T02:57:08.764 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/156 2024-05-04T02:57:08.837 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 45/156 2024-05-04T02:57:08.928 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 46/156 2024-05-04T02:57:08.992 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 47/156 2024-05-04T02:57:09.065 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-04T02:57:09.097 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-04T02:57:09.133 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-04T02:57:09.179 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-04T02:57:09.216 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-04T02:57:09.259 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-04T02:57:09.301 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-04T02:57:09.436 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-04T02:57:09.740 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-04T02:57:09.740 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-04T02:57:09.741 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-04T02:57:09.741 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-04T02:57:09.741 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-04T02:57:09.741 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-04T02:57:09.745 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-04T02:57:09.770 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-04T02:57:09.847 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-04T02:57:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-05-04T02:57:09.970 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-04T02:57:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-04T02:57:10.002 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-04T02:57:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-04T02:57:10.029 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-04T02:57:10.063 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-04T02:57:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-04T02:57:10.103 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-04T02:57:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-04T02:57:10.134 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-04T02:57:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-04T02:57:10.164 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-04T02:57:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-04T02:57:10.199 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-04T02:57:10.333 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-04T02:57:10.441 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-04T02:57:10.475 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-04T02:57:10.535 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-04T02:57:10.563 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-04T02:57:10.606 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-04T02:57:10.640 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:57:10.645 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-04T02:57:10.693 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-04T02:57:10.751 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:57:10.782 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:57:10.790 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-04T02:57:10.838 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-04T02:57:10.858 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:57:10.887 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T02:57:11.047 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-04T02:57:11.093 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-04T02:57:11.120 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:57:11.227 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:57:11.254 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T02:57:11.317 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T02:57:11.338 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T02:57:11.363 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T02:57:11.564 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T02:57:11.567 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-04T02:57:11.597 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T02:57:11.619 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-04T02:57:11.644 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-04T02:57:11.708 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-04T02:57:11.736 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-04T02:57:11.743 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-04T02:57:11.771 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/156 2024-05-04T02:57:11.802 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-04T02:57:11.847 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/156 2024-05-04T02:57:11.890 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-04T02:57:11.896 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/156 2024-05-04T02:57:11.921 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-04T02:57:11.939 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/156 2024-05-04T02:57:11.972 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/156 2024-05-04T02:57:11.999 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/156 2024-05-04T02:57:12.071 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/156 2024-05-04T02:57:12.104 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-04T02:57:12.122 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/156 2024-05-04T02:57:12.135 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-04T02:57:12.152 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/156 2024-05-04T02:57:12.187 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/156 2024-05-04T02:57:12.213 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-04T02:57:12.266 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/156 2024-05-04T02:57:12.298 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 30/156 2024-05-04T02:57:12.342 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 31/156 2024-05-04T02:57:12.370 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 32/156 2024-05-04T02:57:12.443 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/156 2024-05-04T02:57:12.485 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/156 2024-05-04T02:57:12.562 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/156 2024-05-04T02:57:12.708 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/156 2024-05-04T02:57:12.763 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/156 2024-05-04T02:57:12.851 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-04T02:57:12.967 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-04T02:57:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/156 2024-05-04T02:57:13.418 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/156 2024-05-04T02:57:13.453 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/156 2024-05-04T02:57:13.496 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/156 2024-05-04T02:57:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 42/156 2024-05-04T02:57:13.576 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/156 2024-05-04T02:57:13.728 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/156 2024-05-04T02:57:13.787 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 45/156 2024-05-04T02:57:13.881 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 46/156 2024-05-04T02:57:13.944 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 47/156 2024-05-04T02:57:14.018 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-04T02:57:14.055 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-04T02:57:14.091 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-04T02:57:14.139 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-04T02:57:14.174 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-04T02:57:14.182 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-04T02:57:14.217 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-04T02:57:14.257 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-04T02:57:14.259 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-04T02:57:14.285 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-04T02:57:14.318 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-04T02:57:14.403 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-04T02:57:14.448 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-04T02:57:14.486 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-04T02:57:14.528 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-04T02:57:14.626 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-04T02:57:14.638 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-04T02:57:14.661 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-04T02:57:14.683 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T02:57:14.701 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-04T02:57:14.726 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-04T02:57:14.751 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-04T02:57:14.774 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T02:57:14.805 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 100/156 2024-05-04T02:57:14.830 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-04T02:57:14.840 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-04T02:57:14.959 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-04T02:57:15.000 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-04T02:57:15.020 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-04T02:57:15.054 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-04T02:57:15.093 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-04T02:57:15.125 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-04T02:57:15.155 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-04T02:57:15.189 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-04T02:57:15.222 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-04T02:57:15.264 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-04T02:57:15.298 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-04T02:57:15.331 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-04T02:57:15.389 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-04T02:57:15.453 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T02:57:15.454 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-04T02:57:15.485 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-04T02:57:15.524 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T02:57:15.533 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-04T02:57:15.554 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 104/156 2024-05-04T02:57:15.644 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-04T02:57:15.696 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-04T02:57:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-04T02:57:16.516 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-04T02:57:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-04T02:57:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-04T02:57:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-04T02:57:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-04T02:57:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-04T02:57:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-04T02:57:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-04T02:57:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-04T02:57:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:17.743 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:17.834 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-04T02:57:17.951 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-04T02:57:18.302 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:18.354 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T02:57:18.443 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T02:57:18.443 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-04T02:57:18.443 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-04T02:57:18.443 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:18.472 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T02:57:19.252 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-04T02:57:19.336 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-04T02:57:19.364 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-04T02:57:19.397 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-04T02:57:19.731 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-04T02:57:19.776 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T02:57:19.868 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T02:57:19.869 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T02:57:19.869 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-05-04T02:57:19.869 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-05-04T02:57:19.869 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-05-04T02:57:19.869 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-05-04T02:57:19.869 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-05-04T02:57:19.869 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-05-04T02:57:19.870 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-05-04T02:57:19.870 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:19.898 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 100/156 2024-05-04T02:57:19.935 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-04T02:57:20.060 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T02:57:20.078 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T02:57:20.078 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:20.078 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-04T02:57:20.078 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:20.450 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-04T02:57:20.480 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T02:57:20.551 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T02:57:20.581 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 104/156 2024-05-04T02:57:21.721 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T02:57:21.749 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T02:57:21.749 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:21.749 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-04T02:57:21.749 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T02:57:21.749 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T02:57:21.749 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:21.989 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T02:57:22.013 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T02:57:22.014 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:22.014 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-04T02:57:22.014 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T02:57:22.014 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T02:57:22.014 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:22.506 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T02:57:22.530 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T02:57:22.530 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:22.530 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-04T02:57:22.530 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T02:57:22.531 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T02:57:22.531 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:22.637 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-04T02:57:22.696 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-04T02:57:22.759 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-04T02:57:22.919 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:22.952 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:22.955 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-04T02:57:23.008 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-04T02:57:23.088 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-04T02:57:23.187 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T02:57:23.329 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T02:57:23.382 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-04T02:57:23.414 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-04T02:57:23.442 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-04T02:57:23.539 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T02:57:23.581 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T02:57:23.680 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T02:57:23.680 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-04T02:57:23.680 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-04T02:57:23.681 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:23.707 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T02:57:23.911 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T02:57:23.940 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T02:57:23.990 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 124/156 2024-05-04T02:57:24.053 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 125/156 2024-05-04T02:57:24.079 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T02:57:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T02:57:25.216 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T02:57:26.233 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T02:57:26.257 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T02:57:26.375 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T02:57:26.550 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 129/156 2024-05-04T02:57:26.584 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T02:57:26.609 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T02:57:26.610 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:26.610 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-04T02:57:26.610 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T02:57:26.610 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T02:57:26.610 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:26.660 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T02:57:26.671 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T02:57:26.773 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/156 2024-05-04T02:57:26.826 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 133/156 2024-05-04T02:57:27.815 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-04T02:57:27.845 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T02:57:27.869 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T02:57:27.870 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:27.870 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-04T02:57:27.870 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T02:57:27.870 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T02:57:27.870 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:27.912 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T02:57:27.936 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T02:57:27.936 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:27.936 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-04T02:57:27.937 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:27.999 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T02:57:28.023 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T02:57:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-04T02:57:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T02:57:28.024 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T02:57:28.024 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:28.347 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T02:57:28.371 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T02:57:28.371 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:28.372 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-04T02:57:28.372 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T02:57:28.372 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T02:57:28.372 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:30.198 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T02:57:30.198 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-05-04T02:57:30.198 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-05-04T02:57:30.198 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-05-04T02:57:30.198 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-05-04T02:57:30.198 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-05-04T02:57:30.198 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-05-04T02:57:30.198 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-05-04T02:57:30.199 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:30.454 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T02:57:30.469 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T02:57:30.470 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:30.470 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-04T02:57:30.470 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:32.066 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T02:57:32.091 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T02:57:32.091 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:32.091 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-04T02:57:32.091 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T02:57:32.091 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T02:57:32.091 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:32.316 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T02:57:32.339 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T02:57:32.339 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:32.339 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-04T02:57:32.339 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T02:57:32.339 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T02:57:32.339 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:32.800 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T02:57:32.823 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T02:57:32.823 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:32.823 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-04T02:57:32.823 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T02:57:32.823 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T02:57:32.823 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:32.893 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/156 2024-05-04T02:57:32.931 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-04T02:57:32.935 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/156 2024-05-04T02:57:32.975 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/156 2024-05-04T02:57:32.979 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-04T02:57:33.007 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-04T02:57:33.034 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-04T02:57:33.034 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-04T02:57:33.220 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-04T02:57:33.228 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 144/156 2024-05-04T02:57:33.268 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-04T02:57:33.272 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-04T02:57:33.302 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 146/156 2024-05-04T02:57:33.345 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-04T02:57:33.434 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T02:57:33.574 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T02:57:33.618 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-04T02:57:33.650 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-04T02:57:33.677 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-04T02:57:33.888 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:33.926 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:33.947 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:34.155 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T02:57:34.184 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T02:57:34.225 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 124/156 2024-05-04T02:57:34.295 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 125/156 2024-05-04T02:57:34.324 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T02:57:34.860 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 148/156 2024-05-04T02:57:34.956 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-04T02:57:34.998 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-04T02:57:35.081 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-04T02:57:35.109 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-04T02:57:35.109 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T02:57:35.194 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T02:57:35.226 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T02:57:35.227 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T02:57:35.318 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T02:57:35.378 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T02:57:35.386 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:35.418 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T02:57:35.627 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T02:57:35.627 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-05-04T02:57:35.627 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-05-04T02:57:35.627 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-05-04T02:57:35.627 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-05-04T02:57:35.627 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-05-04T02:57:35.627 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /root 2024-05-04T02:57:35.627 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-05-04T02:57:35.628 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:35.822 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T02:57:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T02:57:35.839 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:35.839 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-04T02:57:35.839 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:36.389 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T02:57:36.418 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T02:57:36.530 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T02:57:36.694 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 129/156 2024-05-04T02:57:36.721 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T02:57:36.743 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T02:57:36.743 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:36.744 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-04T02:57:36.744 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T02:57:36.744 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T02:57:36.744 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:36.788 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T02:57:36.799 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T02:57:36.899 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/156 2024-05-04T02:57:36.945 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 133/156 2024-05-04T02:57:37.476 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T02:57:37.503 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T02:57:37.503 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:37.503 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-04T02:57:37.503 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T02:57:37.503 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T02:57:37.504 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:37.734 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T02:57:37.758 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T02:57:37.758 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:37.758 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-04T02:57:37.758 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T02:57:37.758 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T02:57:37.759 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:37.896 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-04T02:57:37.932 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T02:57:37.955 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T02:57:37.955 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:37.955 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-04T02:57:37.955 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T02:57:37.956 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T02:57:37.956 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:38.000 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T02:57:38.022 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T02:57:38.022 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:38.022 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-04T02:57:38.022 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:38.067 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T02:57:38.090 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T02:57:38.090 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:38.090 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-04T02:57:38.090 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T02:57:38.090 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T02:57:38.090 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:38.252 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T02:57:38.275 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T02:57:38.276 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:38.276 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-04T02:57:38.276 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T02:57:38.276 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T02:57:38.276 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:38.392 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-04T02:57:38.401 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T02:57:38.424 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T02:57:38.424 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:38.424 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-04T02:57:38.424 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T02:57:38.424 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T02:57:38.424 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:38.450 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-04T02:57:38.515 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-04T02:57:38.711 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-04T02:57:38.762 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-04T02:57:38.844 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-04T02:57:38.942 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T02:57:39.086 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T02:57:39.129 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-04T02:57:39.153 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-04T02:57:39.180 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-04T02:57:39.543 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T02:57:39.571 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T02:57:39.603 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 124/156 2024-05-04T02:57:39.667 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 125/156 2024-05-04T02:57:39.694 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T02:57:40.792 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T02:57:40.829 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T02:57:41.247 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:41.247 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/156 2024-05-04T02:57:41.247 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/156 2024-05-04T02:57:41.247 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/156 2024-05-04T02:57:41.247 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/156 2024-05-04T02:57:41.248 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/156 2024-05-04T02:57:41.249 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/156 2024-05-04T02:57:41.250 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/156 2024-05-04T02:57:41.250 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/156 2024-05-04T02:57:41.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/156 2024-05-04T02:57:41.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/156 2024-05-04T02:57:41.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/156 2024-05-04T02:57:41.261 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-04T02:57:41.262 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-04T02:57:41.263 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-04T02:57:41.264 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-04T02:57:41.265 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-04T02:57:41.266 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T02:57:41.267 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-04T02:57:41.268 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-04T02:57:41.269 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/156 2024-05-04T02:57:41.270 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-04T02:57:41.270 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/156 2024-05-04T02:57:41.270 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-04T02:57:41.270 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-04T02:57:41.270 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T02:57:41.270 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-04T02:57:41.844 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T02:57:41.872 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T02:57:41.989 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T02:57:42.170 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 129/156 2024-05-04T02:57:42.207 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T02:57:42.231 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T02:57:42.231 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:42.231 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-04T02:57:42.232 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T02:57:42.232 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T02:57:42.232 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:42.283 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T02:57:42.294 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T02:57:42.396 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/156 2024-05-04T02:57:42.449 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 133/156 2024-05-04T02:57:42.705 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/156 2024-05-04T02:57:42.746 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/156 2024-05-04T02:57:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/156 2024-05-04T02:57:42.836 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-04T02:57:42.863 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-04T02:57:43.046 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 144/156 2024-05-04T02:57:43.084 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-04T02:57:43.114 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 146/156 2024-05-04T02:57:43.329 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.2-8.el9.noarch 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.7.4-5.el9.x86_64 2024-05-04T02:57:43.330 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.331 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.4-9.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-04T02:57:43.332 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-04T02:57:43.333 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-04T02:57:43.334 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-13.14-1.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-04T02:57:43.335 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-04T02:57:43.336 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-04T02:57:43.337 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-04T02:57:43.338 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-04T02:57:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-04T02:57:43.342 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.343 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-3.el9.x86_64 2024-05-04T02:57:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-04T02:57:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-04T02:57:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-04T02:57:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-04T02:57:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-04T02:57:43.344 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-04T02:57:43.345 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-05-04T02:57:43.345 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-05-04T02:57:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-04T02:57:43.469 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T02:57:43.492 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T02:57:43.492 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:43.493 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-04T02:57:43.493 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T02:57:43.493 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T02:57:43.493 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:43.536 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T02:57:43.559 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T02:57:43.559 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:43.559 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-04T02:57:43.559 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:43.612 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T02:57:43.635 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T02:57:43.635 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:43.635 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-04T02:57:43.635 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T02:57:43.635 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T02:57:43.636 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:43.708 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:43.745 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:43.750 DEBUG:teuthology.parallel:result is None 2024-05-04T02:57:43.764 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:43.962 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T02:57:43.986 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T02:57:43.986 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T02:57:43.986 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-04T02:57:43.986 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T02:57:43.986 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T02:57:43.986 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:44.603 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 148/156 2024-05-04T02:57:44.691 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-04T02:57:44.733 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-04T02:57:44.822 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-04T02:57:44.854 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-04T02:57:44.855 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T02:57:44.963 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T02:57:45.005 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T02:57:45.005 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T02:57:45.108 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T02:57:45.178 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:48.509 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/156 2024-05-04T02:57:48.551 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/156 2024-05-04T02:57:48.592 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/156 2024-05-04T02:57:48.623 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-04T02:57:48.650 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-04T02:57:48.855 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 144/156 2024-05-04T02:57:49.030 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-04T02:57:49.059 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 146/156 2024-05-04T02:57:49.701 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:49.858 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:49.878 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T02:57:50.791 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 148/156 2024-05-04T02:57:50.913 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-04T02:57:50.957 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-04T02:57:51.051 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-04T02:57:51.093 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-04T02:57:51.093 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T02:57:51.177 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T02:57:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T02:57:51.210 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T02:57:51.302 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T02:57:51.370 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/156 2024-05-04T02:57:51.500 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/156 2024-05-04T02:57:51.501 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/156 2024-05-04T02:57:51.502 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/156 2024-05-04T02:57:51.502 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/156 2024-05-04T02:57:51.502 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/156 2024-05-04T02:57:51.502 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/156 2024-05-04T02:57:51.502 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/156 2024-05-04T02:57:51.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/156 2024-05-04T02:57:51.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/156 2024-05-04T02:57:51.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/156 2024-05-04T02:57:51.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/156 2024-05-04T02:57:51.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/156 2024-05-04T02:57:51.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/156 2024-05-04T02:57:51.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/156 2024-05-04T02:57:51.514 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-04T02:57:51.515 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-04T02:57:51.516 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-04T02:57:51.517 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-04T02:57:51.518 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-04T02:57:51.519 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-04T02:57:51.522 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-04T02:57:51.522 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-04T02:57:51.522 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-04T02:57:51.522 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-04T02:57:51.522 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-04T02:57:51.522 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-04T02:57:51.522 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-04T02:57:51.523 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-04T02:57:51.524 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T02:57:51.525 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-04T02:57:53.584 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:53.584 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:53.584 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-05-04T02:57:53.584 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.584 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.584 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:53.584 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:53.584 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.585 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.586 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-04T02:57:53.587 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.588 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.14-1.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-04T02:57:53.589 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-04T02:57:53.590 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-04T02:57:53.591 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-04T02:57:53.592 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.593 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-04T02:57:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-04T02:57:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-04T02:57:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-04T02:57:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T02:57:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-05-04T02:57:53.963 DEBUG:teuthology.parallel:result is None 2024-05-04T02:57:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/156 2024-05-04T02:57:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/156 2024-05-04T02:57:57.182 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/156 2024-05-04T02:57:57.183 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/156 2024-05-04T02:57:57.184 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/156 2024-05-04T02:57:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/156 2024-05-04T02:57:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/156 2024-05-04T02:57:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/156 2024-05-04T02:57:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/156 2024-05-04T02:57:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/156 2024-05-04T02:57:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/156 2024-05-04T02:57:57.196 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-04T02:57:57.197 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-04T02:57:57.198 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-04T02:57:57.199 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-04T02:57:57.200 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-04T02:57:57.201 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-04T02:57:57.202 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-04T02:57:57.203 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-04T02:57:57.204 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T02:57:57.205 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-04T02:57:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.2-8.el9.noarch 2024-05-04T02:57:59.268 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.7.4-5.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.269 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.270 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.4-9.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.271 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-04T02:57:59.272 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: lua-5.4.4-4.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-13.14-1.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-04T02:57:59.273 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-04T02:57:59.274 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-04T02:57:59.275 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-04T02:57:59.276 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply-3.11-14.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-04T02:57:59.277 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-04T02:57:59.278 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-04T02:57:59.282 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: re2-1:20211101-3.el9.x86_64 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-04T02:57:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-04T02:57:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-04T02:57:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-04T02:57:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-04T02:57:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-04T02:57:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:57:59.284 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-05-04T02:57:59.630 DEBUG:teuthology.parallel:result is None 2024-05-04T02:57:59.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:57:59.847 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-04T02:57:59.869 INFO:teuthology.orchestra.run.smithi039.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-04T02:57:59.870 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-04T02:57:59.870 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-04T02:57:59.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:58:00.018 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-04T02:58:00.045 INFO:teuthology.orchestra.run.smithi066.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-04T02:58:00.045 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-04T02:58:00.045 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-04T02:58:00.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:58:00.257 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-04T02:58:00.283 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-04T02:58:00.283 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-04T02:58:00.283 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-04T02:58:00.285 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-04T02:58:00.285 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T02:58:00.286 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-04T02:58:00.312 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T02:58:00.312 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-04T02:58:00.344 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T02:58:00.344 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-04T02:58:00.374 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-04T02:58:00.374 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T02:58:00.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-05-04T02:58:00.399 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-04T02:58:00.463 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T02:58:00.463 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-05-04T02:58:00.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-04T02:58:00.561 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T02:58:00.561 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-05-04T02:58:00.586 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-04T02:58:00.650 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-04T02:58:00.650 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T02:58:00.650 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-04T02:58:00.675 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-04T02:58:00.742 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T02:58:00.742 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-04T02:58:00.771 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-04T02:58:00.840 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T02:58:00.840 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-04T02:58:00.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-04T02:58:00.927 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-04T02:58:00.927 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T02:58:00.927 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-05-04T02:58:00.955 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-04T02:58:01.020 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T02:58:01.020 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-05-04T02:58:01.044 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-04T02:58:01.109 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T02:58:01.109 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-05-04T02:58:01.132 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-04T02:58:01.195 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-04T02:58:01.290 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-04T02:58:01.291 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:58:01.291 INFO:tasks.cephadm:Cluster fsid is 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:58:01.291 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-04T02:58:01.291 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.39', 'mon.b': '172.21.15.66', 'mon.c': '172.21.15.167'} 2024-05-04T02:58:01.292 INFO:tasks.cephadm:First mon is mon.a on smithi039 2024-05-04T02:58:01.292 INFO:tasks.cephadm:First mgr is x 2024-05-04T02:58:01.292 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-04T02:58:01.292 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname $(hostname -s) 2024-05-04T02:58:01.317 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2024-05-04T02:58:01.342 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-05-04T02:58:01.366 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-04T02:58:01.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:58:01.518 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79090/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3522-gbd4e00a3', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3522.gbd4e00a3'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-03 18:03:30.054903', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240503.163550-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-04T02:58:01.659 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-pdonnell-testing-20240503.163550-debug, sha1 bd4e00a39d766328b458e4c953f4d722fed96a31 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:58:01.661 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm 2024-05-04T02:58:01.661 DEBUG:teuthology.orchestra.run.smithi039:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:02.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 4 02:58 /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:02.281 DEBUG:teuthology.orchestra.run.smithi066:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:02.916 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 4 02:58 /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:02.916 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:03.517 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 4 02:58 /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:03.518 DEBUG:teuthology.orchestra.run.smithi039:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:03.538 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:03.557 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-04T02:58:03.594 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 on all hosts... 2024-05-04T02:58:03.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-04T02:58:03.597 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-04T02:58:03.600 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-04T02:58:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T02:58:05.217 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T02:58:05.395 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T02:58:36.538 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T02:58:36.539 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-04T02:58:36.539 INFO:teuthology.orchestra.run.smithi039.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-04T02:58:36.539 INFO:teuthology.orchestra.run.smithi039.stdout: "repo_digests": [ 2024-05-04T02:58:36.539 INFO:teuthology.orchestra.run.smithi039.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-04T02:58:36.539 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T02:58:36.539 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T02:59:05.473 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-04T02:59:05.474 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-04T02:59:05.474 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-04T02:59:05.474 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2024-05-04T02:59:05.474 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-04T02:59:05.474 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-05-04T02:59:05.474 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-04T02:59:11.124 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-04T02:59:11.124 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-04T02:59:11.124 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-04T02:59:11.124 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-05-04T02:59:11.125 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-04T02:59:11.125 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-04T02:59:11.125 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-04T02:59:11.150 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph 2024-05-04T02:59:11.183 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2024-05-04T02:59:11.217 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-05-04T02:59:11.243 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /etc/ceph 2024-05-04T02:59:11.272 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2024-05-04T02:59:11.301 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-05-04T02:59:11.325 INFO:tasks.cephadm:Writing seed config... 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-04T02:59:11.326 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-04T02:59:11.327 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-04T02:59:11.328 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-04T02:59:11.329 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-04T02:59:11.329 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-04T02:59:11.329 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-04T02:59:11.329 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-04T02:59:11.329 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-04T02:59:11.329 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-04T02:59:11.329 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-04T02:59:11.330 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T02:59:11.330 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-04T02:59:11.347 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1e913ae0-09c2-11ef-bc97-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-04T02:59:11.347 DEBUG:teuthology.orchestra.run.smithi039:mon.a> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a.service 2024-05-04T02:59:11.390 DEBUG:teuthology.orchestra.run.smithi039:mgr.x> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.x.service 2024-05-04T02:59:11.433 INFO:tasks.cephadm:Bootstrapping... 2024-05-04T02:59:11.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 -v bootstrap --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.39 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-04T02:59:11.633 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-05-04T02:59:11.633 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31', '-v', 'bootstrap', '--fsid', '1e913ae0-09c2-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.39', '--skip-admin-label'] 2024-05-04T02:59:11.634 INFO:teuthology.orchestra.run.smithi039.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-04T02:59:11.634 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying podman|docker is present... 2024-05-04T02:59:11.664 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-04T02:59:11.664 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying lvm2 is present... 2024-05-04T02:59:11.664 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying time synchronization is in place... 2024-05-04T02:59:11.672 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-04T02:59:11.672 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-04T02:59:11.678 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-04T02:59:11.678 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-04T02:59:11.685 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-05-04T02:59:11.691 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-05-04T02:59:11.691 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-05-04T02:59:11.691 INFO:teuthology.orchestra.run.smithi039.stdout:Repeating the final host check... 2024-05-04T02:59:11.716 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-04T02:59:11.716 INFO:teuthology.orchestra.run.smithi039.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-04T02:59:11.716 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl is present 2024-05-04T02:59:11.716 INFO:teuthology.orchestra.run.smithi039.stdout:lvcreate is present 2024-05-04T02:59:11.722 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-04T02:59:11.722 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-04T02:59:11.730 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-04T02:59:11.730 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-04T02:59:11.735 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-05-04T02:59:11.742 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-05-04T02:59:11.742 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-05-04T02:59:11.742 INFO:teuthology.orchestra.run.smithi039.stdout:Host looks OK 2024-05-04T02:59:11.742 INFO:teuthology.orchestra.run.smithi039.stdout:Cluster fsid: 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:11.743 INFO:teuthology.orchestra.run.smithi039.stdout:Acquiring lock 140145721945392 on /run/cephadm/1e913ae0-09c2-11ef-bc97-c7b262605968.lock 2024-05-04T02:59:11.743 INFO:teuthology.orchestra.run.smithi039.stdout:Lock 140145721945392 acquired on /run/cephadm/1e913ae0-09c2-11ef-bc97-c7b262605968.lock 2024-05-04T02:59:11.743 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 3300 ... 2024-05-04T02:59:11.743 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 6789 ... 2024-05-04T02:59:11.743 INFO:teuthology.orchestra.run.smithi039.stdout:Base mon IP(s) is [172.21.15.39:3300, 172.21.15.39:6789], mon addrv is [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-05-04T02:59:11.746 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.39 metric 100 2024-05-04T02:59:11.746 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.39 metric 100 2024-05-04T02:59:11.749 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-04T02:59:11.749 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-04T02:59:11.749 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-04T02:59:11.751 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-04T02:59:11.751 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-04T02:59:11.751 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-04T02:59:11.752 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-04T02:59:11.752 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1370/64 scope link noprefixroute 2024-05-04T02:59:11.752 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-04T02:59:11.752 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-05-04T02:59:11.752 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-05-04T02:59:11.752 INFO:teuthology.orchestra.run.smithi039.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-04T02:59:11.753 INFO:teuthology.orchestra.run.smithi039.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-04T02:59:11.754 INFO:teuthology.orchestra.run.smithi039.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T02:59:12.189 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 2024-05-04T02:59:12.189 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T02:59:12.189 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Getting image source signatures 2024-05-04T02:59:12.189 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:096f198f13bcfe0dfab6408b113f6513b5ae0fab440afa5e44aec6470f68ac59 2024-05-04T02:59:12.189 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-04T02:59:12.189 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying config sha256:afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 2024-05-04T02:59:12.189 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-04T02:59:13.177 INFO:teuthology.orchestra.run.smithi039.stdout:ceph: stdout ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev) 2024-05-04T02:59:13.177 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph version: ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev) 2024-05-04T02:59:13.177 INFO:teuthology.orchestra.run.smithi039.stdout:Extracting ceph user uid/gid from container image... 2024-05-04T02:59:14.132 INFO:teuthology.orchestra.run.smithi039.stdout:stat: stdout 167 167 2024-05-04T02:59:14.132 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial keys... 2024-05-04T02:59:15.063 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQCCpDVmSsrGHhAAONgMUG0UntYf78d+SuK8gQ== 2024-05-04T02:59:16.110 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQCDpDVm3vubGRAAagrskqimbA90P6moPOA/sg== 2024-05-04T02:59:16.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQCEpDVmGxfJGRAA6F6Zs6lbVKagATwy26Ep6w== 2024-05-04T02:59:16.991 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial monmap... 2024-05-04T02:59:17.871 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool for a [v2:172.21.15.39:3300,v1:172.21.15.39:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = quincy 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: set fsid to 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T02:59:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mon... 2024-05-04T02:59:18.869 INFO:teuthology.orchestra.run.smithi039.stdout:create mon.a on 2024-05-04T02:59:19.178 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-04T02:59:19.350 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-04T02:59:19.556 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968.target. 2024-05-04T02:59:19.556 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968.target. 2024-05-04T02:59:19.808 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a 2024-05-04T02:59:19.808 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-04T02:59:19.980 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968.target.wants/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@.service. 2024-05-04T02:59:20.468 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-04T02:59:20.468 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-05-04T02:59:20.475 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-04T02:59:20.475 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-04T02:59:20.475 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-05-04T02:59:20.475 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-05-04T02:59:20.475 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon to start... 2024-05-04T02:59:20.475 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon... 2024-05-04T02:59:20.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:20 smithi039 systemd[1]: Started Ceph mon.a for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T02:59:21.563 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout cluster: 2024-05-04T02:59:21.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout id: 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:21.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-04T02:59:21.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:21.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout services: 2024-05-04T02:59:21.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.594782s) 2024-05-04T02:59:21.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-04T02:59:21.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout data: 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pgs: 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.952+0000 7f2fe2eb3700 1 Processor -- start 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.952+0000 7f2fe2eb3700 1 -- start start 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.952+0000 7f2fe2eb3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.952+0000 7f2fe2eb3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc10a520 con 0x7f2fdc109b20 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.953+0000 7f2fe0c4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.953+0000 7f2fe0c4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50454/0 (socket says 172.21.15.39:50454) 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.953+0000 7f2fe0c4f700 1 -- 172.21.15.39:0/245910462 learned_addr learned my addr 172.21.15.39:0/245910462 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:21.571 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.953+0000 7f2fe0c4f700 1 -- 172.21.15.39:0/245910462 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc10acf0 con 0x7f2fdc109b20 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.953+0000 7f2fe0c4f700 1 --2- 172.21.15.39:0/245910462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc109f30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00098a0 tx=0x7f2fd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1245f9ec171e569d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.953+0000 7f2fdb7fe700 1 -- 172.21.15.39:0/245910462 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd0004160 con 0x7f2fdc109b20 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.953+0000 7f2fdb7fe700 1 -- 172.21.15.39:0/245910462 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd0004300 con 0x7f2fdc109b20 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.953+0000 7f2fdb7fe700 1 -- 172.21.15.39:0/245910462 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd00036a0 con 0x7f2fdc109b20 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.954+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/245910462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 msgr2=0x7f2fdc109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.954+0000 7f2fe2eb3700 1 --2- 172.21.15.39:0/245910462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc109f30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00098a0 tx=0x7f2fd0009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.954+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/245910462 shutdown_connections 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.954+0000 7f2fe2eb3700 1 --2- 172.21.15.39:0/245910462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc109f30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.954+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/245910462 >> 172.21.15.39:0/245910462 conn(0x7f2fdc105160 msgr2=0x7f2fdc107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.954+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/245910462 shutdown_connections 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.954+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/245910462 wait complete. 2024-05-04T02:59:21.572 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.955+0000 7f2fe2eb3700 1 Processor -- start 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.955+0000 7f2fe2eb3700 1 -- start start 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.955+0000 7f2fe2eb3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.955+0000 7f2fe2eb3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc10b220 con 0x7f2fdc109b20 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.955+0000 7f2fe0c4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.955+0000 7f2fe0c4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50466/0 (socket says 172.21.15.39:50466) 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.955+0000 7f2fe0c4f700 1 -- 172.21.15.39:0/198884613 learned_addr learned my addr 172.21.15.39:0/198884613 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.955+0000 7f2fe0c4f700 1 -- 172.21.15.39:0/198884613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc19db00 con 0x7f2fdc109b20 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.956+0000 7f2fe0c4f700 1 --2- 172.21.15.39:0/198884613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc19d5c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0000c00 tx=0x7f2fd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.956+0000 7f2fd9ffb700 1 -- 172.21.15.39:0/198884613 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd0003db0 con 0x7f2fdc109b20 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.956+0000 7f2fd9ffb700 1 -- 172.21.15.39:0/198884613 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd0011630 con 0x7f2fdc109b20 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.956+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc19dd90 con 0x7f2fdc109b20 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.956+0000 7f2fd9ffb700 1 -- 172.21.15.39:0/198884613 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd0011950 con 0x7f2fdc109b20 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.956+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc1a09a0 con 0x7f2fdc109b20 2024-05-04T02:59:21.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.956+0000 7f2fd9ffb700 1 -- 172.21.15.39:0/198884613 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2fd001a460 con 0x7f2fdc109b20 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.956+0000 7f2fd9ffb700 1 -- 172.21.15.39:0/198884613 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2fd001b780 con 0x7f2fdc109b20 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.957+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc064e20 con 0x7f2fdc109b20 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:20.959+0000 7f2fd9ffb700 1 -- 172.21.15.39:0/198884613 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2fd0031020 con 0x7f2fdc109b20 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.033+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f2fdc02ea40 con 0x7f2fdc109b20 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.033+0000 7f2fd9ffb700 1 -- 172.21.15.39:0/198884613 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2fd001b9d0 con 0x7f2fdc109b20 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.034+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 msgr2=0x7f2fdc19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.034+0000 7f2fe2eb3700 1 --2- 172.21.15.39:0/198884613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc19d5c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0000c00 tx=0x7f2fd0004750 comp rx=0 tx=0).stop 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.034+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 shutdown_connections 2024-05-04T02:59:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.034+0000 7f2fe2eb3700 1 --2- 172.21.15.39:0/198884613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fdc109b20 0x7f2fdc19d5c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:21.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.034+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 >> 172.21.15.39:0/198884613 conn(0x7f2fdc105160 msgr2=0x7f2fdc193cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:21.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.034+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 shutdown_connections 2024-05-04T02:59:21.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:21.034+0000 7f2fe2eb3700 1 -- 172.21.15.39:0/198884613 wait complete. 2024-05-04T02:59:21.575 INFO:teuthology.orchestra.run.smithi039.stdout:mon is available 2024-05-04T02:59:21.575 INFO:teuthology.orchestra.run.smithi039.stdout:Assimilating anything we can from ceph.conf... 2024-05-04T02:59:21.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-04T02:59:21.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: monmap epoch 1 2024-05-04T02:59:21.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: last_changed 2024-05-04T02:59:17.356054+0000 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: created 2024-05-04T02:59:17.356054+0000 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: min_mon_release 19 (squid) 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: election_strategy: 1 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: fsmap 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: osdmap e1: 0 total, 0 up, 0 in 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: mgrmap e1: no daemons active 2024-05-04T02:59:21.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:21 smithi039 ceph-mon[29335]: from='client.? 172.21.15.39:0/198884613' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-04T02:59:22.678 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.060+0000 7fc7c88d8700 1 Processor -- start 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.060+0000 7fc7c88d8700 1 -- start start 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.060+0000 7fc7c88d8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c4108e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:22.679 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.060+0000 7fc7c88d8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4109400 con 0x7fc7c4108a00 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.060+0000 7fc7c259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c4108e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.061+0000 7fc7c259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c4108e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50476/0 (socket says 172.21.15.39:50476) 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.061+0000 7fc7c259c700 1 -- 172.21.15.39:0/3427336791 learned_addr learned my addr 172.21.15.39:0/3427336791 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.061+0000 7fc7c259c700 1 -- 172.21.15.39:0/3427336791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c4109bd0 con 0x7fc7c4108a00 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.061+0000 7fc7c259c700 1 --2- 172.21.15.39:0/3427336791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c4108e10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40098a0 tx=0x7fc7b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b66bc8a9e3be86 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.061+0000 7fc7c159a700 1 -- 172.21.15.39:0/3427336791 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7b4004160 con 0x7fc7c4108a00 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.061+0000 7fc7c159a700 1 -- 172.21.15.39:0/3427336791 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b4004300 con 0x7fc7c4108a00 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.061+0000 7fc7c159a700 1 -- 172.21.15.39:0/3427336791 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7b40036a0 con 0x7fc7c4108a00 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.062+0000 7fc7c88d8700 1 -- 172.21.15.39:0/3427336791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 msgr2=0x7fc7c4108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.062+0000 7fc7c88d8700 1 --2- 172.21.15.39:0/3427336791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c4108e10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40098a0 tx=0x7fc7b4009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.062+0000 7fc7c88d8700 1 -- 172.21.15.39:0/3427336791 shutdown_connections 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.062+0000 7fc7c88d8700 1 --2- 172.21.15.39:0/3427336791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c4108e10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.062+0000 7fc7c88d8700 1 -- 172.21.15.39:0/3427336791 >> 172.21.15.39:0/3427336791 conn(0x7fc7c4104020 msgr2=0x7fc7c41064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:22.680 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.062+0000 7fc7c88d8700 1 -- 172.21.15.39:0/3427336791 shutdown_connections 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.062+0000 7fc7c88d8700 1 -- 172.21.15.39:0/3427336791 wait complete. 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.063+0000 7fc7c88d8700 1 Processor -- start 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.063+0000 7fc7c88d8700 1 -- start start 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.063+0000 7fc7c88d8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.063+0000 7fc7c88d8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c410a0a0 con 0x7fc7c4108a00 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.063+0000 7fc7c259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.063+0000 7fc7c259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50480/0 (socket says 172.21.15.39:50480) 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.063+0000 7fc7c259c700 1 -- 172.21.15.39:0/1981671530 learned_addr learned my addr 172.21.15.39:0/1981671530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7c259c700 1 -- 172.21.15.39:0/1981671530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c419b8f0 con 0x7fc7c4108a00 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7c259c700 1 --2- 172.21.15.39:0/1981671530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c419b3b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4000c00 tx=0x7fc7b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7b37fe700 1 -- 172.21.15.39:0/1981671530 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7b4003db0 con 0x7fc7c4108a00 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7b37fe700 1 -- 172.21.15.39:0/1981671530 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b4011630 con 0x7fc7c4108a00 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c419bb80 con 0x7fc7c4108a00 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7b37fe700 1 -- 172.21.15.39:0/1981671530 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7b4011950 con 0x7fc7c4108a00 2024-05-04T02:59:22.681 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c419e790 con 0x7fc7c4108a00 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7b37fe700 1 -- 172.21.15.39:0/1981671530 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc7b4011af0 con 0x7fc7c4108a00 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.064+0000 7fc7b37fe700 1 -- 172.21.15.39:0/1981671530 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc7b401c790 con 0x7fc7c4108a00 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.065+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c4064e20 con 0x7fc7c4108a00 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.068+0000 7fc7b37fe700 1 -- 172.21.15.39:0/1981671530 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc7b4032020 con 0x7fc7c4108a00 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.138+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc7c402e8f0 con 0x7fc7c4108a00 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.145+0000 7fc7b37fe700 1 -- 172.21.15.39:0/1981671530 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fc7b4004260 con 0x7fc7c4108a00 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.146+0000 7fc7b37fe700 1 -- 172.21.15.39:0/1981671530 <== mon.0 v2:172.21.15.39:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc7b401ce00 con 0x7fc7c4108a00 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.146+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 msgr2=0x7fc7c419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.146+0000 7fc7c88d8700 1 --2- 172.21.15.39:0/1981671530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c419b3b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4000c00 tx=0x7fc7b4004750 comp rx=0 tx=0).stop 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.146+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 shutdown_connections 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.146+0000 7fc7c88d8700 1 --2- 172.21.15.39:0/1981671530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c4108a00 0x7fc7c419b3b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.146+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 >> 172.21.15.39:0/1981671530 conn(0x7fc7c4104020 msgr2=0x7fc7c4191b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:22.682 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.146+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 shutdown_connections 2024-05-04T02:59:22.683 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:22.146+0000 7fc7c88d8700 1 -- 172.21.15.39:0/1981671530 wait complete. 2024-05-04T02:59:22.683 INFO:teuthology.orchestra.run.smithi039.stdout:Generating new minimal ceph.conf... 2024-05-04T02:59:22.776 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:22 smithi039 ceph-mon[29335]: from='client.? 172.21.15.39:0/1981671530' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-04T02:59:22.776 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:22 smithi039 ceph-mon[29335]: from='client.? 172.21.15.39:0/1981671530' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-04T02:59:23.701 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:23 smithi039 ceph-mon[29335]: from='client.? 172.21.15.39:0/2509041573' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T02:59:23.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.074+0000 7f412428e700 1 Processor -- start 2024-05-04T02:59:23.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.074+0000 7f412428e700 1 -- start start 2024-05-04T02:59:23.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.074+0000 7f412428e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c10ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:23.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f412428e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f411c07ce30 con 0x7f411c10a5f0 2024-05-04T02:59:23.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f412202a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c10ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:23.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f412202a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c10ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50492/0 (socket says 172.21.15.39:50492) 2024-05-04T02:59:23.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f412202a700 1 -- 172.21.15.39:0/3710233594 learned_addr learned my addr 172.21.15.39:0/3710233594 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:23.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f412202a700 1 -- 172.21.15.39:0/3710233594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c10cf50 con 0x7f411c10a5f0 2024-05-04T02:59:23.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f412202a700 1 --2- 172.21.15.39:0/3710233594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c10ca10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41100098a0 tx=0x7f4110009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a51596d3c494bfd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:23.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f4121028700 1 -- 172.21.15.39:0/3710233594 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4110004160 con 0x7f411c10a5f0 2024-05-04T02:59:23.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f4121028700 1 -- 172.21.15.39:0/3710233594 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4110004300 con 0x7f411c10a5f0 2024-05-04T02:59:23.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.075+0000 7f4121028700 1 -- 172.21.15.39:0/3710233594 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41100119e0 con 0x7f411c10a5f0 2024-05-04T02:59:23.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.076+0000 7f412428e700 1 -- 172.21.15.39:0/3710233594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 msgr2=0x7f411c10ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:23.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.076+0000 7f412428e700 1 --2- 172.21.15.39:0/3710233594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c10ca10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41100098a0 tx=0x7f4110009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:23.707 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.076+0000 7f412428e700 1 -- 172.21.15.39:0/3710233594 shutdown_connections 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.076+0000 7f412428e700 1 --2- 172.21.15.39:0/3710233594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c10ca10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.076+0000 7f412428e700 1 -- 172.21.15.39:0/3710233594 >> 172.21.15.39:0/3710233594 conn(0x7f411c104010 msgr2=0x7f411c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.076+0000 7f412428e700 1 -- 172.21.15.39:0/3710233594 shutdown_connections 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.076+0000 7f412428e700 1 -- 172.21.15.39:0/3710233594 wait complete. 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.077+0000 7f412428e700 1 Processor -- start 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.077+0000 7f412428e700 1 -- start start 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.077+0000 7f412428e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c109d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.077+0000 7f412428e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f411c10dd70 con 0x7f411c10a5f0 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.077+0000 7f412202a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.077+0000 7f412202a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50498/0 (socket says 172.21.15.39:50498) 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.077+0000 7f412202a700 1 -- 172.21.15.39:0/2509041573 learned_addr learned my addr 172.21.15.39:0/2509041573 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f412202a700 1 -- 172.21.15.39:0/2509041573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c10a2b0 con 0x7f411c10a5f0 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f412202a700 1 --2- 172.21.15.39:0/2509041573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c109d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4110003f20 tx=0x7f4110011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:23.708 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f410f7fe700 1 -- 172.21.15.39:0/2509041573 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41100038b0 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f410f7fe700 1 -- 172.21.15.39:0/2509041573 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f4110003a50 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c108500 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f410f7fe700 1 -- 172.21.15.39:0/2509041573 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4110019550 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c1089e0 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f410f7fe700 1 -- 172.21.15.39:0/2509041573 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4110019be0 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.078+0000 7f410f7fe700 1 -- 172.21.15.39:0/2509041573 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4110024730 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.079+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c064e20 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.081+0000 7f410f7fe700 1 -- 172.21.15.39:0/2509041573 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4110043ec0 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.152+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f411c108c60 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.152+0000 7f410f7fe700 1 -- 172.21.15.39:0/2509041573 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f4110032030 con 0x7f411c10a5f0 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.153+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 msgr2=0x7f411c109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.153+0000 7f412428e700 1 --2- 172.21.15.39:0/2509041573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c109d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4110003f20 tx=0x7f4110011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.153+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 shutdown_connections 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.153+0000 7f412428e700 1 --2- 172.21.15.39:0/2509041573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411c10a5f0 0x7f411c109d70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:23.709 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.153+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 >> 172.21.15.39:0/2509041573 conn(0x7f411c104010 msgr2=0x7f411c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:23.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.153+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 shutdown_connections 2024-05-04T02:59:23.710 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:23.153+0000 7f412428e700 1 -- 172.21.15.39:0/2509041573 wait complete. 2024-05-04T02:59:23.710 INFO:teuthology.orchestra.run.smithi039.stdout:Restarting the monitor... 2024-05-04T02:59:23.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:23 smithi039 systemd[1]: Stopping Ceph mon.a for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T02:59:24.360 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:23 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a[29331]: 2024-05-04T02:59:23.951+0000 7f89868c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T02:59:24.360 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:23 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a[29331]: 2024-05-04T02:59:23.951+0000 7f89868c4700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-04T02:59:24.360 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:24 smithi039 podman[29582]: 2024-05-04 02:59:24.126362318 +0000 UTC m=+0.255207573 container died 4621556b21ddf9dca7ed43813d860f86e4f1b30d014fd7963de08d985bd45046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-04T02:59:24.627 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:24 smithi039 podman[29582]: 2024-05-04 02:59:24.459853353 +0000 UTC m=+0.588698597 container cleanup 4621556b21ddf9dca7ed43813d860f86e4f1b30d014fd7963de08d985bd45046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T02:59:24.627 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:24 smithi039 bash[29582]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a 2024-05-04T02:59:24.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:24 smithi039 podman[29597]: 2024-05-04 02:59:24.627441944 +0000 UTC m=+0.496113743 container remove 4621556b21ddf9dca7ed43813d860f86e4f1b30d014fd7963de08d985bd45046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-04T02:59:25.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:24 smithi039 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-04T02:59:25.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:24 smithi039 systemd[1]: Stopped Ceph mon.a for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T02:59:25.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:24 smithi039 systemd[1]: Starting Ceph mon.a for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T02:59:25.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 podman[29696]: 2024-05-04 02:59:25.095005426 +0000 UTC m=+0.098924776 container create 8812331cba60dbf671af7e283147cc4e35070fb6cfbcd8e4a6c644b432482cfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T02:59:25.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 podman[29696]: 2024-05-04 02:59:25.014497551 +0000 UTC m=+0.018416903 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:59:25.385 INFO:teuthology.orchestra.run.smithi039.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-04T02:59:25.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 podman[29696]: 2024-05-04 02:59:25.306191836 +0000 UTC m=+0.310111186 container init 8812331cba60dbf671af7e283147cc4e35070fb6cfbcd8e4a6c644b432482cfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430) 2024-05-04T02:59:25.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 podman[29696]: 2024-05-04 02:59:25.309343126 +0000 UTC m=+0.313262476 container start 8812331cba60dbf671af7e283147cc4e35070fb6cfbcd8e4a6c644b432482cfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-05-04T02:59:25.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: set uid:gid to 167:167 (ceph:ceph) 2024-05-04T02:59:25.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev), process ceph-mon, pid 2 2024-05-04T02:59:25.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: pidfile_write: ignore empty --pid-file 2024-05-04T02:59:25.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: load: jerasure load: lrc 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: RocksDB version: 7.9.2 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Git sha 0 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Compile date 2024-05-03 18:51:23 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: DB SUMMARY 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: DB Session ID: OK7NR5OXWCY1BECDRV0X 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: CURRENT file: CURRENT 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: IDENTITY file: IDENTITY 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104666 ; 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.error_if_exists: 0 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.create_if_missing: 0 2024-05-04T02:59:25.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.paranoid_checks: 1 2024-05-04T02:59:25.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-04T02:59:25.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-04T02:59:25.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-04T02:59:25.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.env: 0x5644163a5500 2024-05-04T02:59:25.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.fs: PosixFileSystem 2024-05-04T02:59:25.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.info_log: 0x564416dee020 2024-05-04T02:59:25.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_file_opening_threads: 16 2024-05-04T02:59:25.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.statistics: (nil) 2024-05-04T02:59:25.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.use_fsync: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_log_file_size: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.keep_log_file_num: 1000 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.recycle_log_file_num: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.allow_fallocate: 1 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.allow_mmap_reads: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.allow_mmap_writes: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.use_direct_reads: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.create_missing_column_families: 0 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.db_log_dir: 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.wal_dir: 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-04T02:59:25.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.advise_random_on_open: 1 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.db_write_buffer_size: 0 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.write_buffer_manager: 0x564416cb88c0 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-04T02:59:25.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-04T02:59:25.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-04T02:59:25.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.rate_limiter: (nil) 2024-05-04T02:59:25.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-04T02:59:25.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.wal_recovery_mode: 2 2024-05-04T02:59:25.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.enable_thread_tracking: 0 2024-05-04T02:59:25.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.enable_pipelined_write: 0 2024-05-04T02:59:25.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.unordered_write: 0 2024-05-04T02:59:25.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.row_cache: None 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.wal_filter: None 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.allow_ingest_behind: 0 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.two_write_queues: 0 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.manual_wal_flush: 0 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.wal_compression: 0 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.atomic_flush: 0 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-04T02:59:25.450 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.log_readahead_size: 0 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.best_efforts_recovery: 0 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.allow_data_in_errors: 0 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.db_host_id: __hostname__ 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_background_jobs: 2 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_background_compactions: -1 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_subcompactions: 1 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-04T02:59:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_total_wal_size: 0 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_open_files: -1 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bytes_per_sync: 0 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_readahead_size: 0 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_background_flushes: -1 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Compression algorithms supported: 2024-05-04T02:59:25.452 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: kZSTD supported: 0 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: kXpressCompression supported: 0 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: kLZ4HCCompression supported: 1 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: kZlibCompression supported: 1 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: kSnappyCompression supported: 1 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: kLZ4Compression supported: 1 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: kBZip2Compression supported: 0 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-04T02:59:25.453 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.merge_operator: 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_filter: None 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_filter_factory: None 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.sst_partitioner_factory: None 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564416dee1e0) 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks: 1 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_top_level_index_and_filter: 1 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_type: 0 2024-05-04T02:59:25.454 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_index_type: 0 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_shortening: 1 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: checksum: 4 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: no_block_cache: 0 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache: 0x564416ceb1f0 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_name: BinnedLRUCache 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_options: 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: capacity : 536870912 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_shard_bits : 4 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: strict_capacity_limit : 0 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: high_pri_pool_ratio: 0.000 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_compressed: (nil) 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: persistent_cache: (nil) 2024-05-04T02:59:25.455 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size: 4096 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size_deviation: 10 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_restart_interval: 16 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_block_restart_interval: 1 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: metadata_block_size: 4096 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: partition_filters: 0 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: use_delta_encoding: 1 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: filter_policy: bloomfilter 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: whole_key_filtering: 1 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: verify_compression: 0 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: read_amp_bytes_per_bit: 0 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: format_version: 5 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_index_compression: 1 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_align: 0 2024-05-04T02:59:25.456 INFO:journalctl@ceph.mon.a.smithi039.stdout: max_auto_readahead_size: 262144 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout: prepopulate_block_cache: 0 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout: initial_auto_readahead_size: 8192 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_file_reads_for_auto_readahead: 2 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.write_buffer_size: 33554432 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_write_buffer_number: 2 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression: NoCompression 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression: Disabled 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.prefix_extractor: nullptr 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.num_levels: 7 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-04T02:59:25.457 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.level: 32767 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.strategy: 0 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-04T02:59:25.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.enabled: false 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.target_file_size_base: 67108864 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-04T02:59:25.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.arena_block_size: 1048576 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.disable_auto_compactions: 0 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-04T02:59:25.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.inplace_update_support: 0 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.bloom_locality: 0 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.max_successive_merges: 0 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-04T02:59:25.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.paranoid_file_checks: 0 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.force_consistency_checks: 1 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.report_bg_io_stats: 0 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.ttl: 2592000 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.enable_blob_files: false 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.min_blob_size: 0 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.blob_file_size: 268435456 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-04T02:59:25.462 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.blob_file_starting_level: 0 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e3235ab8-54b5-493c-a72c-003e6f2b5a72 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714791565350419, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714791565351672, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99919, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97803, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14056, "raw_average_key_size": 49, "raw_value_size": 90105, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714791565, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e3235ab8-54b5-493c-a72c-003e6f2b5a72", "db_session_id": "OK7NR5OXWCY1BECDRV0X", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714791565351756, "job": 1, "event": "recovery_finished"} 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-04T02:59:25.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564416ccfc00 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: DB pointer 0x56441bac2000 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** DB Stats ** 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-04T02:59:25.464 INFO:journalctl@ceph.mon.a.smithi039.stdout: L0 2/0 99.44 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: Sum 2/0 99.44 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 136.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-04T02:59:25.465 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative compaction: 0.00 GB write, 18.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval compaction: 0.00 GB write, 18.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache BinnedLRUCache@0x564416ceb1f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.7e-05 secs_since: 0 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: starting mon.a rank 0 at public addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] at bind addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a@-1(???) e1 preinit fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a@-1(???).mds e1 new map 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a@-1(???).mds e1 print_map 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: e1 2024-05-04T02:59:25.466 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout: legacy client fscid: -1 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout: No filesystems configured 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 bash[29696]: 8812331cba60dbf671af7e283147cc4e35070fb6cfbcd8e4a6c644b432482cfb 2024-05-04T02:59:25.467 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 systemd[1]: Started Ceph mon.a for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: monmap epoch 1 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: last_changed 2024-05-04T02:59:17.356054+0000 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: created 2024-05-04T02:59:17.356054+0000 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: min_mon_release 19 (squid) 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: election_strategy: 1 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: fsmap 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: osdmap e1: 0 total, 0 up, 0 in 2024-05-04T02:59:25.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:25 smithi039 ceph-mon[29715]: mgrmap e1: no daemons active 2024-05-04T02:59:26.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.943+0000 7fe5625c7700 1 Processor -- start 2024-05-04T02:59:26.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.943+0000 7fe5625c7700 1 -- start start 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.944+0000 7fe5625c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c10ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.944+0000 7fe5625c7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c07d6a0 con 0x7fe55c10a670 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.944+0000 7fe55bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c10ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.944+0000 7fe55bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c10ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49666/0 (socket says 172.21.15.39:49666) 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.944+0000 7fe55bfff700 1 -- 172.21.15.39:0/1821311792 learned_addr learned my addr 172.21.15.39:0/1821311792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.944+0000 7fe55bfff700 1 -- 172.21.15.39:0/1821311792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c10cfd0 con 0x7fe55c10a670 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.944+0000 7fe55bfff700 1 --2- 172.21.15.39:0/1821311792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c10ca90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0098a0 tx=0x7fe54c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ced2f1a5baade8b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe55affd700 1 -- 172.21.15.39:0/1821311792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe54c004160 con 0x7fe55c10a670 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe55affd700 1 -- 172.21.15.39:0/1821311792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe54c004300 con 0x7fe55c10a670 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe55affd700 1 -- 172.21.15.39:0/1821311792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe54c0119e0 con 0x7fe55c10a670 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe5625c7700 1 -- 172.21.15.39:0/1821311792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 msgr2=0x7fe55c10ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe5625c7700 1 --2- 172.21.15.39:0/1821311792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c10ca90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0098a0 tx=0x7fe54c009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe5625c7700 1 -- 172.21.15.39:0/1821311792 shutdown_connections 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe5625c7700 1 --2- 172.21.15.39:0/1821311792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c10ca90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:26.621 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe5625c7700 1 -- 172.21.15.39:0/1821311792 >> 172.21.15.39:0/1821311792 conn(0x7fe55c104050 msgr2=0x7fe55c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.945+0000 7fe5625c7700 1 -- 172.21.15.39:0/1821311792 shutdown_connections 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.946+0000 7fe5625c7700 1 -- 172.21.15.39:0/1821311792 wait complete. 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.946+0000 7fe5625c7700 1 Processor -- start 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.946+0000 7fe5625c7700 1 -- start start 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe5625c7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe5625c7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c10ddf0 con 0x7fe55c10a670 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe55bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe55bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c19dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49672/0 (socket says 172.21.15.39:49672) 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe55bfff700 1 -- 172.21.15.39:0/345050599 learned_addr learned my addr 172.21.15.39:0/345050599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe55bfff700 1 -- 172.21.15.39:0/345050599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c19e220 con 0x7fe55c10a670 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe55bfff700 1 --2- 172.21.15.39:0/345050599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c19dce0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0119b0 tx=0x7fe54c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe5597fa700 1 -- 172.21.15.39:0/345050599 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe54c003af0 con 0x7fe55c10a670 2024-05-04T02:59:26.622 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe5597fa700 1 -- 172.21.15.39:0/345050599 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe54c01a8a0 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.947+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c19e4b0 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.948+0000 7fe5597fa700 1 -- 172.21.15.39:0/345050599 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe54c024830 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.948+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c1a10c0 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.948+0000 7fe5597fa700 1 -- 172.21.15.39:0/345050599 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe54c022020 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.948+0000 7fe5597fa700 1 -- 172.21.15.39:0/345050599 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe54c0193c0 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.949+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55c064e20 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:25.951+0000 7fe5597fa700 1 -- 172.21.15.39:0/345050599 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe54c02ddf0 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.021+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe55c1a12e0 con 0x7fe55c10a670 2024-05-04T02:59:26.623 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.025+0000 7fe5597fa700 1 -- 172.21.15.39:0/345050599 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe54c032030 con 0x7fe55c10a670 2024-05-04T02:59:26.624 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.026+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 msgr2=0x7fe55c19dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:26.624 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.026+0000 7fe5625c7700 1 --2- 172.21.15.39:0/345050599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c19dce0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0119b0 tx=0x7fe54c011a90 comp rx=0 tx=0).stop 2024-05-04T02:59:26.624 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.026+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 shutdown_connections 2024-05-04T02:59:26.624 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.026+0000 7fe5625c7700 1 --2- 172.21.15.39:0/345050599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55c10a670 0x7fe55c19dce0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:26.624 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.026+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 >> 172.21.15.39:0/345050599 conn(0x7fe55c104050 msgr2=0x7fe55c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:26.624 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.026+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 shutdown_connections 2024-05-04T02:59:26.624 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:26.026+0000 7fe5625c7700 1 -- 172.21.15.39:0/345050599 wait complete. 2024-05-04T02:59:26.624 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-04T02:59:26.645 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-04T02:59:26.646 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mgr... 2024-05-04T02:59:26.646 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-04T02:59:26.646 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-04T02:59:26.926 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.x 2024-05-04T02:59:26.927 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-04T02:59:27.079 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968.target.wants/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@.service. 2024-05-04T02:59:27.307 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:27 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/345050599' entity='client.admin' 2024-05-04T02:59:27.307 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 systemd[1]: Starting Ceph mgr.x for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T02:59:27.550 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-04T02:59:27.550 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-05-04T02:59:27.557 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-04T02:59:27.557 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-04T02:59:27.558 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-05-04T02:59:27.558 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-05-04T02:59:27.566 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-04T02:59:27.566 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-05-04T02:59:27.574 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-04T02:59:27.574 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-04T02:59:27.574 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-05-04T02:59:27.574 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-04T02:59:27.574 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr to start... 2024-05-04T02:59:27.574 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr... 2024-05-04T02:59:27.651 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 podman[29928]: 2024-05-04 02:59:27.306841703 +0000 UTC m=+0.105227731 container create 4730d304cd2a7bcf36eaae11af669c817c52992f45b6b40d7e321e06951d0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0) 2024-05-04T02:59:27.651 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 podman[29928]: 2024-05-04 02:59:27.219754355 +0000 UTC m=+0.018140408 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:59:27.651 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 podman[29928]: 2024-05-04 02:59:27.476105104 +0000 UTC m=+0.274491140 container init 4730d304cd2a7bcf36eaae11af669c817c52992f45b6b40d7e321e06951d0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240430) 2024-05-04T02:59:27.651 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 podman[29928]: 2024-05-04 02:59:27.478655018 +0000 UTC m=+0.277041048 container start 4730d304cd2a7bcf36eaae11af669c817c52992f45b6b40d7e321e06951d0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T02:59:27.651 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 bash[29928]: 4730d304cd2a7bcf36eaae11af669c817c52992f45b6b40d7e321e06951d0ea2 2024-05-04T02:59:27.651 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 systemd[1]: Started Ceph mgr.x for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T02:59:27.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:27.650+0000 7fe91cb0e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T02:59:27.941 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:27.845+0000 7fe91cb0e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T02:59:28.289 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:28 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1315635630' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-04T02:59:28.290 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:27.952+0000 7fe91cb0e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T02:59:28.640 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:28.288+0000 7fe91cb0e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T02:59:28.640 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:28.379+0000 7fe91cb0e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T02:59:28.653 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:28.653 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-04T02:59:28.653 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T02:59:28.653 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-05-04T02:59:28.653 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-04T02:59:28.654 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-04T02:59:28.655 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-04T02:59:28.656 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-05-04T02:59:20.441047+0000", 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-04T02:59:28.657 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.000+0000 7f178c64d700 1 Processor -- start 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.000+0000 7f178c64d700 1 -- start start 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.000+0000 7f178c64d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f1784108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.000+0000 7f178c64d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17841093b0 con 0x7f17841089b0 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.001+0000 7f178a3e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f1784108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.001+0000 7f178a3e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f1784108dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49684/0 (socket says 172.21.15.39:49684) 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.001+0000 7f178a3e9700 1 -- 172.21.15.39:0/3377845380 learned_addr learned my addr 172.21.15.39:0/3377845380 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.001+0000 7f178a3e9700 1 -- 172.21.15.39:0/3377845380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1784109b80 con 0x7f17841089b0 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.001+0000 7f178a3e9700 1 --2- 172.21.15.39:0/3377845380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f1784108dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f17780098a0 tx=0x7f1778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3b45b01bcc3aae server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.002+0000 7f17893e7700 1 -- 172.21.15.39:0/3377845380 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1778004160 con 0x7f17841089b0 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.002+0000 7f17893e7700 1 -- 172.21.15.39:0/3377845380 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1778004300 con 0x7f17841089b0 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.002+0000 7f17893e7700 1 -- 172.21.15.39:0/3377845380 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17780119e0 con 0x7f17841089b0 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.002+0000 7f178c64d700 1 -- 172.21.15.39:0/3377845380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 msgr2=0x7f1784108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.002+0000 7f178c64d700 1 --2- 172.21.15.39:0/3377845380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f1784108dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f17780098a0 tx=0x7f1778009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:28.658 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.003+0000 7f178c64d700 1 -- 172.21.15.39:0/3377845380 shutdown_connections 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.003+0000 7f178c64d700 1 --2- 172.21.15.39:0/3377845380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f1784108dc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.003+0000 7f178c64d700 1 -- 172.21.15.39:0/3377845380 >> 172.21.15.39:0/3377845380 conn(0x7f1784104010 msgr2=0x7f17841064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.003+0000 7f178c64d700 1 -- 172.21.15.39:0/3377845380 shutdown_connections 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.003+0000 7f178c64d700 1 -- 172.21.15.39:0/3377845380 wait complete. 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.003+0000 7f178c64d700 1 Processor -- start 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.003+0000 7f178c64d700 1 -- start start 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f178c64d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f178407c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f178c64d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f178410cdb0 con 0x7f17841089b0 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f178a3e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f178407c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f178a3e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f178407c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49690/0 (socket says 172.21.15.39:49690) 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f178a3e9700 1 -- 172.21.15.39:0/1315635630 learned_addr learned my addr 172.21.15.39:0/1315635630 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f178a3e9700 1 -- 172.21.15.39:0/1315635630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178407cf00 con 0x7f17841089b0 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f178a3e9700 1 --2- 172.21.15.39:0/1315635630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f178407c9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1778009fb0 tx=0x7f1778011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:28.659 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f17777fe700 1 -- 172.21.15.39:0/1315635630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17780038b0 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17841a0ae0 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.004+0000 7f17777fe700 1 -- 172.21.15.39:0/1315635630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1778003a50 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.005+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841a0fc0 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.005+0000 7f17777fe700 1 -- 172.21.15.39:0/1315635630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177801a5c0 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.005+0000 7f17777fe700 1 -- 172.21.15.39:0/1315635630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f177801ac70 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.005+0000 7f17777fe700 1 -- 172.21.15.39:0/1315635630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1778019700 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.005+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1784064e20 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.008+0000 7f17777fe700 1 -- 172.21.15.39:0/1315635630 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1778024a60 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.083+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f17841a1310 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.084+0000 7f17777fe700 1 -- 172.21.15.39:0/1315635630 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1778024440 con 0x7f17841089b0 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.085+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 msgr2=0x7f178407c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.085+0000 7f178c64d700 1 --2- 172.21.15.39:0/1315635630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f178407c9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1778009fb0 tx=0x7f1778011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.085+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 shutdown_connections 2024-05-04T02:59:28.660 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.085+0000 7f178c64d700 1 --2- 172.21.15.39:0/1315635630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17841089b0 0x7f178407c9c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:28.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.085+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 >> 172.21.15.39:0/1315635630 conn(0x7f1784104010 msgr2=0x7f1784104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:28.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.086+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 shutdown_connections 2024-05-04T02:59:28.661 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:28.086+0000 7f178c64d700 1 -- 172.21.15.39:0/1315635630 wait complete. 2024-05-04T02:59:28.661 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (1/15)... 2024-05-04T02:59:29.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:29.636+0000 7fe91cb0e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T02:59:29.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:29.811+0000 7fe91cb0e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T02:59:30.287 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:29.987+0000 7fe91cb0e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T02:59:30.287 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:30.286+0000 7fe91cb0e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T02:59:30.591 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:30.381+0000 7fe91cb0e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T02:59:30.591 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:30.590+0000 7fe91cb0e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T02:59:30.864 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:30.796+0000 7fe91cb0e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T02:59:31.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:31 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1529350022' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-04T02:59:31.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:31 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:31.438+0000 7fe91cb0e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T02:59:31.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:31 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:31.668+0000 7fe91cb0e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-05-04T02:59:31.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-04T02:59:31.719 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:31.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:31.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-05-04T02:59:20.441047+0000", 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.071+0000 7f7736b97700 1 Processor -- start 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.072+0000 7f7736b97700 1 -- start start 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.072+0000 7f7736b97700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f773010ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.072+0000 7f7736b97700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f773007ce30 con 0x7f773010a5f0 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.072+0000 7f7734933700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f773010ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.072+0000 7f7734933700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f773010ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49702/0 (socket says 172.21.15.39:49702) 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.072+0000 7f7734933700 1 -- 172.21.15.39:0/2813555930 learned_addr learned my addr 172.21.15.39:0/2813555930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.072+0000 7f7734933700 1 -- 172.21.15.39:0/2813555930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773010cf50 con 0x7f773010a5f0 2024-05-04T02:59:31.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.073+0000 7f7734933700 1 --2- 172.21.15.39:0/2813555930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f773010ca10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77240098a0 tx=0x7f7724009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ded123056f08ba server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.073+0000 7f772f7fe700 1 -- 172.21.15.39:0/2813555930 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7724004160 con 0x7f773010a5f0 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.073+0000 7f772f7fe700 1 -- 172.21.15.39:0/2813555930 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7724004300 con 0x7f773010a5f0 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.073+0000 7f772f7fe700 1 -- 172.21.15.39:0/2813555930 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77240119e0 con 0x7f773010a5f0 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 -- 172.21.15.39:0/2813555930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 msgr2=0x7f773010ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 --2- 172.21.15.39:0/2813555930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f773010ca10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77240098a0 tx=0x7f7724009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 -- 172.21.15.39:0/2813555930 shutdown_connections 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 --2- 172.21.15.39:0/2813555930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f773010ca10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 -- 172.21.15.39:0/2813555930 >> 172.21.15.39:0/2813555930 conn(0x7f7730104010 msgr2=0x7f77301064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 -- 172.21.15.39:0/2813555930 shutdown_connections 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 -- 172.21.15.39:0/2813555930 wait complete. 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 Processor -- start 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.074+0000 7f7736b97700 1 -- start start 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f7736b97700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f7730109d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:31.723 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f7736b97700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f773010dd70 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f7734933700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f7730109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f7734933700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f7730109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49704/0 (socket says 172.21.15.39:49704) 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f7734933700 1 -- 172.21.15.39:0/1529350022 learned_addr learned my addr 172.21.15.39:0/1529350022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f7734933700 1 -- 172.21.15.39:0/1529350022 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773010a2b0 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f7734933700 1 --2- 172.21.15.39:0/1529350022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f7730109d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77240116a0 tx=0x7f7724011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f772dffb700 1 -- 172.21.15.39:0/1529350022 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7724003970 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f772dffb700 1 -- 172.21.15.39:0/1529350022 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7724003b10 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7730108500 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.075+0000 7f772dffb700 1 -- 172.21.15.39:0/1529350022 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7724019550 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.076+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77301089e0 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.076+0000 7f772dffb700 1 -- 172.21.15.39:0/1529350022 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7724019b70 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.076+0000 7f772dffb700 1 -- 172.21.15.39:0/1529350022 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f772400b720 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.077+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7730064e20 con 0x7f773010a5f0 2024-05-04T02:59:31.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.079+0000 7f772dffb700 1 -- 172.21.15.39:0/1529350022 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f772401aa50 con 0x7f773010a5f0 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.153+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7730108c60 con 0x7f773010a5f0 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.154+0000 7f772dffb700 1 -- 172.21.15.39:0/1529350022 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f772401a430 con 0x7f773010a5f0 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.155+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 msgr2=0x7f7730109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.155+0000 7f7736b97700 1 --2- 172.21.15.39:0/1529350022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f7730109d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77240116a0 tx=0x7f7724011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.155+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 shutdown_connections 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.155+0000 7f7736b97700 1 --2- 172.21.15.39:0/1529350022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f773010a5f0 0x7f7730109d70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.155+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 >> 172.21.15.39:0/1529350022 conn(0x7f7730104010 msgr2=0x7f77301048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.155+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 shutdown_connections 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:31.156+0000 7f7736b97700 1 -- 172.21.15.39:0/1529350022 wait complete. 2024-05-04T02:59:31.725 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (2/15)... 2024-05-04T02:59:32.440 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:32 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:32.110+0000 7fe91cb0e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T02:59:32.440 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:32 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:32.210+0000 7fe91cb0e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T02:59:32.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:32 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:32.611+0000 7fe91cb0e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T02:59:33.331 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:32 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:32.983+0000 7fe91cb0e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T02:59:33.331 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:33.078+0000 7fe91cb0e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T02:59:33.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:33.330+0000 7fe91cb0e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T02:59:33.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:33.468+0000 7fe91cb0e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T02:59:34.116 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:33.804+0000 7fe91cb0e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T02:59:34.116 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:33.924+0000 7fe91cb0e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T02:59:34.116 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:34.022+0000 7fe91cb0e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T02:59:34.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:34 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/268893855' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-04T02:59:34.828 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:34.668+0000 7fe91cb0e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-04T02:59:34.840 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-04T02:59:34.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-04T02:59:34.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T02:59:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-05-04T02:59:20.441047+0000", 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.127+0000 7f8c82819700 1 Processor -- start 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.127+0000 7f8c82819700 1 -- start start 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.127+0000 7f8c82819700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c10c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.127+0000 7f8c82819700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c07ce50 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.127+0000 7f8c7bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c10c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.127+0000 7f8c7bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c10c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49706/0 (socket says 172.21.15.39:49706) 2024-05-04T02:59:34.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.127+0000 7f8c7bfff700 1 -- 172.21.15.39:0/2027988449 learned_addr learned my addr 172.21.15.39:0/2027988449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.127+0000 7f8c7bfff700 1 -- 172.21.15.39:0/2027988449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c10cf30 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.128+0000 7f8c7bfff700 1 --2- 172.21.15.39:0/2027988449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c10c9f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0098a0 tx=0x7f8c6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98db51dc6a7d242 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.128+0000 7f8c7affd700 1 -- 172.21.15.39:0/2027988449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c6c004160 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.128+0000 7f8c7affd700 1 -- 172.21.15.39:0/2027988449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8c6c004300 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.128+0000 7f8c7affd700 1 -- 172.21.15.39:0/2027988449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c6c0119e0 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.128+0000 7f8c82819700 1 -- 172.21.15.39:0/2027988449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 msgr2=0x7f8c7c10c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.128+0000 7f8c82819700 1 --2- 172.21.15.39:0/2027988449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c10c9f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0098a0 tx=0x7f8c6c009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.129+0000 7f8c82819700 1 -- 172.21.15.39:0/2027988449 shutdown_connections 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.129+0000 7f8c82819700 1 --2- 172.21.15.39:0/2027988449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c10c9f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.129+0000 7f8c82819700 1 -- 172.21.15.39:0/2027988449 >> 172.21.15.39:0/2027988449 conn(0x7f8c7c104010 msgr2=0x7f8c7c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.129+0000 7f8c82819700 1 -- 172.21.15.39:0/2027988449 shutdown_connections 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.129+0000 7f8c82819700 1 -- 172.21.15.39:0/2027988449 wait complete. 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.129+0000 7f8c82819700 1 Processor -- start 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.129+0000 7f8c82819700 1 -- start start 2024-05-04T02:59:34.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c82819700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c82819700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c10dd50 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c7bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c7bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c19dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49714/0 (socket says 172.21.15.39:49714) 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c7bfff700 1 -- 172.21.15.39:0/268893855 learned_addr learned my addr 172.21.15.39:0/268893855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c7bfff700 1 -- 172.21.15.39:0/268893855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c19e270 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c7bfff700 1 --2- 172.21.15.39:0/268893855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c19dd30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0116a0 tx=0x7f8c6c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c797fa700 1 -- 172.21.15.39:0/268893855 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c6c003970 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c797fa700 1 -- 172.21.15.39:0/268893855 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8c6c003b10 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c19e500 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.130+0000 7f8c797fa700 1 -- 172.21.15.39:0/268893855 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c6c019550 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.131+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c1a1100 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.131+0000 7f8c797fa700 1 -- 172.21.15.39:0/268893855 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c6c019bc0 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.131+0000 7f8c797fa700 1 -- 172.21.15.39:0/268893855 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c6c00b720 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.132+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7c064e20 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.134+0000 7f8c797fa700 1 -- 172.21.15.39:0/268893855 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8c6c01aa50 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.208+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8c7c1a12d0 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.208+0000 7f8c797fa700 1 -- 172.21.15.39:0/268893855 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8c6c01a430 con 0x7f8c7c10a5d0 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.209+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 msgr2=0x7f8c7c19dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.209+0000 7f8c82819700 1 --2- 172.21.15.39:0/268893855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c19dd30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0116a0 tx=0x7f8c6c011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.209+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 shutdown_connections 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.209+0000 7f8c82819700 1 --2- 172.21.15.39:0/268893855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c7c10a5d0 0x7f8c7c19dd30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.209+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 >> 172.21.15.39:0/268893855 conn(0x7f8c7c104010 msgr2=0x7f8c7c1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.210+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 shutdown_connections 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:34.210+0000 7f8c82819700 1 -- 172.21.15.39:0/268893855 wait complete. 2024-05-04T02:59:34.847 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (3/15)... 2024-05-04T02:59:35.191 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:34.892+0000 7fe91cb0e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T02:59:35.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: Activating manager daemon x 2024-05-04T02:59:35.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: mgrmap e2: x(active, starting, since 0.00632369s) 2024-05-04T02:59:35.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-04T02:59:35.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-04T02:59:35.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-04T02:59:35.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T02:59:35.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-04T02:59:35.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: Manager daemon x is now available 2024-05-04T02:59:35.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-04T02:59:35.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-04T02:59:35.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' 2024-05-04T02:59:35.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' 2024-05-04T02:59:35.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:35 smithi039 ceph-mon[29715]: from='mgr.14100 172.21.15.39:0/1567466144' entity='mgr.x' 2024-05-04T02:59:37.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:36 smithi039 ceph-mon[29715]: mgrmap e3: x(active, since 1.01255s) 2024-05-04T02:59:38.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:37 smithi039 ceph-mon[29715]: mgrmap e4: x(active, since 2s) 2024-05-04T02:59:38.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:37 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1566735847' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-04T02:59:38.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-05-04T02:59:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-04T02:59:38.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:38.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T02:59:38.201 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-05-04T02:59:20.441047+0000", 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.308+0000 7f88cac15700 1 Processor -- start 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.308+0000 7f88cac15700 1 -- start start 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.308+0000 7f88cac15700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c410ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.308+0000 7f88cac15700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88c407d720 con 0x7f88c410a5f0 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.308+0000 7f88c89b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c410ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.308+0000 7f88c89b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c410ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36330/0 (socket says 172.21.15.39:36330) 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.308+0000 7f88c89b1700 1 -- 172.21.15.39:0/4001998108 learned_addr learned my addr 172.21.15.39:0/4001998108 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.308+0000 7f88c89b1700 1 -- 172.21.15.39:0/4001998108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c410cf50 con 0x7f88c410a5f0 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88c89b1700 1 --2- 172.21.15.39:0/4001998108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c410ca10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f88b80098a0 tx=0x7f88b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16587d84bdcfb005 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88c37fe700 1 -- 172.21.15.39:0/4001998108 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88b8004160 con 0x7f88c410a5f0 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88c37fe700 1 -- 172.21.15.39:0/4001998108 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f88b8004300 con 0x7f88c410a5f0 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88c37fe700 1 -- 172.21.15.39:0/4001998108 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88b80119e0 con 0x7f88c410a5f0 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88cac15700 1 -- 172.21.15.39:0/4001998108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 msgr2=0x7f88c410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88cac15700 1 --2- 172.21.15.39:0/4001998108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c410ca10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f88b80098a0 tx=0x7f88b8009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88cac15700 1 -- 172.21.15.39:0/4001998108 shutdown_connections 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88cac15700 1 --2- 172.21.15.39:0/4001998108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c410ca10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.309+0000 7f88cac15700 1 -- 172.21.15.39:0/4001998108 >> 172.21.15.39:0/4001998108 conn(0x7f88c4104050 msgr2=0x7f88c4106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.310+0000 7f88cac15700 1 -- 172.21.15.39:0/4001998108 shutdown_connections 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.310+0000 7f88cac15700 1 -- 172.21.15.39:0/4001998108 wait complete. 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.310+0000 7f88cac15700 1 Processor -- start 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.310+0000 7f88cac15700 1 -- start start 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88cac15700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88cac15700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88c410ddd0 con 0x7f88c410a5f0 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88c89b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88c89b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36340/0 (socket says 172.21.15.39:36340) 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88c89b1700 1 -- 172.21.15.39:0/1566735847 learned_addr learned my addr 172.21.15.39:0/1566735847 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88c89b1700 1 -- 172.21.15.39:0/1566735847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c419e310 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88c89b1700 1 --2- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c419ddd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f88b80116a0 tx=0x7f88b8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88c1ffb700 1 -- 172.21.15.39:0/1566735847 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88b80039f0 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88c1ffb700 1 -- 172.21.15.39:0/1566735847 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f88b8003b90 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c419e5a0 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.311+0000 7f88c1ffb700 1 -- 172.21.15.39:0/1566735847 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88b801a600 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.312+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c41a1170 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.313+0000 7f88c1ffb700 1 -- 172.21.15.39:0/1566735847 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f88b801a900 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.313+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c4064e20 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.313+0000 7f88c1ffb700 1 --2- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f88b403e3d0 0x7f88b40408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.313+0000 7f88c1ffb700 1 -- 172.21.15.39:0/1566735847 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f88b8053d40 con 0x7f88c410a5f0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.313+0000 7f88c3fff700 1 --2- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f88b403e3d0 0x7f88b40408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:38.204 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.313+0000 7f88c3fff700 1 --2- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f88b403e3d0 0x7f88b40408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f88b0006fd0 tx=0x7f88b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.318+0000 7f88c1ffb700 1 -- 172.21.15.39:0/1566735847 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88b8053760 con 0x7f88c410a5f0 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.629+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f88c41a1890 con 0x7f88c410a5f0 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.630+0000 7f88c1ffb700 1 -- 172.21.15.39:0/1566735847 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f88b8024770 con 0x7f88c410a5f0 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.633+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f88b403e3d0 msgr2=0x7f88b40408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.633+0000 7f88cac15700 1 --2- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f88b403e3d0 0x7f88b40408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f88b0006fd0 tx=0x7f88b0006e40 comp rx=0 tx=0).stop 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.633+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 msgr2=0x7f88c419ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.633+0000 7f88cac15700 1 --2- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c419ddd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f88b80116a0 tx=0x7f88b8011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.633+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 shutdown_connections 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.633+0000 7f88cac15700 1 --2- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f88b403e3d0 0x7f88b40408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.633+0000 7f88cac15700 1 --2- 172.21.15.39:0/1566735847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88c410a5f0 0x7f88c419ddd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.633+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 >> 172.21.15.39:0/1566735847 conn(0x7f88c4104050 msgr2=0x7f88c4104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.634+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 shutdown_connections 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:37.634+0000 7f88cac15700 1 -- 172.21.15.39:0/1566735847 wait complete. 2024-05-04T02:59:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:mgr is available 2024-05-04T02:59:39.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:38 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2328915607' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-04T02:59:39.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:39.395 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-04T02:59:39.396 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.607+0000 7faac2ce5700 1 Processor -- start 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.607+0000 7faac2ce5700 1 -- start start 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.608+0000 7faac2ce5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc108e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.608+0000 7faac2ce5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faabc109400 con 0x7faabc108a00 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.608+0000 7faac0a81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc108e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.608+0000 7faac0a81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc108e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36344/0 (socket says 172.21.15.39:36344) 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.608+0000 7faac0a81700 1 -- 172.21.15.39:0/3878265485 learned_addr learned my addr 172.21.15.39:0/3878265485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.608+0000 7faac0a81700 1 -- 172.21.15.39:0/3878265485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc109bd0 con 0x7faabc108a00 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.608+0000 7faac0a81700 1 --2- 172.21.15.39:0/3878265485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc108e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faab00098a0 tx=0x7faab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d58d2f14f9fe43f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.608+0000 7faabb7fe700 1 -- 172.21.15.39:0/3878265485 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab0004160 con 0x7faabc108a00 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faabb7fe700 1 -- 172.21.15.39:0/3878265485 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7faab0004300 con 0x7faabc108a00 2024-05-04T02:59:39.397 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faabb7fe700 1 -- 172.21.15.39:0/3878265485 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab00119e0 con 0x7faabc108a00 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faac2ce5700 1 -- 172.21.15.39:0/3878265485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 msgr2=0x7faabc108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faac2ce5700 1 --2- 172.21.15.39:0/3878265485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc108e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faab00098a0 tx=0x7faab0009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faac2ce5700 1 -- 172.21.15.39:0/3878265485 shutdown_connections 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faac2ce5700 1 --2- 172.21.15.39:0/3878265485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc108e10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faac2ce5700 1 -- 172.21.15.39:0/3878265485 >> 172.21.15.39:0/3878265485 conn(0x7faabc104020 msgr2=0x7faabc1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faac2ce5700 1 -- 172.21.15.39:0/3878265485 shutdown_connections 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.609+0000 7faac2ce5700 1 -- 172.21.15.39:0/3878265485 wait complete. 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.610+0000 7faac2ce5700 1 Processor -- start 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.610+0000 7faac2ce5700 1 -- start start 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.610+0000 7faac2ce5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.610+0000 7faac2ce5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faabc10ce00 con 0x7faabc108a00 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.610+0000 7faac0a81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.610+0000 7faac0a81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36360/0 (socket says 172.21.15.39:36360) 2024-05-04T02:59:39.398 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.610+0000 7faac0a81700 1 -- 172.21.15.39:0/2328915607 learned_addr learned my addr 172.21.15.39:0/2328915607 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.611+0000 7faac0a81700 1 -- 172.21.15.39:0/2328915607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc19d200 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.611+0000 7faac0a81700 1 --2- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc19ccc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faab00046e0 tx=0x7faab0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.611+0000 7faab9ffb700 1 -- 172.21.15.39:0/2328915607 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab00039e0 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.611+0000 7faab9ffb700 1 -- 172.21.15.39:0/2328915607 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7faab0003b80 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.611+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc19d490 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.611+0000 7faab9ffb700 1 -- 172.21.15.39:0/2328915607 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab0024700 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.611+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc1a00a0 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.612+0000 7faab9ffb700 1 -- 172.21.15.39:0/2328915607 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7faab0022020 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.612+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc064e20 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.612+0000 7faab9ffb700 1 --2- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7faaac03e420 0x7faaac040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.613+0000 7faab9ffb700 1 -- 172.21.15.39:0/2328915607 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faab00536f0 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.613+0000 7faabbfff700 1 --2- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7faaac03e420 0x7faaac040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.613+0000 7faabbfff700 1 --2- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7faaac03e420 0x7faaac040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.618+0000 7faab9ffb700 1 -- 172.21.15.39:0/2328915607 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faab001a910 con 0x7faabc108a00 2024-05-04T02:59:39.399 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.829+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7faabc1a18a0 con 0x7faabc108a00 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.830+0000 7faab9ffb700 1 -- 172.21.15.39:0/2328915607 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7faab0053450 con 0x7faabc108a00 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.833+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7faaac03e420 msgr2=0x7faaac040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.833+0000 7faac2ce5700 1 --2- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7faaac03e420 0x7faaac040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8006e40 comp rx=0 tx=0).stop 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.833+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 msgr2=0x7faabc19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.833+0000 7faac2ce5700 1 --2- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc19ccc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faab00046e0 tx=0x7faab0011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.834+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 shutdown_connections 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.834+0000 7faac2ce5700 1 --2- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7faaac03e420 0x7faaac040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.834+0000 7faac2ce5700 1 --2- 172.21.15.39:0/2328915607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faabc108a00 0x7faabc19ccc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.834+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 >> 172.21.15.39:0/2328915607 conn(0x7faabc104020 msgr2=0x7faabc104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.834+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 shutdown_connections 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:38.834+0000 7faac2ce5700 1 -- 172.21.15.39:0/2328915607 wait complete. 2024-05-04T02:59:39.400 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling cephadm module... 2024-05-04T02:59:40.379 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:40 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3939664930' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-04T02:59:40.380 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: ignoring --setuser ceph since I am not root 2024-05-04T02:59:40.380 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: ignoring --setgroup ceph since I am not root 2024-05-04T02:59:40.380 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:40.379+0000 7f8a78eff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T02:59:40.681 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:40.573+0000 7f8a78eff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T02:59:40.681 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:40.681+0000 7f8a78eff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T02:59:40.788 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.830+0000 7f9265337700 1 Processor -- start 2024-05-04T02:59:40.788 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.830+0000 7f9265337700 1 -- start start 2024-05-04T02:59:40.788 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f9265337700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926010c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:40.788 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f9265337700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f926010cce0 con 0x7f926007d550 2024-05-04T02:59:40.788 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f925effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926010c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:40.788 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f925effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926010c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36368/0 (socket says 172.21.15.39:36368) 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f925effd700 1 -- 172.21.15.39:0/809542557 learned_addr learned my addr 172.21.15.39:0/809542557 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f925effd700 1 -- 172.21.15.39:0/809542557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926010ce60 con 0x7f926007d550 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f925effd700 1 --2- 172.21.15.39:0/809542557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926010c6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f92500098a0 tx=0x7f9250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b6e427e5a1caf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f925dffb700 1 -- 172.21.15.39:0/809542557 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9250004160 con 0x7f926007d550 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f925dffb700 1 -- 172.21.15.39:0/809542557 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9250004300 con 0x7f926007d550 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.831+0000 7f925dffb700 1 -- 172.21.15.39:0/809542557 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92500119e0 con 0x7f926007d550 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.832+0000 7f9265337700 1 -- 172.21.15.39:0/809542557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 msgr2=0x7f926010c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.832+0000 7f9265337700 1 --2- 172.21.15.39:0/809542557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926010c6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f92500098a0 tx=0x7f9250009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.832+0000 7f9265337700 1 -- 172.21.15.39:0/809542557 shutdown_connections 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.832+0000 7f9265337700 1 --2- 172.21.15.39:0/809542557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926010c6f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.832+0000 7f9265337700 1 -- 172.21.15.39:0/809542557 >> 172.21.15.39:0/809542557 conn(0x7f92601040c0 msgr2=0x7f9260106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.832+0000 7f9265337700 1 -- 172.21.15.39:0/809542557 shutdown_connections 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.832+0000 7f9265337700 1 -- 172.21.15.39:0/809542557 wait complete. 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.833+0000 7f9265337700 1 Processor -- start 2024-05-04T02:59:40.789 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.833+0000 7f9265337700 1 -- start start 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.833+0000 7f9265337700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.833+0000 7f9265337700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f926010dda0 con 0x7f926007d550 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f925effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f925effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926019dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36374/0 (socket says 172.21.15.39:36374) 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f925effd700 1 -- 172.21.15.39:0/3939664930 learned_addr learned my addr 172.21.15.39:0/3939664930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f925effd700 1 -- 172.21.15.39:0/3939664930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926019e2c0 con 0x7f926007d550 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f925effd700 1 --2- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926019dd80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f92500116a0 tx=0x7f9250011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f9247fff700 1 -- 172.21.15.39:0/3939664930 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9250003970 con 0x7f926007d550 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f9247fff700 1 -- 172.21.15.39:0/3939664930 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9250003b10 con 0x7f926007d550 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f9247fff700 1 -- 172.21.15.39:0/3939664930 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9250019550 con 0x7f926007d550 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926019e550 con 0x7f926007d550 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.834+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92601a1160 con 0x7f926007d550 2024-05-04T02:59:40.790 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.835+0000 7f9247fff700 1 -- 172.21.15.39:0/3939664930 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f9250019810 con 0x7f926007d550 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.835+0000 7f9247fff700 1 --2- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f924803e420 0x7f9248040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.835+0000 7f9247fff700 1 -- 172.21.15.39:0/3939664930 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f92500546d0 con 0x7f926007d550 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.835+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9260064e20 con 0x7f926007d550 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.836+0000 7f925e7fc700 1 --2- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f924803e420 0x7f9248040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.836+0000 7f925e7fc700 1 --2- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f924803e420 0x7f9248040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9254006fd0 tx=0x7f9254006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:39.841+0000 7f9247fff700 1 -- 172.21.15.39:0/3939664930 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f925002adc0 con 0x7f926007d550 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.118+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f92601a1730 con 0x7f926007d550 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.170+0000 7f9247fff700 1 -- 172.21.15.39:0/3939664930 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f92500540f0 con 0x7f926007d550 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.172+0000 7f9247fff700 1 -- 172.21.15.39:0/3939664930 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f9250010d70 con 0x7f926007d550 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f924803e420 msgr2=0x7f9248040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 --2- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f924803e420 0x7f9248040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9254006fd0 tx=0x7f9254006e40 comp rx=0 tx=0).stop 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 msgr2=0x7f926019dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 --2- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926019dd80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f92500116a0 tx=0x7f9250011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:40.791 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 shutdown_connections 2024-05-04T02:59:40.792 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 --2- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f924803e420 0x7f9248040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:40.792 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 --2- 172.21.15.39:0/3939664930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926007d550 0x7f926019dd80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:40.792 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 >> 172.21.15.39:0/3939664930 conn(0x7f92601040c0 msgr2=0x7f9260104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:40.792 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.173+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 shutdown_connections 2024-05-04T02:59:40.792 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:40.174+0000 7f9265337700 1 -- 172.21.15.39:0/3939664930 wait complete. 2024-05-04T02:59:41.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:41 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3939664930' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-04T02:59:41.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:41 smithi039 ceph-mon[29715]: mgrmap e5: x(active, since 5s) 2024-05-04T02:59:41.421 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:41 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:41.009+0000 7f8a78eff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T02:59:41.421 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:41 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:41.098+0000 7f8a78eff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T02:59:42.022 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-04T02:59:42.022 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.213+0000 7fccef79d700 1 Processor -- start 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.213+0000 7fccef79d700 1 -- start start 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.213+0000 7fccef79d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce810b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.213+0000 7fccef79d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcce810bab0 con 0x7fcce810b0d0 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.213+0000 7fcced539700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce810b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.213+0000 7fcced539700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce810b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36388/0 (socket says 172.21.15.39:36388) 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.213+0000 7fcced539700 1 -- 172.21.15.39:0/738579943 learned_addr learned my addr 172.21.15.39:0/738579943 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.214+0000 7fcced539700 1 -- 172.21.15.39:0/738579943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce810bc30 con 0x7fcce810b0d0 2024-05-04T02:59:42.023 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.214+0000 7fcced539700 1 --2- 172.21.15.39:0/738579943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce810b4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcce40098a0 tx=0x7fcce4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcd412dd1b12888 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.214+0000 7fccdffff700 1 -- 172.21.15.39:0/738579943 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcce4004160 con 0x7fcce810b0d0 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.214+0000 7fccdffff700 1 -- 172.21.15.39:0/738579943 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcce4004300 con 0x7fcce810b0d0 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.214+0000 7fccdffff700 1 -- 172.21.15.39:0/738579943 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcce40119e0 con 0x7fcce810b0d0 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.215+0000 7fccef79d700 1 -- 172.21.15.39:0/738579943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 msgr2=0x7fcce810b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.215+0000 7fccef79d700 1 --2- 172.21.15.39:0/738579943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce810b4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcce40098a0 tx=0x7fcce4009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.215+0000 7fccef79d700 1 -- 172.21.15.39:0/738579943 shutdown_connections 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.215+0000 7fccef79d700 1 --2- 172.21.15.39:0/738579943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce810b4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.215+0000 7fccef79d700 1 -- 172.21.15.39:0/738579943 >> 172.21.15.39:0/738579943 conn(0x7fcce8103790 msgr2=0x7fcce8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.215+0000 7fccef79d700 1 -- 172.21.15.39:0/738579943 shutdown_connections 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.215+0000 7fccef79d700 1 -- 172.21.15.39:0/738579943 wait complete. 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.216+0000 7fccef79d700 1 Processor -- start 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.216+0000 7fccef79d700 1 -- start start 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.216+0000 7fccef79d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:42.024 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.216+0000 7fccef79d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8108840 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.216+0000 7fcced539700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.216+0000 7fcced539700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce819dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36390/0 (socket says 172.21.15.39:36390) 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.216+0000 7fcced539700 1 -- 172.21.15.39:0/97157052 learned_addr learned my addr 172.21.15.39:0/97157052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.216+0000 7fcced539700 1 -- 172.21.15.39:0/97157052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce819e280 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.217+0000 7fcced539700 1 --2- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce819dd40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcce40116a0 tx=0x7fcce4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.217+0000 7fccde7fc700 1 -- 172.21.15.39:0/97157052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcce4003970 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.217+0000 7fccde7fc700 1 -- 172.21.15.39:0/97157052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcce4003b10 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.217+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce819e510 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.217+0000 7fccde7fc700 1 -- 172.21.15.39:0/97157052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcce4019550 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.217+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce81a1110 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.218+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce8064e20 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.218+0000 7fccde7fc700 1 -- 172.21.15.39:0/97157052 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fcce40197c0 con 0x7fcce810b0d0 2024-05-04T02:59:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.218+0000 7fccde7fc700 1 --2- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7fccd403e4c0 0x7fccd40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.218+0000 7fccde7fc700 1 -- 172.21.15.39:0/97157052 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcce40538a0 con 0x7fcce810b0d0 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.218+0000 7fccecd38700 1 -- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7fccd403e4c0 msgr2=0x7fccd40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3124453164 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.218+0000 7fccecd38700 1 --2- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7fccd403e4c0 0x7fccd40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.224+0000 7fccde7fc700 1 -- 172.21.15.39:0/97157052 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcce4016260 con 0x7fcce810b0d0 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.420+0000 7fccecd38700 1 -- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7fccd403e4c0 msgr2=0x7fccd40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3124453164 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.420+0000 7fccecd38700 1 --2- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7fccd403e4c0 0x7fccd40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.511+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcce81a12e0 con 0x7fcce810b0d0 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.512+0000 7fccde7fc700 1 -- 172.21.15.39:0/97157052 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcce40532c0 con 0x7fcce810b0d0 2024-05-04T02:59:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.516+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7fccd403e4c0 msgr2=0x7fccd40409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.516+0000 7fccef79d700 1 --2- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7fccd403e4c0 0x7fccd40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.516+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 msgr2=0x7fcce819dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.516+0000 7fccef79d700 1 --2- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce819dd40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcce40116a0 tx=0x7fcce4011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.516+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 shutdown_connections 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.516+0000 7fccef79d700 1 --2- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7fccd403e4c0 0x7fccd40409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.516+0000 7fccef79d700 1 --2- 172.21.15.39:0/97157052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcce810b0d0 0x7fcce819dd40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.516+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 >> 172.21.15.39:0/97157052 conn(0x7fcce8103790 msgr2=0x7fcce8104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.517+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 shutdown_connections 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:41.517+0000 7fccef79d700 1 -- 172.21.15.39:0/97157052 wait complete. 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-05-04T02:59:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 5... 2024-05-04T02:59:42.398 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:42 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/97157052' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-04T02:59:42.399 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:42 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:42.375+0000 7f8a78eff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T02:59:42.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:42 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:42.564+0000 7f8a78eff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T02:59:43.059 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:42 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:42.748+0000 7f8a78eff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T02:59:43.059 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:43.059+0000 7f8a78eff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T02:59:43.369 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:43.156+0000 7f8a78eff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T02:59:43.369 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:43.369+0000 7f8a78eff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T02:59:43.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:43.573+0000 7f8a78eff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T02:59:44.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:44.201+0000 7f8a78eff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T02:59:44.695 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:44.433+0000 7f8a78eff200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T02:59:45.190 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:44.877+0000 7f8a78eff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T02:59:45.190 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:44.978+0000 7f8a78eff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T02:59:45.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:45.363+0000 7f8a78eff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T02:59:46.089 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:45.737+0000 7f8a78eff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T02:59:46.089 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:45.834+0000 7f8a78eff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T02:59:46.440 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:46.088+0000 7f8a78eff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T02:59:46.440 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:46.228+0000 7f8a78eff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T02:59:46.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:46.544+0000 7f8a78eff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T02:59:46.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:46.665+0000 7f8a78eff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T02:59:46.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:46.761+0000 7f8a78eff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T02:59:47.674 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:47 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:47.392+0000 7f8a78eff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T02:59:47.674 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:47 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T02:59:47.620+0000 7f8a78eff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T02:59:47.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: Active manager daemon x restarted 2024-05-04T02:59:47.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: Activating manager daemon x 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: osdmap e2: 0 total, 0 up, 0 in 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: mgrmap e6: x(active, starting, since 0.00417669s) 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: Manager daemon x is now available 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T02:59:47.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:47 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T02:59:48.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:48 smithi039 ceph-mon[29715]: Found migration_current of "None". Setting to last migration. 2024-05-04T02:59:48.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:48 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-04T02:59:48.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:48 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-04T02:59:48.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:48 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T02:59:48.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:48 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T02:59:48.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:48 smithi039 ceph-mon[29715]: mgrmap e7: x(active, since 1.00678s) 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.514+0000 7f2509228700 1 Processor -- start 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.514+0000 7f2509228700 1 -- start start 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.514+0000 7f2509228700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250410ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.514+0000 7f2509228700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f250407d6e0 con 0x7f250410a620 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.514+0000 7f2502d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250410ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.514+0000 7f2502d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250410ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36400/0 (socket says 172.21.15.39:36400) 2024-05-04T02:59:49.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.514+0000 7f2502d9d700 1 -- 172.21.15.39:0/4174093033 learned_addr learned my addr 172.21.15.39:0/4174093033 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.515+0000 7f2502d9d700 1 -- 172.21.15.39:0/4174093033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250410cf80 con 0x7f250410a620 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.515+0000 7f2502d9d700 1 --2- 172.21.15.39:0/4174093033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250410ca40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24f40098a0 tx=0x7f24f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a69dd6a2ee89191 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.515+0000 7f2501d9b700 1 -- 172.21.15.39:0/4174093033 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f4004160 con 0x7f250410a620 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.515+0000 7f2501d9b700 1 -- 172.21.15.39:0/4174093033 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f24f4004300 con 0x7f250410a620 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.515+0000 7f2501d9b700 1 -- 172.21.15.39:0/4174093033 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f40119e0 con 0x7f250410a620 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.516+0000 7f2509228700 1 -- 172.21.15.39:0/4174093033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 msgr2=0x7f250410ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.516+0000 7f2509228700 1 --2- 172.21.15.39:0/4174093033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250410ca40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24f40098a0 tx=0x7f24f4009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.516+0000 7f2509228700 1 -- 172.21.15.39:0/4174093033 shutdown_connections 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.516+0000 7f2509228700 1 --2- 172.21.15.39:0/4174093033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250410ca40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.516+0000 7f2509228700 1 -- 172.21.15.39:0/4174093033 >> 172.21.15.39:0/4174093033 conn(0x7f2504104020 msgr2=0x7f25041064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.516+0000 7f2509228700 1 -- 172.21.15.39:0/4174093033 shutdown_connections 2024-05-04T02:59:49.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.516+0000 7f2509228700 1 -- 172.21.15.39:0/4174093033 wait complete. 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.517+0000 7f2509228700 1 Processor -- start 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.517+0000 7f2509228700 1 -- start start 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.518+0000 7f2509228700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.518+0000 7f2509228700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f250410dda0 con 0x7f250410a620 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.518+0000 7f2502d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.518+0000 7f2502d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250419dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36406/0 (socket says 172.21.15.39:36406) 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.518+0000 7f2502d9d700 1 -- 172.21.15.39:0/1374199542 learned_addr learned my addr 172.21.15.39:0/1374199542 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.518+0000 7f2502d9d700 1 -- 172.21.15.39:0/1374199542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250419e1b0 con 0x7f250410a620 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.518+0000 7f2502d9d700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250419dc70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f24f40116a0 tx=0x7f24f4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.519+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f4003970 con 0x7f250410a620 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.519+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f24f4003b10 con 0x7f250410a620 2024-05-04T02:59:49.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.519+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250419e440 con 0x7f250410a620 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.519+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f4019550 con 0x7f250410a620 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.519+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25041a1050 con 0x7f250410a620 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.520+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f24f4019810 con 0x7f250410a620 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.520+0000 7f24ebfff700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 0x7f24ec040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.520+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 --> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f24ec041150 con 0x7f24ec03e530 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.520+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f24f40546e0 con 0x7f250410a620 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.520+0000 7f250259c700 1 -- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 msgr2=0x7f24ec040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3124453164 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.520+0000 7f250259c700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 0x7f24ec040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.721+0000 7f250259c700 1 -- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 msgr2=0x7f24ec040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3124453164 2024-05-04T02:59:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:42.721+0000 7f250259c700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 0x7f24ec040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:43.122+0000 7f250259c700 1 -- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 msgr2=0x7f24ec040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3124453164 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:43.122+0000 7f250259c700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 0x7f24ec040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:43.923+0000 7f250259c700 1 -- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 msgr2=0x7f24ec040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3124453164 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:43.923+0000 7f250259c700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 0x7f24ec040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:45.524+0000 7f250259c700 1 -- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 msgr2=0x7f24ec040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3124453164 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:45.524+0000 7f250259c700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 0x7f24ec040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:47.624+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f24f4019b50 con 0x7f250410a620 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:47.624+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 msgr2=0x7f24ec040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:47.624+0000 7f24ebfff700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 0x7f24ec040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.626+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f24f40554c0 con 0x7f250410a620 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.626+0000 7f24ebfff700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f24ec041e70 0x7f24ec044290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.626+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f24f4025e80 con 0x7f24ec041e70 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.627+0000 7f250259c700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f24ec041e70 0x7f24ec044290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.628+0000 7f250259c700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f24ec041e70 0x7f24ec044290 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f24f8003a10 tx=0x7f24f80092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.629+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mgr.14120 v2:172.21.15.39:6800/3594679658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f24f4025e80 con 0x7f24ec041e70 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.637+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2504064e20 con 0x7f24ec041e70 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.637+0000 7f24ebfff700 1 -- 172.21.15.39:0/1374199542 <== mgr.14120 v2:172.21.15.39:6800/3594679658 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f2504064e20 con 0x7f24ec041e70 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.637+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f24ec041e70 msgr2=0x7f24ec044290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.637+0000 7f2509228700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f24ec041e70 0x7f24ec044290 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f24f8003a10 tx=0x7f24f80092b0 comp rx=0 tx=0).stop 2024-05-04T02:59:49.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.637+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 msgr2=0x7f250419dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.637+0000 7f2509228700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250419dc70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f24f40116a0 tx=0x7f24f4011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.638+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 shutdown_connections 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.638+0000 7f2509228700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f24ec041e70 0x7f24ec044290 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.638+0000 7f2509228700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f250410a620 0x7f250419dc70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.638+0000 7f2509228700 1 --2- 172.21.15.39:0/1374199542 >> [v2:172.21.15.39:6800/3124453164,v1:172.21.15.39:6801/3124453164] conn(0x7f24ec03e530 0x7f24ec040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.638+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 >> 172.21.15.39:0/1374199542 conn(0x7f2504104020 msgr2=0x7f2504104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.638+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 shutdown_connections 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:48.638+0000 7f2509228700 1 -- 172.21.15.39:0/1374199542 wait complete. 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 5 is available 2024-05-04T02:59:49.177 INFO:teuthology.orchestra.run.smithi039.stdout:Setting orchestrator backend to cephadm... 2024-05-04T02:59:49.847 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:49 smithi039 ceph-mon[29715]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-04T02:59:49.847 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:49 smithi039 ceph-mon[29715]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.594+0000 7fca61dab700 1 Processor -- start 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.594+0000 7fca61dab700 1 -- start start 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.594+0000 7fca61dab700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c10c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.594+0000 7fca61dab700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c07ce50 con 0x7fca5c10a5d0 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.595+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c10c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.595+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c10c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38694/0 (socket says 172.21.15.39:38694) 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.595+0000 7fca5b7fe700 1 -- 172.21.15.39:0/913472306 learned_addr learned my addr 172.21.15.39:0/913472306 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.595+0000 7fca5b7fe700 1 -- 172.21.15.39:0/913472306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c10cf30 con 0x7fca5c10a5d0 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.595+0000 7fca5b7fe700 1 --2- 172.21.15.39:0/913472306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c10c9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fca5000f960 tx=0x7fca5000fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df5b47de46513fe server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:50.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.595+0000 7fca5a7fc700 1 -- 172.21.15.39:0/913472306 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca5001c070 con 0x7fca5c10a5d0 2024-05-04T02:59:50.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.595+0000 7fca5a7fc700 1 -- 172.21.15.39:0/913472306 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fca50004030 con 0x7fca5c10a5d0 2024-05-04T02:59:50.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.595+0000 7fca5a7fc700 1 -- 172.21.15.39:0/913472306 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca50003be0 con 0x7fca5c10a5d0 2024-05-04T02:59:50.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.596+0000 7fca61dab700 1 -- 172.21.15.39:0/913472306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 msgr2=0x7fca5c10c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:50.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.596+0000 7fca61dab700 1 --2- 172.21.15.39:0/913472306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c10c9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fca5000f960 tx=0x7fca5000fc70 comp rx=0 tx=0).stop 2024-05-04T02:59:50.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.596+0000 7fca61dab700 1 -- 172.21.15.39:0/913472306 shutdown_connections 2024-05-04T02:59:50.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.596+0000 7fca61dab700 1 --2- 172.21.15.39:0/913472306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c10c9f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:50.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.596+0000 7fca61dab700 1 -- 172.21.15.39:0/913472306 >> 172.21.15.39:0/913472306 conn(0x7fca5c104010 msgr2=0x7fca5c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:50.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.596+0000 7fca61dab700 1 -- 172.21.15.39:0/913472306 shutdown_connections 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.596+0000 7fca61dab700 1 -- 172.21.15.39:0/913472306 wait complete. 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.597+0000 7fca61dab700 1 Processor -- start 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.597+0000 7fca61dab700 1 -- start start 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca61dab700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca61dab700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c109010 con 0x7fca5c10a5d0 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38708/0 (socket says 172.21.15.39:38708) 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca5b7fe700 1 -- 172.21.15.39:0/1835259329 learned_addr learned my addr 172.21.15.39:0/1835259329 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca5b7fe700 1 -- 172.21.15.39:0/1835259329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c19c070 con 0x7fca5c10a5d0 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca5b7fe700 1 --2- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c19bb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca50004410 tx=0x7fca500044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca58ff9700 1 -- 172.21.15.39:0/1835259329 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca5001c040 con 0x7fca5c10a5d0 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca58ff9700 1 -- 172.21.15.39:0/1835259329 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fca50025070 con 0x7fca5c10a5d0 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.598+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c19c300 con 0x7fca5c10a5d0 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.599+0000 7fca58ff9700 1 -- 172.21.15.39:0/1835259329 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca500206a0 con 0x7fca5c10a5d0 2024-05-04T02:59:50.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.599+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c19ef10 con 0x7fca5c10a5d0 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.600+0000 7fca58ff9700 1 -- 172.21.15.39:0/1835259329 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fca500208f0 con 0x7fca5c10a5d0 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.600+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5c19c780 con 0x7fca5c10a5d0 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.600+0000 7fca58ff9700 1 --2- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fca4403e340 0x7fca44040850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.600+0000 7fca58ff9700 1 -- 172.21.15.39:0/1835259329 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fca50014070 con 0x7fca5c10a5d0 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.600+0000 7fca5affd700 1 --2- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fca4403e340 0x7fca44040850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.600+0000 7fca5affd700 1 --2- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fca4403e340 0x7fca44040850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006fd0 tx=0x7fca4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.605+0000 7fca58ff9700 1 -- 172.21.15.39:0/1835259329 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca50014dd0 con 0x7fca5c10a5d0 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.677+0000 7fca58ff9700 1 -- 172.21.15.39:0/1835259329 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fca50017b00 con 0x7fca5c10a5d0 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.846+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fca5c02e900 con 0x7fca4403e340 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.852+0000 7fca58ff9700 1 -- 172.21.15.39:0/1835259329 <== mgr.14120 v2:172.21.15.39:6800/3594679658 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca5c02e900 con 0x7fca4403e340 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.854+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fca4403e340 msgr2=0x7fca44040850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.854+0000 7fca61dab700 1 --2- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fca4403e340 0x7fca44040850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006fd0 tx=0x7fca4c006e40 comp rx=0 tx=0).stop 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.855+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 msgr2=0x7fca5c19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.855+0000 7fca61dab700 1 --2- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c19bb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca50004410 tx=0x7fca500044f0 comp rx=0 tx=0).stop 2024-05-04T02:59:50.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.855+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 shutdown_connections 2024-05-04T02:59:50.381 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.855+0000 7fca61dab700 1 --2- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fca4403e340 0x7fca44040850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:50.381 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.855+0000 7fca61dab700 1 --2- 172.21.15.39:0/1835259329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca5c10a5d0 0x7fca5c19bb30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:50.381 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.855+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 >> 172.21.15.39:0/1835259329 conn(0x7fca5c104010 msgr2=0x7fca5c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:50.381 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.855+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 shutdown_connections 2024-05-04T02:59:50.381 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:49.855+0000 7fca61dab700 1 -- 172.21.15.39:0/1835259329 wait complete. 2024-05-04T02:59:50.761 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: [04/May/2024:02:59:49] ENGINE Bus STARTING 2024-05-04T02:59:50.761 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: mgrmap e8: x(active, since 2s) 2024-05-04T02:59:50.761 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: [04/May/2024:02:59:49] ENGINE Serving on http://172.21.15.39:8765 2024-05-04T02:59:50.761 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T02:59:50.761 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T02:59:50.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T02:59:50.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: [04/May/2024:02:59:49] ENGINE Serving on https://172.21.15.39:7150 2024-05-04T02:59:50.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: [04/May/2024:02:59:49] ENGINE Bus STARTED 2024-05-04T02:59:50.762 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:50 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.800+0000 7f1fedbd6700 1 Processor -- start 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.801+0000 7f1fedbd6700 1 -- start start 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.801+0000 7f1fedbd6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe8109f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.801+0000 7f1fedbd6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe810a500 con 0x7f1fe8109b00 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.801+0000 7f1fe77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe8109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.801+0000 7f1fe77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe8109f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38714/0 (socket says 172.21.15.39:38714) 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.801+0000 7f1fe77fe700 1 -- 172.21.15.39:0/3876866409 learned_addr learned my addr 172.21.15.39:0/3876866409 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.801+0000 7f1fe77fe700 1 -- 172.21.15.39:0/3876866409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe810acd0 con 0x7f1fe8109b00 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.801+0000 7f1fe77fe700 1 --2- 172.21.15.39:0/3876866409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe8109f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098a0 tx=0x7f1fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51ca5a1030c0a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:51.607 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fe67fc700 1 -- 172.21.15.39:0/3876866409 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd8004160 con 0x7f1fe8109b00 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fe67fc700 1 -- 172.21.15.39:0/3876866409 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1fd8004300 con 0x7f1fe8109b00 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fe67fc700 1 -- 172.21.15.39:0/3876866409 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd80119e0 con 0x7f1fe8109b00 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fedbd6700 1 -- 172.21.15.39:0/3876866409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 msgr2=0x7f1fe8109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fedbd6700 1 --2- 172.21.15.39:0/3876866409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe8109f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098a0 tx=0x7f1fd8009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fedbd6700 1 -- 172.21.15.39:0/3876866409 shutdown_connections 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fedbd6700 1 --2- 172.21.15.39:0/3876866409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe8109f10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fedbd6700 1 -- 172.21.15.39:0/3876866409 >> 172.21.15.39:0/3876866409 conn(0x7f1fe8105140 msgr2=0x7f1fe81075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fedbd6700 1 -- 172.21.15.39:0/3876866409 shutdown_connections 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.802+0000 7f1fedbd6700 1 -- 172.21.15.39:0/3876866409 wait complete. 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.803+0000 7f1fedbd6700 1 Processor -- start 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.803+0000 7f1fedbd6700 1 -- start start 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.803+0000 7f1fedbd6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.803+0000 7f1fedbd6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe810df00 con 0x7f1fe8109b00 2024-05-04T02:59:51.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fe77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fe77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38724/0 (socket says 172.21.15.39:38724) 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fe77fe700 1 -- 172.21.15.39:0/4273194626 learned_addr learned my addr 172.21.15.39:0/4273194626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fe77fe700 1 -- 172.21.15.39:0/4273194626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe819e320 con 0x7f1fe8109b00 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fe77fe700 1 --2- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe819dde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009fb0 tx=0x7f1fd8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fe4ff9700 1 -- 172.21.15.39:0/4273194626 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd80038b0 con 0x7f1fe8109b00 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fe4ff9700 1 -- 172.21.15.39:0/4273194626 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1fd8003a50 con 0x7f1fe8109b00 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe819e5b0 con 0x7f1fe8109b00 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fe4ff9700 1 -- 172.21.15.39:0/4273194626 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd801a5c0 con 0x7f1fe8109b00 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.804+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81a1180 con 0x7f1fe8109b00 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.805+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe8064e20 con 0x7f1fe8109b00 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.805+0000 7f1fe4ff9700 1 -- 172.21.15.39:0/4273194626 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1fd801a8c0 con 0x7f1fe8109b00 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.806+0000 7f1fe4ff9700 1 --2- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1fd003e420 0x7f1fd0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:51.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.806+0000 7f1fe4ff9700 1 -- 172.21.15.39:0/4273194626 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1fd8053bf0 con 0x7f1fe8109b00 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.806+0000 7f1fe6ffd700 1 --2- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1fd003e420 0x7f1fd0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.806+0000 7f1fe6ffd700 1 --2- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1fd003e420 0x7f1fd0040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006fd0 tx=0x7f1fdc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:50.811+0000 7f1fe4ff9700 1 -- 172.21.15.39:0/4273194626 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1fd8017e90 con 0x7f1fe8109b00 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.034+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1fe8106180 con 0x7f1fd003e420 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.035+0000 7f1fe4ff9700 1 -- 172.21.15.39:0/4273194626 <== mgr.14120 v2:172.21.15.39:6800/3594679658 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1fe8106180 con 0x7f1fd003e420 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.037+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1fd003e420 msgr2=0x7f1fd0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.037+0000 7f1fedbd6700 1 --2- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1fd003e420 0x7f1fd0040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006fd0 tx=0x7f1fdc006e40 comp rx=0 tx=0).stop 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.037+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 msgr2=0x7f1fe819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.037+0000 7f1fedbd6700 1 --2- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe819dde0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009fb0 tx=0x7f1fd8011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.037+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 shutdown_connections 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.037+0000 7f1fedbd6700 1 --2- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1fd003e420 0x7f1fd0040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:51.610 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.037+0000 7f1fedbd6700 1 --2- 172.21.15.39:0/4273194626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fe8109b00 0x7f1fe819dde0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:51.611 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.037+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 >> 172.21.15.39:0/4273194626 conn(0x7f1fe8105140 msgr2=0x7f1fe8105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:51.611 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.038+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 shutdown_connections 2024-05-04T02:59:51.611 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:51.038+0000 7f1fedbd6700 1 -- 172.21.15.39:0/4273194626 wait complete. 2024-05-04T02:59:51.611 INFO:teuthology.orchestra.run.smithi039.stdout:Generating ssh key... 2024-05-04T02:59:51.848 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:51 smithi039 ceph-mon[29715]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T02:59:52.928 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: Generating public/private rsa key pair. 2024-05-04T02:59:52.928 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: Your identification has been saved in /tmp/tmp9pkrqrc7/key. 2024-05-04T02:59:52.928 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: Your public key has been saved in /tmp/tmp9pkrqrc7/key.pub. 2024-05-04T02:59:52.928 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: The key fingerprint is: 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: SHA256:Txsv/wkb4irJ+vRdmxyAz9PuHrMytuSVAVCdOaKB0Yc ceph-1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: The key's randomart image is: 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: +---[RSA 3072]----+ 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | .+.o.. o | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | . E.o = | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | +.. . | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | .. . | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | S + . | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | = * o | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | ... X % | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | .+. =+X @ . | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: | .o..oo=*@.o | 2024-05-04T02:59:52.929 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 02:59:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: +----[SHA256]-----+ 2024-05-04T02:59:53.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.063+0000 7f2382bec700 1 Processor -- start 2024-05-04T02:59:53.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.063+0000 7f2382bec700 1 -- start start 2024-05-04T02:59:53.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.063+0000 7f2382bec700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c108e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:53.228 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.063+0000 7f2382bec700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f237c109400 con 0x7f237c108a00 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.064+0000 7f2380988700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c108e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.064+0000 7f2380988700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c108e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38728/0 (socket says 172.21.15.39:38728) 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.064+0000 7f2380988700 1 -- 172.21.15.39:0/3900036248 learned_addr learned my addr 172.21.15.39:0/3900036248 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.064+0000 7f2380988700 1 -- 172.21.15.39:0/3900036248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c109bd0 con 0x7f237c108a00 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.064+0000 7f2380988700 1 --2- 172.21.15.39:0/3900036248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c108e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2374009e80 tx=0x7f237400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e13543875db6bd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.064+0000 7f237b7fe700 1 -- 172.21.15.39:0/3900036248 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2374017070 con 0x7f237c108a00 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.064+0000 7f237b7fe700 1 -- 172.21.15.39:0/3900036248 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f237400b8c0 con 0x7f237c108a00 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.064+0000 7f237b7fe700 1 -- 172.21.15.39:0/3900036248 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2374004420 con 0x7f237c108a00 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.065+0000 7f2382bec700 1 -- 172.21.15.39:0/3900036248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 msgr2=0x7f237c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.065+0000 7f2382bec700 1 --2- 172.21.15.39:0/3900036248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c108e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2374009e80 tx=0x7f237400b330 comp rx=0 tx=0).stop 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.065+0000 7f2382bec700 1 -- 172.21.15.39:0/3900036248 shutdown_connections 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.065+0000 7f2382bec700 1 --2- 172.21.15.39:0/3900036248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c108e10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:53.229 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.065+0000 7f2382bec700 1 -- 172.21.15.39:0/3900036248 >> 172.21.15.39:0/3900036248 conn(0x7f237c104020 msgr2=0x7f237c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.065+0000 7f2382bec700 1 -- 172.21.15.39:0/3900036248 shutdown_connections 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.065+0000 7f2382bec700 1 -- 172.21.15.39:0/3900036248 wait complete. 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.066+0000 7f2382bec700 1 Processor -- start 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.066+0000 7f2382bec700 1 -- start start 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.066+0000 7f2382bec700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c07d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.066+0000 7f2382bec700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10ce00 con 0x7f237c108a00 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.066+0000 7f2380988700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c07d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.066+0000 7f2380988700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c07d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38730/0 (socket says 172.21.15.39:38730) 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.066+0000 7f2380988700 1 -- 172.21.15.39:0/3120532675 learned_addr learned my addr 172.21.15.39:0/3120532675 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.066+0000 7f2380988700 1 -- 172.21.15.39:0/3120532675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c07d730 con 0x7f237c108a00 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.067+0000 7f2380988700 1 --2- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c07d1f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2374004000 tx=0x7f23740093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.067+0000 7f2379ffb700 1 -- 172.21.15.39:0/3120532675 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2374020020 con 0x7f237c108a00 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.067+0000 7f2379ffb700 1 -- 172.21.15.39:0/3120532675 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2374003d00 con 0x7f237c108a00 2024-05-04T02:59:53.230 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.067+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c1a0bd0 con 0x7f237c108a00 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.067+0000 7f2379ffb700 1 -- 172.21.15.39:0/3120532675 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2374017040 con 0x7f237c108a00 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.067+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c1a10b0 con 0x7f237c108a00 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.068+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c064e20 con 0x7f237c108a00 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.068+0000 7f2379ffb700 1 -- 172.21.15.39:0/3120532675 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2374023020 con 0x7f237c108a00 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.068+0000 7f2379ffb700 1 --2- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f236403e470 0x7f2364040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.068+0000 7f2379ffb700 1 -- 172.21.15.39:0/3120532675 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f237400f050 con 0x7f237c108a00 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.068+0000 7f237bfff700 1 --2- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f236403e470 0x7f2364040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.069+0000 7f237bfff700 1 --2- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f236403e470 0x7f2364040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2370006fd0 tx=0x7f2370006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.074+0000 7f2379ffb700 1 -- 172.21.15.39:0/3120532675 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f237402a020 con 0x7f237c108a00 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.287+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f237c1050c0 con 0x7f236403e470 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.626+0000 7f2379ffb700 1 -- 172.21.15.39:0/3120532675 <== mgr.14120 v2:172.21.15.39:6800/3594679658 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f237c1050c0 con 0x7f236403e470 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f236403e470 msgr2=0x7f2364040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 --2- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f236403e470 0x7f2364040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2370006fd0 tx=0x7f2370006e40 comp rx=0 tx=0).stop 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 msgr2=0x7f237c07d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:53.231 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 --2- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c07d1f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2374004000 tx=0x7f23740093b0 comp rx=0 tx=0).stop 2024-05-04T02:59:53.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 shutdown_connections 2024-05-04T02:59:53.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 --2- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f236403e470 0x7f2364040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:53.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 --2- 172.21.15.39:0/3120532675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c108a00 0x7f237c07d1f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:53.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 >> 172.21.15.39:0/3120532675 conn(0x7f237c104020 msgr2=0x7f237c104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:53.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 shutdown_connections 2024-05-04T02:59:53.232 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:52.628+0000 7f2382bec700 1 -- 172.21.15.39:0/3120532675 wait complete. 2024-05-04T02:59:53.855 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:53 smithi039 ceph-mon[29715]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T02:59:53.855 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:53 smithi039 ceph-mon[29715]: Generating ssh key... 2024-05-04T02:59:53.855 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:53 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T02:59:53.855 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:53 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T02:59:54.418 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS2jk0AYNqqUa9VytIdCOlrdvOzBQigMpHBQjA5Pd3R083U+v/xHXVYxu8B1sjJ8maoUB4twM8jAxF2pHacZBbA/yvYMy3lZraAD+MD/s0mx4iJX8uZ64TM8F/EZZ6jdTJzuFXNkw30Rv4CfFxD2Ae947qZHXp3Q154AW1KQoyFUsj0ixmKAgqxw//GWU7cQsSVM3vp6K+vyhN41qxbU5BNCDkpJXmgRaHYcZyVjUTdTv5GY/dFWGKmy00LmcH72XY6RUczEOBodiOgaJYrEEERlRmbWON/1GLFWTClDcJqZpaBUev8HE8k5bZ567L/R6BQGfV4XOnYl294CAkCSelqX4BpOtELLvR39POl+3+FVQOMLsWnLOrrxAHtfBGuxAQdS+5DzCvbJHy6bqlTLZXMTcUyVw1cOoPD6GA8zSejS5kFEPl29BzYWh2hd4jR0PRxRWPoigYrn3iMxq33EtAlFj32ArBPmtfbXDG72HKXYhFNqtXJSgQnjIUiE9OMD0= ceph-1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T02:59:54.418 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.628+0000 7f96ca4e1700 1 Processor -- start 2024-05-04T02:59:54.418 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.628+0000 7f96ca4e1700 1 -- start start 2024-05-04T02:59:54.418 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.628+0000 7f96ca4e1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c410ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:54.418 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.628+0000 7f96ca4e1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96c407ce30 con 0x7f96c410a5f0 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.628+0000 7f96c3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c410ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.628+0000 7f96c3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c410ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38746/0 (socket says 172.21.15.39:38746) 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.628+0000 7f96c3fff700 1 -- 172.21.15.39:0/3312877881 learned_addr learned my addr 172.21.15.39:0/3312877881 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.629+0000 7f96c3fff700 1 -- 172.21.15.39:0/3312877881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c410cf50 con 0x7f96c410a5f0 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.629+0000 7f96c3fff700 1 --2- 172.21.15.39:0/3312877881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c410ca10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96b40098a0 tx=0x7f96b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c596e5f3590c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.629+0000 7f96c2ffd700 1 -- 172.21.15.39:0/3312877881 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b4004160 con 0x7f96c410a5f0 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.629+0000 7f96c2ffd700 1 -- 172.21.15.39:0/3312877881 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f96b4004300 con 0x7f96c410a5f0 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.629+0000 7f96c2ffd700 1 -- 172.21.15.39:0/3312877881 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b40119e0 con 0x7f96c410a5f0 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.630+0000 7f96ca4e1700 1 -- 172.21.15.39:0/3312877881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 msgr2=0x7f96c410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.630+0000 7f96ca4e1700 1 --2- 172.21.15.39:0/3312877881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c410ca10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96b40098a0 tx=0x7f96b4009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.630+0000 7f96ca4e1700 1 -- 172.21.15.39:0/3312877881 shutdown_connections 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.630+0000 7f96ca4e1700 1 --2- 172.21.15.39:0/3312877881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c410ca10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.630+0000 7f96ca4e1700 1 -- 172.21.15.39:0/3312877881 >> 172.21.15.39:0/3312877881 conn(0x7f96c4104010 msgr2=0x7f96c41064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.630+0000 7f96ca4e1700 1 -- 172.21.15.39:0/3312877881 shutdown_connections 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.630+0000 7f96ca4e1700 1 -- 172.21.15.39:0/3312877881 wait complete. 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.631+0000 7f96ca4e1700 1 Processor -- start 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.631+0000 7f96ca4e1700 1 -- start start 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.631+0000 7f96ca4e1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.631+0000 7f96ca4e1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96c410dd70 con 0x7f96c410a5f0 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.631+0000 7f96c3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.631+0000 7f96c3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c419dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38756/0 (socket says 172.21.15.39:38756) 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.631+0000 7f96c3fff700 1 -- 172.21.15.39:0/2859855760 learned_addr learned my addr 172.21.15.39:0/2859855760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.631+0000 7f96c3fff700 1 -- 172.21.15.39:0/2859855760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c419e290 con 0x7f96c410a5f0 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.632+0000 7f96c3fff700 1 --2- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c419dd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96b4003f20 tx=0x7f96b4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.632+0000 7f96c17fa700 1 -- 172.21.15.39:0/2859855760 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b40038b0 con 0x7f96c410a5f0 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.632+0000 7f96c17fa700 1 -- 172.21.15.39:0/2859855760 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f96b4003a50 con 0x7f96c410a5f0 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.632+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c419e520 con 0x7f96c410a5f0 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.632+0000 7f96c17fa700 1 -- 172.21.15.39:0/2859855760 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b4019550 con 0x7f96c410a5f0 2024-05-04T02:59:54.420 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.632+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c41a1120 con 0x7f96c410a5f0 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.633+0000 7f96c17fa700 1 -- 172.21.15.39:0/2859855760 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f96b4019840 con 0x7f96c410a5f0 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.633+0000 7f96c17fa700 1 --2- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f96ac03e470 0x7f96ac040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.633+0000 7f96c17fa700 1 -- 172.21.15.39:0/2859855760 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f96b4054a40 con 0x7f96c410a5f0 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.633+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c4064e20 con 0x7f96c410a5f0 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.633+0000 7f96c37fe700 1 --2- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f96ac03e470 0x7f96ac040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.634+0000 7f96c37fe700 1 --2- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f96ac03e470 0x7f96ac040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.639+0000 7f96c17fa700 1 -- 172.21.15.39:0/2859855760 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96b4017e90 con 0x7f96c410a5f0 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.854+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f96c4105010 con 0x7f96ac03e470 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.855+0000 7f96c17fa700 1 -- 172.21.15.39:0/2859855760 <== mgr.14120 v2:172.21.15.39:6800/3594679658 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f96c4105010 con 0x7f96ac03e470 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f96ac03e470 msgr2=0x7f96ac040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 --2- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f96ac03e470 0x7f96ac040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8006e40 comp rx=0 tx=0).stop 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 msgr2=0x7f96c419dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 --2- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c419dd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96b4003f20 tx=0x7f96b4011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:54.421 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 shutdown_connections 2024-05-04T02:59:54.422 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 --2- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f96ac03e470 0x7f96ac040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:54.422 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 --2- 172.21.15.39:0/2859855760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96c410a5f0 0x7f96c419dd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:54.422 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 >> 172.21.15.39:0/2859855760 conn(0x7f96c4104010 msgr2=0x7f96c41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:54.422 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 shutdown_connections 2024-05-04T02:59:54.422 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:53.858+0000 7f96ca4e1700 1 -- 172.21.15.39:0/2859855760 wait complete. 2024-05-04T02:59:54.422 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-04T02:59:54.422 INFO:teuthology.orchestra.run.smithi039.stdout:Adding key to root@localhost authorized_keys... 2024-05-04T02:59:54.422 INFO:teuthology.orchestra.run.smithi039.stdout:Adding host smithi039... 2024-05-04T02:59:54.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:54 smithi039 ceph-mon[29715]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T02:59:55.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:55 smithi039 ceph-mon[29715]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T02:59:56.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 02:59:56 smithi039 ceph-mon[29715]: Deploying cephadm binary to smithi039 2024-05-04T02:59:59.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Added host 'smithi039' with addr '172.21.15.39' 2024-05-04T02:59:59.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.882+0000 7f1be22c1700 1 Processor -- start 2024-05-04T02:59:59.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.882+0000 7f1be22c1700 1 -- start start 2024-05-04T02:59:59.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.882+0000 7f1be22c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc10caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:59.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.882+0000 7f1be22c1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc07d680 con 0x7f1bdc10a680 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.882+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc10caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.882+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc10caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38770/0 (socket says 172.21.15.39:38770) 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.882+0000 7f1bdbfff700 1 -- 172.21.15.39:0/109730626 learned_addr learned my addr 172.21.15.39:0/109730626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.883+0000 7f1bdbfff700 1 -- 172.21.15.39:0/109730626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc10cfe0 con 0x7f1bdc10a680 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.883+0000 7f1bdbfff700 1 --2- 172.21.15.39:0/109730626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc10caa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0098a0 tx=0x7f1bcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80744978a36b7f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.883+0000 7f1bdaffd700 1 -- 172.21.15.39:0/109730626 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc004160 con 0x7f1bdc10a680 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.883+0000 7f1bdaffd700 1 -- 172.21.15.39:0/109730626 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1bcc004300 con 0x7f1bdc10a680 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.883+0000 7f1bdaffd700 1 -- 172.21.15.39:0/109730626 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc0119e0 con 0x7f1bdc10a680 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.884+0000 7f1be22c1700 1 -- 172.21.15.39:0/109730626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 msgr2=0x7f1bdc10caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.884+0000 7f1be22c1700 1 --2- 172.21.15.39:0/109730626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc10caa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0098a0 tx=0x7f1bcc009bb0 comp rx=0 tx=0).stop 2024-05-04T02:59:59.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.884+0000 7f1be22c1700 1 -- 172.21.15.39:0/109730626 shutdown_connections 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.884+0000 7f1be22c1700 1 --2- 172.21.15.39:0/109730626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc10caa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.884+0000 7f1be22c1700 1 -- 172.21.15.39:0/109730626 >> 172.21.15.39:0/109730626 conn(0x7f1bdc1040c0 msgr2=0x7f1bdc106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.884+0000 7f1be22c1700 1 -- 172.21.15.39:0/109730626 shutdown_connections 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.884+0000 7f1be22c1700 1 -- 172.21.15.39:0/109730626 wait complete. 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.884+0000 7f1be22c1700 1 Processor -- start 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.885+0000 7f1be22c1700 1 -- start start 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.885+0000 7f1be22c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.885+0000 7f1be22c1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10dda0 con 0x7f1bdc10a680 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.885+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.885+0000 7f1bdbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc109e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38786/0 (socket says 172.21.15.39:38786) 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.885+0000 7f1bdbfff700 1 -- 172.21.15.39:0/409146118 learned_addr learned my addr 172.21.15.39:0/409146118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.885+0000 7f1bdbfff700 1 -- 172.21.15.39:0/409146118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc10a340 con 0x7f1bdc10a680 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.885+0000 7f1bdbfff700 1 --2- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc109e00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc003f20 tx=0x7f1bcc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.886+0000 7f1bd97fa700 1 -- 172.21.15.39:0/409146118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc0038b0 con 0x7f1bdc10a680 2024-05-04T02:59:59.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.886+0000 7f1bd97fa700 1 -- 172.21.15.39:0/409146118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1bcc003a50 con 0x7f1bdc10a680 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.886+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc108590 con 0x7f1bdc10a680 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.886+0000 7f1bd97fa700 1 -- 172.21.15.39:0/409146118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc019550 con 0x7f1bdc10a680 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.886+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc108a70 con 0x7f1bdc10a680 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.887+0000 7f1bd97fa700 1 -- 172.21.15.39:0/409146118 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1bcc019800 con 0x7f1bdc10a680 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.887+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bdc064e20 con 0x7f1bdc10a680 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.887+0000 7f1bd97fa700 1 --2- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1bc403e470 0x7f1bc4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.887+0000 7f1bd97fa700 1 -- 172.21.15.39:0/409146118 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1bcc053c80 con 0x7f1bdc10a680 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.887+0000 7f1bdb7fe700 1 --2- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1bc403e470 0x7f1bc4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T02:59:59.920 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.888+0000 7f1bdb7fe700 1 --2- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1bc403e470 0x7f1bc4040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0006fd0 tx=0x7f1bd0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:54.892+0000 7f1bd97fa700 1 -- 172.21.15.39:0/409146118 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1bcc05a080 con 0x7f1bdc10a680 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:55.110+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}) -- 0x7f1bdc109220 con 0x7f1bc403e470 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.364+0000 7f1bd97fa700 1 -- 172.21.15.39:0/409146118 <== mgr.14120 v2:172.21.15.39:6800/3594679658 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f1bdc109220 con 0x7f1bc403e470 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.366+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1bc403e470 msgr2=0x7f1bc4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.366+0000 7f1be22c1700 1 --2- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1bc403e470 0x7f1bc4040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0006fd0 tx=0x7f1bd0006e40 comp rx=0 tx=0).stop 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.366+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 msgr2=0x7f1bdc109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.366+0000 7f1be22c1700 1 --2- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc109e00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc003f20 tx=0x7f1bcc011a60 comp rx=0 tx=0).stop 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.367+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 shutdown_connections 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.367+0000 7f1be22c1700 1 --2- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f1bc403e470 0x7f1bc4040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.367+0000 7f1be22c1700 1 --2- 172.21.15.39:0/409146118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bdc10a680 0x7f1bdc109e00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.367+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 >> 172.21.15.39:0/409146118 conn(0x7f1bdc1040c0 msgr2=0x7f1bdc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.367+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 shutdown_connections 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T02:59:59.367+0000 7f1be22c1700 1 -- 172.21.15.39:0/409146118 wait complete. 2024-05-04T02:59:59.921 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mon service... 2024-05-04T03:00:00.424 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:00 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T03:00:00.424 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:00 smithi039 ceph-mon[29715]: Added host smithi039 2024-05-04T03:00:00.424 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:00 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:00:01.799 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-04T03:00:01.799 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.412+0000 7f67d8a57700 1 Processor -- start 2024-05-04T03:00:01.799 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.412+0000 7f67d8a57700 1 -- start start 2024-05-04T03:00:01.799 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.412+0000 7f67d8a57700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.412+0000 7f67d8a57700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67d410bb60 con 0x7f67d410b180 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.412+0000 7f67d259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.412+0000 7f67d259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57872/0 (socket says 172.21.15.39:57872) 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.412+0000 7f67d259c700 1 -- 172.21.15.39:0/3419242746 learned_addr learned my addr 172.21.15.39:0/3419242746 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.413+0000 7f67d259c700 1 -- 172.21.15.39:0/3419242746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d410bce0 con 0x7f67d410b180 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.413+0000 7f67d259c700 1 --2- 172.21.15.39:0/3419242746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410b590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f67c40098a0 tx=0x7f67c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ad79efe78b6a57 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.413+0000 7f67d159a700 1 -- 172.21.15.39:0/3419242746 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c4004160 con 0x7f67d410b180 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.413+0000 7f67d159a700 1 -- 172.21.15.39:0/3419242746 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f67c4004300 con 0x7f67d410b180 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.413+0000 7f67d159a700 1 -- 172.21.15.39:0/3419242746 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c40119e0 con 0x7f67d410b180 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.414+0000 7f67d8a57700 1 -- 172.21.15.39:0/3419242746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 msgr2=0x7f67d410b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.414+0000 7f67d8a57700 1 --2- 172.21.15.39:0/3419242746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410b590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f67c40098a0 tx=0x7f67c4009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.414+0000 7f67d8a57700 1 -- 172.21.15.39:0/3419242746 shutdown_connections 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.414+0000 7f67d8a57700 1 --2- 172.21.15.39:0/3419242746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410b590 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:01.800 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.414+0000 7f67d8a57700 1 -- 172.21.15.39:0/3419242746 >> 172.21.15.39:0/3419242746 conn(0x7f67d4103860 msgr2=0x7f67d4105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.414+0000 7f67d8a57700 1 -- 172.21.15.39:0/3419242746 shutdown_connections 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.414+0000 7f67d8a57700 1 -- 172.21.15.39:0/3419242746 wait complete. 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d8a57700 1 Processor -- start 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d8a57700 1 -- start start 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d8a57700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d8a57700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4108890 con 0x7f67d410b180 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57880/0 (socket says 172.21.15.39:57880) 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d259c700 1 -- 172.21.15.39:0/2573344566 learned_addr learned my addr 172.21.15.39:0/2573344566 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d259c700 1 -- 172.21.15.39:0/2573344566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d410abb0 con 0x7f67d410b180 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.415+0000 7f67d259c700 1 --2- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410a670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f67c40116a0 tx=0x7f67c4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.416+0000 7f67cb7fe700 1 -- 172.21.15.39:0/2573344566 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c40038b0 con 0x7f67d410b180 2024-05-04T03:00:01.801 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.416+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d4108e00 con 0x7f67d410b180 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.416+0000 7f67cb7fe700 1 -- 172.21.15.39:0/2573344566 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f67c4003a50 con 0x7f67d410b180 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.416+0000 7f67cb7fe700 1 -- 172.21.15.39:0/2573344566 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c4019550 con 0x7f67d410b180 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.416+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d41092e0 con 0x7f67d410b180 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.417+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d4064e20 con 0x7f67d410b180 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.417+0000 7f67cb7fe700 1 -- 172.21.15.39:0/2573344566 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f67c4019850 con 0x7f67d410b180 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.417+0000 7f67cb7fe700 1 --2- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f67c003e420 0x7f67c0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.417+0000 7f67cb7fe700 1 -- 172.21.15.39:0/2573344566 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f67c4053c40 con 0x7f67d410b180 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.417+0000 7f67d1d9b700 1 --2- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f67c003e420 0x7f67c0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.417+0000 7f67d1d9b700 1 --2- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f67c003e420 0x7f67c0040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67bc006fd0 tx=0x7f67bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.422+0000 7f67cb7fe700 1 -- 172.21.15.39:0/2573344566 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67c4017e90 con 0x7f67d410b180 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.655+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f67d4109a90 con 0x7f67c003e420 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.663+0000 7f67cb7fe700 1 -- 172.21.15.39:0/2573344566 <== mgr.14120 v2:172.21.15.39:6800/3594679658 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f67d4109a90 con 0x7f67c003e420 2024-05-04T03:00:01.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f67c003e420 msgr2=0x7f67c0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 --2- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f67c003e420 0x7f67c0040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67bc006fd0 tx=0x7f67bc006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 msgr2=0x7f67d410a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 --2- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410a670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f67c40116a0 tx=0x7f67c4011a60 comp rx=0 tx=0).stop 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 shutdown_connections 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 --2- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f67c003e420 0x7f67c0040930 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 --2- 172.21.15.39:0/2573344566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d410b180 0x7f67d410a670 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 >> 172.21.15.39:0/2573344566 conn(0x7f67d4103860 msgr2=0x7f67d4105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 shutdown_connections 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:00.665+0000 7f67d8a57700 1 -- 172.21.15.39:0/2573344566 wait complete. 2024-05-04T03:00:01.803 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mgr service... 2024-05-04T03:00:01.907 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:01 smithi039 ceph-mon[29715]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:00:01.908 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:01 smithi039 ceph-mon[29715]: Saving service mon spec with placement count:5 2024-05-04T03:00:01.908 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:01 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T03:00:03.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-04T03:00:03.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.218+0000 7fdf903f1700 1 Processor -- start 2024-05-04T03:00:03.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.219+0000 7fdf903f1700 1 -- start start 2024-05-04T03:00:03.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.219+0000 7fdf903f1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8810c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:03.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.219+0000 7fdf903f1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8810cd40 con 0x7fdf8807d4d0 2024-05-04T03:00:03.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.219+0000 7fdf8e18d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8810c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:03.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.219+0000 7fdf8e18d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8810c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57884/0 (socket says 172.21.15.39:57884) 2024-05-04T03:00:03.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.219+0000 7fdf8e18d700 1 -- 172.21.15.39:0/680174512 learned_addr learned my addr 172.21.15.39:0/680174512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.220+0000 7fdf8e18d700 1 -- 172.21.15.39:0/680174512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8810cec0 con 0x7fdf8807d4d0 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.220+0000 7fdf8e18d700 1 --2- 172.21.15.39:0/680174512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8810c770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098a0 tx=0x7fdf84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61f8321c53915d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.220+0000 7fdf8d18b700 1 -- 172.21.15.39:0/680174512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf84004160 con 0x7fdf8807d4d0 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.220+0000 7fdf8d18b700 1 -- 172.21.15.39:0/680174512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdf84004300 con 0x7fdf8807d4d0 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.220+0000 7fdf8d18b700 1 -- 172.21.15.39:0/680174512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf840119e0 con 0x7fdf8807d4d0 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.221+0000 7fdf903f1700 1 -- 172.21.15.39:0/680174512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 msgr2=0x7fdf8810c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.221+0000 7fdf903f1700 1 --2- 172.21.15.39:0/680174512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8810c770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098a0 tx=0x7fdf84009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.221+0000 7fdf903f1700 1 -- 172.21.15.39:0/680174512 shutdown_connections 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.221+0000 7fdf903f1700 1 --2- 172.21.15.39:0/680174512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8810c770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.221+0000 7fdf903f1700 1 -- 172.21.15.39:0/680174512 >> 172.21.15.39:0/680174512 conn(0x7fdf881040c0 msgr2=0x7fdf88106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:03.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.221+0000 7fdf903f1700 1 -- 172.21.15.39:0/680174512 shutdown_connections 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.221+0000 7fdf903f1700 1 -- 172.21.15.39:0/680174512 wait complete. 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf903f1700 1 Processor -- start 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf903f1700 1 -- start start 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf903f1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8819dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf903f1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8810dda0 con 0x7fdf8807d4d0 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf8e18d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8819dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf8e18d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8819dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57886/0 (socket says 172.21.15.39:57886) 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf8e18d700 1 -- 172.21.15.39:0/3985666798 learned_addr learned my addr 172.21.15.39:0/3985666798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf8e18d700 1 -- 172.21.15.39:0/3985666798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8819e2c0 con 0x7fdf8807d4d0 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.222+0000 7fdf8e18d700 1 --2- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8819dd80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdf840116a0 tx=0x7fdf84011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.223+0000 7fdf7f7fe700 1 -- 172.21.15.39:0/3985666798 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf84003970 con 0x7fdf8807d4d0 2024-05-04T03:00:03.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.223+0000 7fdf7f7fe700 1 -- 172.21.15.39:0/3985666798 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdf84003b10 con 0x7fdf8807d4d0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.223+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8819e550 con 0x7fdf8807d4d0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.223+0000 7fdf7f7fe700 1 -- 172.21.15.39:0/3985666798 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf84019550 con 0x7fdf8807d4d0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.223+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881a1160 con 0x7fdf8807d4d0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.224+0000 7fdf7f7fe700 1 -- 172.21.15.39:0/3985666798 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fdf840197c0 con 0x7fdf8807d4d0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.224+0000 7fdf7f7fe700 1 --2- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdf7403e4c0 0x7fdf740409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.224+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf88064e20 con 0x7fdf8807d4d0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.224+0000 7fdf7f7fe700 1 -- 172.21.15.39:0/3985666798 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf8401a6e0 con 0x7fdf8807d4d0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.224+0000 7fdf8d98c700 1 --2- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdf7403e4c0 0x7fdf740409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.224+0000 7fdf8d98c700 1 --2- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdf7403e4c0 0x7fdf740409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf78006fd0 tx=0x7fdf78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:03.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.229+0000 7fdf7f7fe700 1 -- 172.21.15.39:0/3985666798 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf84016260 con 0x7fdf8807d4d0 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.461+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdf88105160 con 0x7fdf7403e4c0 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.466+0000 7fdf7f7fe700 1 -- 172.21.15.39:0/3985666798 <== mgr.14120 v2:172.21.15.39:6800/3594679658 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdf88105160 con 0x7fdf7403e4c0 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdf7403e4c0 msgr2=0x7fdf740409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 --2- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdf7403e4c0 0x7fdf740409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf78006fd0 tx=0x7fdf78006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 msgr2=0x7fdf8819dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 --2- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8819dd80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdf840116a0 tx=0x7fdf84011a60 comp rx=0 tx=0).stop 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 shutdown_connections 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 --2- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdf7403e4c0 0x7fdf740409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 --2- 172.21.15.39:0/3985666798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8807d4d0 0x7fdf8819dd80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 >> 172.21.15.39:0/3985666798 conn(0x7fdf881040c0 msgr2=0x7fdf88104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:03.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 shutdown_connections 2024-05-04T03:00:03.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:02.468+0000 7fdf903f1700 1 -- 172.21.15.39:0/3985666798 wait complete. 2024-05-04T03:00:03.560 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:03 smithi039 ceph-mon[29715]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:00:03.561 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:03 smithi039 ceph-mon[29715]: Saving service mgr spec with placement count:2 2024-05-04T03:00:03.561 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:03 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T03:00:03.561 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:03 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T03:00:03.561 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:03 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T03:00:04.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.578+0000 7f61c6181700 1 Processor -- start 2024-05-04T03:00:04.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.578+0000 7f61c6181700 1 -- start start 2024-05-04T03:00:04.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.579+0000 7f61c6181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c010c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:04.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.579+0000 7f61c6181700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61c007ce90 con 0x7f61c010a590 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.579+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c010c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.579+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c010c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57898/0 (socket says 172.21.15.39:57898) 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.579+0000 7f61bf7fe700 1 -- 172.21.15.39:0/397385456 learned_addr learned my addr 172.21.15.39:0/397385456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.579+0000 7f61bf7fe700 1 -- 172.21.15.39:0/397385456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c010cef0 con 0x7f61c010a590 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.579+0000 7f61bf7fe700 1 --2- 172.21.15.39:0/397385456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c010c9b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f61b00098a0 tx=0x7f61b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b612143fe58079 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.579+0000 7f61be7fc700 1 -- 172.21.15.39:0/397385456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0004160 con 0x7f61c010a590 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61be7fc700 1 -- 172.21.15.39:0/397385456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61b0004300 con 0x7f61c010a590 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61be7fc700 1 -- 172.21.15.39:0/397385456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0011960 con 0x7f61c010a590 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61c6181700 1 -- 172.21.15.39:0/397385456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 msgr2=0x7f61c010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61c6181700 1 --2- 172.21.15.39:0/397385456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c010c9b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f61b00098a0 tx=0x7f61b0009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61c6181700 1 -- 172.21.15.39:0/397385456 shutdown_connections 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61c6181700 1 --2- 172.21.15.39:0/397385456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c010c9b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:04.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61c6181700 1 -- 172.21.15.39:0/397385456 >> 172.21.15.39:0/397385456 conn(0x7f61c0104010 msgr2=0x7f61c01064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61c6181700 1 -- 172.21.15.39:0/397385456 shutdown_connections 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.580+0000 7f61c6181700 1 -- 172.21.15.39:0/397385456 wait complete. 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.581+0000 7f61c6181700 1 Processor -- start 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.581+0000 7f61c6181700 1 -- start start 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61c6181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61c6181700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61c010de90 con 0x7f61c010a590 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c019dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57904/0 (socket says 172.21.15.39:57904) 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61bf7fe700 1 -- 172.21.15.39:0/3637044904 learned_addr learned my addr 172.21.15.39:0/3637044904 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61bf7fe700 1 -- 172.21.15.39:0/3637044904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c019e290 con 0x7f61c010a590 2024-05-04T03:00:04.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61bf7fe700 1 --2- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c019dd50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61b0011670 tx=0x7f61b0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61bcff9700 1 -- 172.21.15.39:0/3637044904 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0003aa0 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61bcff9700 1 -- 172.21.15.39:0/3637044904 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61b001b8d0 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c019e520 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.582+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01a1120 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.583+0000 7f61bcff9700 1 -- 172.21.15.39:0/3637044904 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b00197f0 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.584+0000 7f61bcff9700 1 -- 172.21.15.39:0/3637044904 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f61b00199d0 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.584+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a0005360 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.584+0000 7f61bcff9700 1 --2- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f61ac03e420 0x7f61ac040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.584+0000 7f61bcff9700 1 -- 172.21.15.39:0/3637044904 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f61b00540a0 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.584+0000 7f61beffd700 1 --2- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f61ac03e420 0x7f61ac040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.585+0000 7f61beffd700 1 --2- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f61ac03e420 0x7f61ac040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f61a8006fd0 tx=0x7f61a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.589+0000 7f61bcff9700 1 -- 172.21.15.39:0/3637044904 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61b001f030 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.804+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f61a0005650 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.810+0000 7f61bcff9700 1 -- 172.21.15.39:0/3637044904 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f61b0019dc0 con 0x7f61c010a590 2024-05-04T03:00:04.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.813+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f61ac03e420 msgr2=0x7f61ac040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.813+0000 7f61c6181700 1 --2- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f61ac03e420 0x7f61ac040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f61a8006fd0 tx=0x7f61a8006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.813+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 msgr2=0x7f61c019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.813+0000 7f61c6181700 1 --2- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c019dd50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61b0011670 tx=0x7f61b0011a30 comp rx=0 tx=0).stop 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.813+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 shutdown_connections 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.813+0000 7f61c6181700 1 --2- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f61ac03e420 0x7f61ac040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.813+0000 7f61c6181700 1 --2- 172.21.15.39:0/3637044904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61c010a590 0x7f61c019dd50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.813+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 >> 172.21.15.39:0/3637044904 conn(0x7f61c0104010 msgr2=0x7f61c01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.814+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 shutdown_connections 2024-05-04T03:00:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:03.814+0000 7f61c6181700 1 -- 172.21.15.39:0/3637044904 wait complete. 2024-05-04T03:00:05.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:04 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3637044904' entity='client.admin' 2024-05-04T03:00:05.883 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.801+0000 7fdc80a33700 1 Processor -- start 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.801+0000 7fdc80a33700 1 -- start start 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.801+0000 7fdc80a33700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c10ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.801+0000 7fdc80a33700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c07d6e0 con 0x7fdc7c10a620 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.802+0000 7fdc7a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c10ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.802+0000 7fdc7a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c10ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57914/0 (socket says 172.21.15.39:57914) 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.802+0000 7fdc7a59c700 1 -- 172.21.15.39:0/1086316337 learned_addr learned my addr 172.21.15.39:0/1086316337 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.802+0000 7fdc7a59c700 1 -- 172.21.15.39:0/1086316337 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c10cf80 con 0x7fdc7c10a620 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.802+0000 7fdc7a59c700 1 --2- 172.21.15.39:0/1086316337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c10ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0098a0 tx=0x7fdc6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648e4428de021c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.802+0000 7fdc7959a700 1 -- 172.21.15.39:0/1086316337 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc6c004160 con 0x7fdc7c10a620 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.802+0000 7fdc7959a700 1 -- 172.21.15.39:0/1086316337 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc6c004300 con 0x7fdc7c10a620 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.802+0000 7fdc7959a700 1 -- 172.21.15.39:0/1086316337 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc6c011960 con 0x7fdc7c10a620 2024-05-04T03:00:05.884 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.803+0000 7fdc80a33700 1 -- 172.21.15.39:0/1086316337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 msgr2=0x7fdc7c10ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.803+0000 7fdc80a33700 1 --2- 172.21.15.39:0/1086316337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c10ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0098a0 tx=0x7fdc6c009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.803+0000 7fdc80a33700 1 -- 172.21.15.39:0/1086316337 shutdown_connections 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.803+0000 7fdc80a33700 1 --2- 172.21.15.39:0/1086316337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c10ca40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.803+0000 7fdc80a33700 1 -- 172.21.15.39:0/1086316337 >> 172.21.15.39:0/1086316337 conn(0x7fdc7c104020 msgr2=0x7fdc7c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.803+0000 7fdc80a33700 1 -- 172.21.15.39:0/1086316337 shutdown_connections 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.803+0000 7fdc80a33700 1 -- 172.21.15.39:0/1086316337 wait complete. 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.804+0000 7fdc80a33700 1 Processor -- start 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.804+0000 7fdc80a33700 1 -- start start 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc80a33700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc80a33700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c10dec0 con 0x7fdc7c10a620 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc7a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc7a59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c19dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57926/0 (socket says 172.21.15.39:57926) 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc7a59c700 1 -- 172.21.15.39:0/2676062664 learned_addr learned my addr 172.21.15.39:0/2676062664 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc7a59c700 1 -- 172.21.15.39:0/2676062664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c19e2c0 con 0x7fdc7c10a620 2024-05-04T03:00:05.885 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc7a59c700 1 --2- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c19dd80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c011670 tx=0x7fdc6c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc6b7fe700 1 -- 172.21.15.39:0/2676062664 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc6c003aa0 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc6b7fe700 1 -- 172.21.15.39:0/2676062664 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc6c01b8d0 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c19e550 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc6b7fe700 1 -- 172.21.15.39:0/2676062664 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc6c0197f0 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.805+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c1a1160 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.807+0000 7fdc6b7fe700 1 -- 172.21.15.39:0/2676062664 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fdc6c019990 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.807+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c064e20 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.807+0000 7fdc6b7fe700 1 --2- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdc6403e470 0x7fdc64040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.807+0000 7fdc6b7fe700 1 -- 172.21.15.39:0/2676062664 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc6c053fd0 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.807+0000 7fdc79d9b700 1 --2- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdc6403e470 0x7fdc64040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.808+0000 7fdc79d9b700 1 --2- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdc6403e470 0x7fdc64040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdc70006fd0 tx=0x7fdc70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:04.813+0000 7fdc6b7fe700 1 -- 172.21.15.39:0/2676062664 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc6c01f030 con 0x7fdc7c10a620 2024-05-04T03:00:05.886 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.030+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fdc7c1a1330 con 0x7fdc7c10a620 2024-05-04T03:00:05.887 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.033+0000 7fdc6b7fe700 1 -- 172.21.15.39:0/2676062664 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fdc6c019d80 con 0x7fdc7c10a620 2024-05-04T03:00:05.887 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdc6403e470 msgr2=0x7fdc64040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:05.889 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 --2- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdc6403e470 0x7fdc64040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdc70006fd0 tx=0x7fdc70006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:05.889 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 msgr2=0x7fdc7c19dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 --2- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c19dd80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c011670 tx=0x7fdc6c011a30 comp rx=0 tx=0).stop 2024-05-04T03:00:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 shutdown_connections 2024-05-04T03:00:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 --2- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fdc6403e470 0x7fdc64040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 --2- 172.21.15.39:0/2676062664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7c10a620 0x7fdc7c19dd80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 >> 172.21.15.39:0/2676062664 conn(0x7fdc7c104020 msgr2=0x7fdc7c104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 shutdown_connections 2024-05-04T03:00:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:05.036+0000 7fdc80a33700 1 -- 172.21.15.39:0/2676062664 wait complete. 2024-05-04T03:00:05.890 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling the dashboard module... 2024-05-04T03:00:06.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:06 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2676062664' entity='client.admin' 2024-05-04T03:00:07.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:07 smithi039 ceph-mon[29715]: from='mgr.14120 172.21.15.39:0/4168026819' entity='mgr.x' 2024-05-04T03:00:07.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:07 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3341444609' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-04T03:00:07.586 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:07 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: ignoring --setuser ceph since I am not root 2024-05-04T03:00:07.586 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:07 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: ignoring --setgroup ceph since I am not root 2024-05-04T03:00:07.586 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:07 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:07.519+0000 7fc0cfc0b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T03:00:07.838 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:07 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:07.730+0000 7fc0cfc0b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:00:07.839 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:07 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:07.838+0000 7fc0cfc0b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.663+0000 7f5a9c65d700 1 Processor -- start 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.663+0000 7f5a9c65d700 1 -- start start 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.663+0000 7f5a9c65d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a94108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.663+0000 7f5a9c65d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941093b0 con 0x7f5a941089b0 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.663+0000 7f5a9a3f9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a94108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.663+0000 7f5a9a3f9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a94108dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55330/0 (socket says 172.21.15.39:55330) 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.663+0000 7f5a9a3f9700 1 -- 172.21.15.39:0/3415800953 learned_addr learned my addr 172.21.15.39:0/3415800953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.664+0000 7f5a9a3f9700 1 -- 172.21.15.39:0/3415800953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a94109b80 con 0x7f5a941089b0 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.664+0000 7f5a9a3f9700 1 --2- 172.21.15.39:0/3415800953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a94108dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800bb40 tx=0x7f5a8800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e00e2003ff30d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.664+0000 7f5a993f7700 1 -- 172.21.15.39:0/3415800953 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a8800dbe0 con 0x7f5a941089b0 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.664+0000 7f5a993f7700 1 -- 172.21.15.39:0/3415800953 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a880044c0 con 0x7f5a941089b0 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.664+0000 7f5a993f7700 1 -- 172.21.15.39:0/3415800953 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88003e20 con 0x7f5a941089b0 2024-05-04T03:00:07.943 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.665+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3415800953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 msgr2=0x7f5a94108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.665+0000 7f5a9c65d700 1 --2- 172.21.15.39:0/3415800953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a94108dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800bb40 tx=0x7f5a8800be50 comp rx=0 tx=0).stop 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.665+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3415800953 shutdown_connections 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.665+0000 7f5a9c65d700 1 --2- 172.21.15.39:0/3415800953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a94108dc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.665+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3415800953 >> 172.21.15.39:0/3415800953 conn(0x7f5a94104010 msgr2=0x7f5a941064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.665+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3415800953 shutdown_connections 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.665+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3415800953 wait complete. 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.666+0000 7f5a9c65d700 1 Processor -- start 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.666+0000 7f5a9c65d700 1 -- start start 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.666+0000 7f5a9c65d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a941a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.666+0000 7f5a9c65d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9407bfe0 con 0x7f5a941089b0 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.666+0000 7f5a9a3f9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a941a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.666+0000 7f5a9a3f9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a941a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55340/0 (socket says 172.21.15.39:55340) 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.666+0000 7f5a9a3f9700 1 -- 172.21.15.39:0/3341444609 learned_addr learned my addr 172.21.15.39:0/3341444609 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:07.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.667+0000 7f5a9a3f9700 1 -- 172.21.15.39:0/3341444609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a941a06e0 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.667+0000 7f5a9a3f9700 1 --2- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a941a01a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e010 tx=0x7f5a88003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.667+0000 7f5a877fe700 1 -- 172.21.15.39:0/3341444609 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88010630 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.667+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a941a0970 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.667+0000 7f5a877fe700 1 -- 172.21.15.39:0/3341444609 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a8801d070 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.667+0000 7f5a877fe700 1 -- 172.21.15.39:0/3341444609 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88019940 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.667+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a3580 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.668+0000 7f5a877fe700 1 -- 172.21.15.39:0/3341444609 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5a88028060 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.668+0000 7f5a877fe700 1 --2- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f5a8003e470 0x7f5a80040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.668+0000 7f5a877fe700 1 -- 172.21.15.39:0/3341444609 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5a880538a0 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.668+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a94064e20 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.669+0000 7f5a99bf8700 1 --2- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f5a8003e470 0x7f5a80040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.669+0000 7f5a99bf8700 1 --2- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f5a8003e470 0x7f5a80040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a90006fd0 tx=0x7f5a90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.679+0000 7f5a877fe700 1 -- 172.21.15.39:0/3341444609 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a88015070 con 0x7f5a941089b0 2024-05-04T03:00:07.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:06.973+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5a94194fa0 con 0x7f5a941089b0 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.302+0000 7f5a877fe700 1 -- 172.21.15.39:0/3341444609 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5a880162a0 con 0x7f5a941089b0 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.304+0000 7f5a877fe700 1 -- 172.21.15.39:0/3341444609 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f5a8801f030 con 0x7f5a941089b0 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f5a8003e470 msgr2=0x7f5a80040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 --2- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f5a8003e470 0x7f5a80040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a90006fd0 tx=0x7f5a90006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 msgr2=0x7f5a941a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 --2- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a941a01a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e010 tx=0x7f5a88003fb0 comp rx=0 tx=0).stop 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 shutdown_connections 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 --2- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f5a8003e470 0x7f5a80040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 --2- 172.21.15.39:0/3341444609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a941089b0 0x7f5a941a01a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:07.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 >> 172.21.15.39:0/3341444609 conn(0x7f5a94104010 msgr2=0x7f5a94192060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:07.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 shutdown_connections 2024-05-04T03:00:07.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:07.306+0000 7f5a9c65d700 1 -- 172.21.15.39:0/3341444609 wait complete. 2024-05-04T03:00:08.422 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:08 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3341444609' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-04T03:00:08.422 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:08 smithi039 ceph-mon[29715]: mgrmap e9: x(active, since 19s) 2024-05-04T03:00:08.423 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:08 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:08.171+0000 7fc0cfc0b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T03:00:08.423 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:08 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:08.262+0000 7fc0cfc0b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T03:00:09.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:00:09.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-04T03:00:09.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-04T03:00:09.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-04T03:00:09.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-04T03:00:09.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.410+0000 7fab9dd18700 1 Processor -- start 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.411+0000 7fab9dd18700 1 -- start start 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.411+0000 7fab9dd18700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab98108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.411+0000 7fab9dd18700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab981093b0 con 0x7fab981089b0 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.411+0000 7fab977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab98108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.411+0000 7fab977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab98108dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55372/0 (socket says 172.21.15.39:55372) 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.411+0000 7fab977fe700 1 -- 172.21.15.39:0/2254482870 learned_addr learned my addr 172.21.15.39:0/2254482870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.411+0000 7fab977fe700 1 -- 172.21.15.39:0/2254482870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab98109b80 con 0x7fab981089b0 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.412+0000 7fab977fe700 1 --2- 172.21.15.39:0/2254482870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab98108dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab800098a0 tx=0x7fab80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6bdf2b888de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.412+0000 7fab967fc700 1 -- 172.21.15.39:0/2254482870 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab80004160 con 0x7fab981089b0 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.412+0000 7fab967fc700 1 -- 172.21.15.39:0/2254482870 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab80004300 con 0x7fab981089b0 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.412+0000 7fab967fc700 1 -- 172.21.15.39:0/2254482870 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab80011960 con 0x7fab981089b0 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.413+0000 7fab9dd18700 1 -- 172.21.15.39:0/2254482870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 msgr2=0x7fab98108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:09.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.413+0000 7fab9dd18700 1 --2- 172.21.15.39:0/2254482870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab98108dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab800098a0 tx=0x7fab80009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.413+0000 7fab9dd18700 1 -- 172.21.15.39:0/2254482870 shutdown_connections 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.413+0000 7fab9dd18700 1 --2- 172.21.15.39:0/2254482870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab98108dc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.413+0000 7fab9dd18700 1 -- 172.21.15.39:0/2254482870 >> 172.21.15.39:0/2254482870 conn(0x7fab98104010 msgr2=0x7fab981064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.413+0000 7fab9dd18700 1 -- 172.21.15.39:0/2254482870 shutdown_connections 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.413+0000 7fab9dd18700 1 -- 172.21.15.39:0/2254482870 wait complete. 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.413+0000 7fab9dd18700 1 Processor -- start 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.414+0000 7fab9dd18700 1 -- start start 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.414+0000 7fab9dd18700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab9819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.414+0000 7fab9dd18700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab9810cf30 con 0x7fab981089b0 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.414+0000 7fab977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab9819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.414+0000 7fab977fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab9819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55378/0 (socket says 172.21.15.39:55378) 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.414+0000 7fab977fe700 1 -- 172.21.15.39:0/3894418622 learned_addr learned my addr 172.21.15.39:0/3894418622 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.414+0000 7fab977fe700 1 -- 172.21.15.39:0/3894418622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9819e320 con 0x7fab981089b0 2024-05-04T03:00:09.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.414+0000 7fab977fe700 1 --2- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab9819dde0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab800119b0 tx=0x7fab80011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.415+0000 7fab94ff9700 1 -- 172.21.15.39:0/3894418622 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab80003aa0 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.415+0000 7fab94ff9700 1 -- 172.21.15.39:0/3894418622 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab8001b8d0 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.415+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9819e5b0 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.415+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab981a1180 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.415+0000 7fab94ff9700 1 -- 172.21.15.39:0/3894418622 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab800237d0 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.416+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9810d0b0 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.416+0000 7fab94ff9700 1 -- 172.21.15.39:0/3894418622 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fab8002c3f0 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.416+0000 7fab94ff9700 1 --2- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fab8403e470 0x7fab84040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.416+0000 7fab94ff9700 1 -- 172.21.15.39:0/3894418622 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fab80053f00 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.416+0000 7fab96ffd700 1 -- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fab8403e470 msgr2=0x7fab84040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3594679658 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.417+0000 7fab96ffd700 1 --2- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fab8403e470 0x7fab84040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.422+0000 7fab94ff9700 1 -- 172.21.15.39:0/3894418622 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab8001f030 con 0x7fab981089b0 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.618+0000 7fab96ffd700 1 -- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fab8403e470 msgr2=0x7fab84040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3594679658 2024-05-04T03:00:09.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.618+0000 7fab96ffd700 1 --2- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fab8403e470 0x7fab84040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.740+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fab9802e930 con 0x7fab981089b0 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.741+0000 7fab94ff9700 1 -- 172.21.15.39:0/3894418622 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fab8000f2e0 con 0x7fab981089b0 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.743+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fab8403e470 msgr2=0x7fab84040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.743+0000 7fab9dd18700 1 --2- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fab8403e470 0x7fab84040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.743+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 msgr2=0x7fab9819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.743+0000 7fab9dd18700 1 --2- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab9819dde0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab800119b0 tx=0x7fab80011a90 comp rx=0 tx=0).stop 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.743+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 shutdown_connections 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.743+0000 7fab9dd18700 1 --2- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7fab8403e470 0x7fab84040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.743+0000 7fab9dd18700 1 --2- 172.21.15.39:0/3894418622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab981089b0 0x7fab9819dde0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.743+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 >> 172.21.15.39:0/3894418622 conn(0x7fab98104010 msgr2=0x7fab98194370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:09.297 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.744+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 shutdown_connections 2024-05-04T03:00:09.298 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:08.744+0000 7fab9dd18700 1 -- 172.21.15.39:0/3894418622 wait complete. 2024-05-04T03:00:09.298 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-05-04T03:00:09.298 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 9... 2024-05-04T03:00:09.401 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:09 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3894418622' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-04T03:00:09.669 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:09 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:09.542+0000 7fc0cfc0b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T03:00:09.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:09 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:09.732+0000 7fc0cfc0b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T03:00:09.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:09 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:09.910+0000 7fc0cfc0b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T03:00:10.531 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:10 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:10.220+0000 7fc0cfc0b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T03:00:10.531 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:10 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:10.318+0000 7fc0cfc0b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T03:00:10.532 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:10 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:10.531+0000 7fc0cfc0b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T03:00:10.940 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:10 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:10.736+0000 7fc0cfc0b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:00:11.647 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:11 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:11.368+0000 7fc0cfc0b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T03:00:11.648 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:11 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:11.609+0000 7fc0cfc0b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T03:00:12.440 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:12 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:12.076+0000 7fc0cfc0b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T03:00:12.440 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:12 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:12.181+0000 7fc0cfc0b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T03:00:12.942 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:12 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:12.582+0000 7fc0cfc0b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T03:00:13.323 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:12 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:12.964+0000 7fc0cfc0b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T03:00:13.323 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:13.062+0000 7fc0cfc0b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T03:00:13.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:13.322+0000 7fc0cfc0b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T03:00:13.690 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:13.464+0000 7fc0cfc0b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T03:00:14.190 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:13.782+0000 7fc0cfc0b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T03:00:14.190 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:13.905+0000 7fc0cfc0b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T03:00:14.191 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:14.003+0000 7fc0cfc0b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T03:00:14.927 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:14.641+0000 7fc0cfc0b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T03:00:14.927 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:00:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:00:14.872+0000 7fc0cfc0b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T03:00:15.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: Active manager daemon x restarted 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: Activating manager daemon x 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: osdmap e3: 0 total, 0 up, 0 in 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: mgrmap e10: x(active, starting, since 0.00815358s) 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: Manager daemon x is now available 2024-05-04T03:00:15.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:00:16.107 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-04T03:00:16.107 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-04T03:00:16.107 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:15 smithi039 ceph-mon[29715]: mgrmap e11: x(active, since 1.0102s) 2024-05-04T03:00:16.802 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:00:16.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-04T03:00:16.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-04T03:00:16.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:00:16.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.709+0000 7f8d06461700 1 Processor -- start 2024-05-04T03:00:16.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.709+0000 7f8d06461700 1 -- start start 2024-05-04T03:00:16.803 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.709+0000 7f8d06461700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0010a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:16.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.709+0000 7f8d06461700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0010a940 con 0x7f8d00109f60 2024-05-04T03:00:16.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.710+0000 7f8cfffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0010a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:16.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.710+0000 7f8cfffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0010a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55390/0 (socket says 172.21.15.39:55390) 2024-05-04T03:00:16.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.710+0000 7f8cfffff700 1 -- 172.21.15.39:0/3043954637 learned_addr learned my addr 172.21.15.39:0/3043954637 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:16.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.710+0000 7f8cfffff700 1 -- 172.21.15.39:0/3043954637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0010aac0 con 0x7f8d00109f60 2024-05-04T03:00:16.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.710+0000 7f8cfffff700 1 --2- 172.21.15.39:0/3043954637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0010a370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80098a0 tx=0x7f8ce8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d751fe3932e5ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:16.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.710+0000 7f8cfeffd700 1 -- 172.21.15.39:0/3043954637 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce8004160 con 0x7f8d00109f60 2024-05-04T03:00:16.804 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.710+0000 7f8cfeffd700 1 -- 172.21.15.39:0/3043954637 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ce8004300 con 0x7f8d00109f60 2024-05-04T03:00:16.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.710+0000 7f8cfeffd700 1 -- 172.21.15.39:0/3043954637 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce8011960 con 0x7f8d00109f60 2024-05-04T03:00:16.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.711+0000 7f8d06461700 1 -- 172.21.15.39:0/3043954637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 msgr2=0x7f8d0010a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:16.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.711+0000 7f8d06461700 1 --2- 172.21.15.39:0/3043954637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0010a370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80098a0 tx=0x7f8ce8009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:16.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.711+0000 7f8d06461700 1 -- 172.21.15.39:0/3043954637 shutdown_connections 2024-05-04T03:00:16.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.711+0000 7f8d06461700 1 --2- 172.21.15.39:0/3043954637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0010a370 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:16.805 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.711+0000 7f8d06461700 1 -- 172.21.15.39:0/3043954637 >> 172.21.15.39:0/3043954637 conn(0x7f8d00103790 msgr2=0x7f8d00105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:16.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.711+0000 7f8d06461700 1 -- 172.21.15.39:0/3043954637 shutdown_connections 2024-05-04T03:00:16.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.711+0000 7f8d06461700 1 -- 172.21.15.39:0/3043954637 wait complete. 2024-05-04T03:00:16.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.712+0000 7f8d06461700 1 Processor -- start 2024-05-04T03:00:16.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.712+0000 7f8d06461700 1 -- start start 2024-05-04T03:00:16.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.712+0000 7f8d06461700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:16.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8d06461700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00108950 con 0x7f8d00109f60 2024-05-04T03:00:16.806 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8cfffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:16.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8cfffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0019ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55398/0 (socket says 172.21.15.39:55398) 2024-05-04T03:00:16.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8cfffff700 1 -- 172.21.15.39:0/4231905257 learned_addr learned my addr 172.21.15.39:0/4231905257 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:16.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8cfffff700 1 -- 172.21.15.39:0/4231905257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d001a04c0 con 0x7f8d00109f60 2024-05-04T03:00:16.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8cfffff700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0019ff80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8011980 tx=0x7f8ce8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:16.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce80039e0 con 0x7f8d00109f60 2024-05-04T03:00:16.807 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ce801b8d0 con 0x7f8d00109f60 2024-05-04T03:00:16.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d001a0750 con 0x7f8d00109f60 2024-05-04T03:00:16.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ce80236b0 con 0x7f8d00109f60 2024-05-04T03:00:16.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.713+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d001a3360 con 0x7f8d00109f60 2024-05-04T03:00:16.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.715+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f8ce8023960 con 0x7f8d00109f60 2024-05-04T03:00:16.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.715+0000 7f8cfd7fa700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 0x7f8cec0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:16.808 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.715+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 --> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8cec0410f0 con 0x7f8cec03e4d0 2024-05-04T03:00:16.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.715+0000 7f8cff7fe700 1 -- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 msgr2=0x7f8cec0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3594679658 2024-05-04T03:00:16.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.715+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8ce8053dd0 con 0x7f8d00109f60 2024-05-04T03:00:16.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.715+0000 7f8cff7fe700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 0x7f8cec0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:00:16.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.916+0000 7f8cff7fe700 1 -- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 msgr2=0x7f8cec0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3594679658 2024-05-04T03:00:16.809 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:09.916+0000 7f8cff7fe700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 0x7f8cec0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T03:00:16.810 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:10.317+0000 7f8cff7fe700 1 -- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 msgr2=0x7f8cec0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3594679658 2024-05-04T03:00:16.810 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:10.317+0000 7f8cff7fe700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 0x7f8cec0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-04T03:00:16.810 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:11.118+0000 7f8cff7fe700 1 -- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 msgr2=0x7f8cec0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3594679658 2024-05-04T03:00:16.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:11.118+0000 7f8cff7fe700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 0x7f8cec0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-04T03:00:16.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:12.719+0000 7f8cff7fe700 1 -- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 msgr2=0x7f8cec0409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3594679658 2024-05-04T03:00:16.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:12.719+0000 7f8cff7fe700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 0x7f8cec0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-04T03:00:16.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:14.881+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f8ce8023ca0 con 0x7f8d00109f60 2024-05-04T03:00:16.811 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:14.881+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 msgr2=0x7f8cec0409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:00:16.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:14.881+0000 7f8cfd7fa700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 0x7f8cec0409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:16.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.881+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f8ce802b350 con 0x7f8d00109f60 2024-05-04T03:00:16.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.881+0000 7f8cfd7fa700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8ce8053960 0x7f8cec043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:16.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.882+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8ce801ad40 con 0x7f8ce8053960 2024-05-04T03:00:16.812 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.882+0000 7f8cff7fe700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8ce8053960 0x7f8cec043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:16.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.883+0000 7f8cff7fe700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8ce8053960 0x7f8cec043fd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0003a10 tx=0x7f8cf0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:16.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.884+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8ce801ad40 con 0x7f8ce8053960 2024-05-04T03:00:16.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.893+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8d00064e20 con 0x7f8ce8053960 2024-05-04T03:00:16.813 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.893+0000 7f8cfd7fa700 1 -- 172.21.15.39:0/4231905257 <== mgr.14152 v2:172.21.15.39:6800/2627629846 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f8d00064e20 con 0x7f8ce8053960 2024-05-04T03:00:16.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.893+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8ce8053960 msgr2=0x7f8cec043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:16.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.893+0000 7f8d06461700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8ce8053960 0x7f8cec043fd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0003a10 tx=0x7f8cf0009270 comp rx=0 tx=0).stop 2024-05-04T03:00:16.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.893+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 msgr2=0x7f8d0019ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:16.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.893+0000 7f8d06461700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0019ff80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8011980 tx=0x7f8ce8011a60 comp rx=0 tx=0).stop 2024-05-04T03:00:16.814 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.894+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 shutdown_connections 2024-05-04T03:00:16.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.894+0000 7f8d06461700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8ce8053960 0x7f8cec043fd0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:16.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.894+0000 7f8d06461700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d00109f60 0x7f8d0019ff80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:16.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.894+0000 7f8d06461700 1 --2- 172.21.15.39:0/4231905257 >> [v2:172.21.15.39:6800/3594679658,v1:172.21.15.39:6801/3594679658] conn(0x7f8cec03e4d0 0x7f8cec0409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:16.815 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.894+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 >> 172.21.15.39:0/4231905257 conn(0x7f8d00103790 msgr2=0x7f8d00196250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:16.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.894+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 shutdown_connections 2024-05-04T03:00:16.816 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:15.894+0000 7f8d06461700 1 -- 172.21.15.39:0/4231905257 wait complete. 2024-05-04T03:00:16.816 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 9 is available 2024-05-04T03:00:16.816 INFO:teuthology.orchestra.run.smithi039.stdout:Generating a dashboard self-signed certificate... 2024-05-04T03:00:17.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:16 smithi039 ceph-mon[29715]: [04/May/2024:03:00:15] ENGINE Bus STARTING 2024-05-04T03:00:17.042 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:16 smithi039 ceph-mon[29715]: [04/May/2024:03:00:15] ENGINE Serving on http://172.21.15.39:8765 2024-05-04T03:00:17.042 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:16 smithi039 ceph-mon[29715]: [04/May/2024:03:00:16] ENGINE Serving on https://172.21.15.39:7150 2024-05-04T03:00:17.042 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:16 smithi039 ceph-mon[29715]: [04/May/2024:03:00:16] ENGINE Bus STARTED 2024-05-04T03:00:17.042 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.246+0000 7f8596939700 1 Processor -- start 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.246+0000 7f8596939700 1 -- start start 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.246+0000 7f8596939700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f8590108810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.246+0000 7f8596939700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f859007ce10 con 0x7f8590108400 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.246+0000 7f858ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f8590108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.246+0000 7f858ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f8590108810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38490/0 (socket says 172.21.15.39:38490) 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.246+0000 7f858ffff700 1 -- 172.21.15.39:0/420539283 learned_addr learned my addr 172.21.15.39:0/420539283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:18.292 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.246+0000 7f858ffff700 1 -- 172.21.15.39:0/420539283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8590108d50 con 0x7f8590108400 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.247+0000 7f858ffff700 1 --2- 172.21.15.39:0/420539283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f8590108810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85800098a0 tx=0x7f8580009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff9a5d81e082cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.247+0000 7f858effd700 1 -- 172.21.15.39:0/420539283 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8580004160 con 0x7f8590108400 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.247+0000 7f858effd700 1 -- 172.21.15.39:0/420539283 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8580004300 con 0x7f8590108400 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.247+0000 7f858effd700 1 -- 172.21.15.39:0/420539283 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8580011960 con 0x7f8590108400 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.248+0000 7f8596939700 1 -- 172.21.15.39:0/420539283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 msgr2=0x7f8590108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.248+0000 7f8596939700 1 --2- 172.21.15.39:0/420539283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f8590108810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85800098a0 tx=0x7f8580009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.248+0000 7f8596939700 1 -- 172.21.15.39:0/420539283 shutdown_connections 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.248+0000 7f8596939700 1 --2- 172.21.15.39:0/420539283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f8590108810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.248+0000 7f8596939700 1 -- 172.21.15.39:0/420539283 >> 172.21.15.39:0/420539283 conn(0x7f8590104010 msgr2=0x7f85901064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.248+0000 7f8596939700 1 -- 172.21.15.39:0/420539283 shutdown_connections 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.248+0000 7f8596939700 1 -- 172.21.15.39:0/420539283 wait complete. 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.248+0000 7f8596939700 1 Processor -- start 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.249+0000 7f8596939700 1 -- start start 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.249+0000 7f8596939700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f859019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:18.293 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.249+0000 7f8596939700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8590109db0 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.249+0000 7f858ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f859019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.249+0000 7f858ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f859019ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38496/0 (socket says 172.21.15.39:38496) 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.249+0000 7f858ffff700 1 -- 172.21.15.39:0/268017899 learned_addr learned my addr 172.21.15.39:0/268017899 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.249+0000 7f858ffff700 1 -- 172.21.15.39:0/268017899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85901a04a0 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.250+0000 7f858ffff700 1 --2- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f859019ff60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8580011670 tx=0x7f8580011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.250+0000 7f858d7fa700 1 -- 172.21.15.39:0/268017899 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f858001f070 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.250+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85901a0730 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.250+0000 7f858d7fa700 1 -- 172.21.15.39:0/268017899 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8580003b20 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.250+0000 7f858d7fa700 1 -- 172.21.15.39:0/268017899 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8580019580 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.250+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85901a3340 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.251+0000 7f858d7fa700 1 -- 172.21.15.39:0/268017899 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f8580022070 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.251+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8590064e20 con 0x7f8590108400 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.251+0000 7f858d7fa700 1 --2- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f857803e410 0x7f8578040920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:18.294 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.251+0000 7f858d7fa700 1 -- 172.21.15.39:0/268017899 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8580053a00 con 0x7f8590108400 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.251+0000 7f858f7fe700 1 --2- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f857803e410 0x7f8578040920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.252+0000 7f858f7fe700 1 --2- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f857803e410 0x7f8578040920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8584006fd0 tx=0x7f8584006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.257+0000 7f858d7fa700 1 -- 172.21.15.39:0/268017899 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85800172b0 con 0x7f8590108400 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.442+0000 7f858d7fa700 1 -- 172.21.15.39:0/268017899 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f858000bce0 con 0x7f8590108400 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.480+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f859002e920 con 0x7f857803e410 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.714+0000 7f858d7fa700 1 -- 172.21.15.39:0/268017899 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f859002e920 con 0x7f857803e410 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f857803e410 msgr2=0x7f8578040920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 --2- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f857803e410 0x7f8578040920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8584006fd0 tx=0x7f8584006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 msgr2=0x7f859019ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 --2- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f859019ff60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8580011670 tx=0x7f8580011a30 comp rx=0 tx=0).stop 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 shutdown_connections 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 --2- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f857803e410 0x7f8578040920 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 --2- 172.21.15.39:0/268017899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8590108400 0x7f859019ff60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:18.295 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 >> 172.21.15.39:0/268017899 conn(0x7f8590104010 msgr2=0x7f8590193f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:18.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 shutdown_connections 2024-05-04T03:00:18.296 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:17.716+0000 7f8596939700 1 -- 172.21.15.39:0/268017899 wait complete. 2024-05-04T03:00:18.296 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial admin user... 2024-05-04T03:00:18.536 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:18 smithi039 ceph-mon[29715]: mgrmap e12: x(active, since 2s) 2024-05-04T03:00:18.536 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:18 smithi039 ceph-mon[29715]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:00:18.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:18.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:18.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$vCzZw1tG7SmWjA18sHvPWOJtbRFSi2h4pPbPt29u3IGbpoMY4bQGa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714791619, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-04T03:00:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.744+0000 7fa0332df700 1 Processor -- start 2024-05-04T03:00:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.744+0000 7fa0332df700 1 -- start start 2024-05-04T03:00:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.744+0000 7fa0332df700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c10ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.744+0000 7fa0332df700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c07d6a0 con 0x7fa02c10a670 2024-05-04T03:00:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.745+0000 7fa03107b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c10ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.745+0000 7fa03107b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c10ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38512/0 (socket says 172.21.15.39:38512) 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.745+0000 7fa03107b700 1 -- 172.21.15.39:0/3173477519 learned_addr learned my addr 172.21.15.39:0/3173477519 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.745+0000 7fa03107b700 1 -- 172.21.15.39:0/3173477519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02c10cfd0 con 0x7fa02c10a670 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.745+0000 7fa03107b700 1 --2- 172.21.15.39:0/3173477519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c10ca90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009e80 tx=0x7fa01c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19152ddd1697b443 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.745+0000 7fa02bfff700 1 -- 172.21.15.39:0/3173477519 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa01c017070 con 0x7fa02c10a670 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.745+0000 7fa02bfff700 1 -- 172.21.15.39:0/3173477519 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa01c00b8c0 con 0x7fa02c10a670 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.745+0000 7fa02bfff700 1 -- 172.21.15.39:0/3173477519 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa01c004420 con 0x7fa02c10a670 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.746+0000 7fa0332df700 1 -- 172.21.15.39:0/3173477519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 msgr2=0x7fa02c10ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.746+0000 7fa0332df700 1 --2- 172.21.15.39:0/3173477519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c10ca90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009e80 tx=0x7fa01c00b330 comp rx=0 tx=0).stop 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.746+0000 7fa0332df700 1 -- 172.21.15.39:0/3173477519 shutdown_connections 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.746+0000 7fa0332df700 1 --2- 172.21.15.39:0/3173477519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c10ca90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.746+0000 7fa0332df700 1 -- 172.21.15.39:0/3173477519 >> 172.21.15.39:0/3173477519 conn(0x7fa02c104050 msgr2=0x7fa02c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.746+0000 7fa0332df700 1 -- 172.21.15.39:0/3173477519 shutdown_connections 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.746+0000 7fa0332df700 1 -- 172.21.15.39:0/3173477519 wait complete. 2024-05-04T03:00:19.922 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.747+0000 7fa0332df700 1 Processor -- start 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.747+0000 7fa0332df700 1 -- start start 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.747+0000 7fa0332df700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c109df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.747+0000 7fa0332df700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c10df10 con 0x7fa02c10a670 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.747+0000 7fa03107b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c109df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.747+0000 7fa03107b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c109df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38520/0 (socket says 172.21.15.39:38520) 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.747+0000 7fa03107b700 1 -- 172.21.15.39:0/4200206831 learned_addr learned my addr 172.21.15.39:0/4200206831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.748+0000 7fa03107b700 1 -- 172.21.15.39:0/4200206831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02c10a330 con 0x7fa02c10a670 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.748+0000 7fa03107b700 1 --2- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c109df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009e50 tx=0x7fa01c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.748+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4200206831 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa01c020020 con 0x7fa02c10a670 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.748+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4200206831 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa01c0128c0 con 0x7fa02c10a670 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.748+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02c108580 con 0x7fa02c10a670 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.748+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4200206831 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa01c017040 con 0x7fa02c10a670 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.748+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa02c108ac0 con 0x7fa02c10a670 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.749+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4200206831 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fa01c023020 con 0x7fa02c10a670 2024-05-04T03:00:19.923 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.749+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa02c064e20 con 0x7fa02c10a670 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.749+0000 7fa02a7fc700 1 --2- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa01403e4d0 0x7fa0140409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.749+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4200206831 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa01c00f050 con 0x7fa02c10a670 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.749+0000 7fa03087a700 1 --2- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa01403e4d0 0x7fa0140409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.750+0000 7fa03087a700 1 --2- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa01403e4d0 0x7fa0140409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa020006fd0 tx=0x7fa020006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.755+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4200206831 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa01c02a020 con 0x7fa02c10a670 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:18.991+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa02c109130 con 0x7fa01403e4d0 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.239+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4200206831 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa02c109130 con 0x7fa01403e4d0 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa01403e4d0 msgr2=0x7fa0140409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 --2- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa01403e4d0 0x7fa0140409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa020006fd0 tx=0x7fa020006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 msgr2=0x7fa02c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 --2- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c109df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009e50 tx=0x7fa01c00bd50 comp rx=0 tx=0).stop 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 shutdown_connections 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 --2- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa01403e4d0 0x7fa0140409e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:19.924 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 --2- 172.21.15.39:0/4200206831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa02c10a670 0x7fa02c109df0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:19.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 >> 172.21.15.39:0/4200206831 conn(0x7fa02c104050 msgr2=0x7fa02c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:19.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 shutdown_connections 2024-05-04T03:00:19.925 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:19.241+0000 7fa0332df700 1 -- 172.21.15.39:0/4200206831 wait complete. 2024-05-04T03:00:19.925 INFO:teuthology.orchestra.run.smithi039.stdout:Fetching dashboard port number... 2024-05-04T03:00:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:20 smithi039 ceph-mon[29715]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:00:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 8443 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.388+0000 7fcb4ffa7700 1 Processor -- start 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.388+0000 7fcb4ffa7700 1 -- start start 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4ffa7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4810ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4ffa7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4807ce30 con 0x7fcb4810a5f0 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4dd43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4810ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4dd43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4810ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38526/0 (socket says 172.21.15.39:38526) 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4dd43700 1 -- 172.21.15.39:0/3464428591 learned_addr learned my addr 172.21.15.39:0/3464428591 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:21.247 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4dd43700 1 -- 172.21.15.39:0/3464428591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4810cf50 con 0x7fcb4810a5f0 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4dd43700 1 --2- 172.21.15.39:0/3464428591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4810ca10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009e80 tx=0x7fcb4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3836ba9c41520c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4cd41700 1 -- 172.21.15.39:0/3464428591 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb40017070 con 0x7fcb4810a5f0 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4cd41700 1 -- 172.21.15.39:0/3464428591 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb4000b8c0 con 0x7fcb4810a5f0 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.389+0000 7fcb4cd41700 1 -- 172.21.15.39:0/3464428591 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb40004420 con 0x7fcb4810a5f0 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.390+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/3464428591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 msgr2=0x7fcb4810ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.390+0000 7fcb4ffa7700 1 --2- 172.21.15.39:0/3464428591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4810ca10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009e80 tx=0x7fcb4000b330 comp rx=0 tx=0).stop 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.390+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/3464428591 shutdown_connections 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.390+0000 7fcb4ffa7700 1 --2- 172.21.15.39:0/3464428591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4810ca10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.390+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/3464428591 >> 172.21.15.39:0/3464428591 conn(0x7fcb48104010 msgr2=0x7fcb481064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.390+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/3464428591 shutdown_connections 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.390+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/3464428591 wait complete. 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.391+0000 7fcb4ffa7700 1 Processor -- start 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.391+0000 7fcb4ffa7700 1 -- start start 2024-05-04T03:00:21.248 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.391+0000 7fcb4ffa7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.391+0000 7fcb4ffa7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4810de90 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.391+0000 7fcb4dd43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.391+0000 7fcb4dd43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4819dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38540/0 (socket says 172.21.15.39:38540) 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.391+0000 7fcb4dd43700 1 -- 172.21.15.39:0/313028998 learned_addr learned my addr 172.21.15.39:0/313028998 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.392+0000 7fcb4dd43700 1 -- 172.21.15.39:0/313028998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4819e2b0 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.392+0000 7fcb4dd43700 1 --2- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4819dd70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009e50 tx=0x7fcb40009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.392+0000 7fcb46ffd700 1 -- 172.21.15.39:0/313028998 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb40020020 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.392+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4819e540 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.392+0000 7fcb46ffd700 1 -- 172.21.15.39:0/313028998 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb400128c0 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.392+0000 7fcb46ffd700 1 -- 172.21.15.39:0/313028998 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb40017040 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.392+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb481a1150 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.393+0000 7fcb46ffd700 1 -- 172.21.15.39:0/313028998 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fcb40023020 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.393+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb48064e20 con 0x7fcb4810a5f0 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.393+0000 7fcb46ffd700 1 --2- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb3003e470 0x7fcb30040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:21.249 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.394+0000 7fcb46ffd700 1 -- 172.21.15.39:0/313028998 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb4000f050 con 0x7fcb4810a5f0 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.394+0000 7fcb4d542700 1 --2- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb3003e470 0x7fcb30040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.394+0000 7fcb4d542700 1 --2- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb3003e470 0x7fcb30040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c006fd0 tx=0x7fcb3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.399+0000 7fcb46ffd700 1 -- 172.21.15.39:0/313028998 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb4002a020 con 0x7fcb4810a5f0 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.617+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fcb481a1320 con 0x7fcb4810a5f0 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.617+0000 7fcb46ffd700 1 -- 172.21.15.39:0/313028998 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fcb40012a70 con 0x7fcb4810a5f0 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.620+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb3003e470 msgr2=0x7fcb30040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.620+0000 7fcb4ffa7700 1 --2- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb3003e470 0x7fcb30040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c006fd0 tx=0x7fcb3c006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.620+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 msgr2=0x7fcb4819dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.620+0000 7fcb4ffa7700 1 --2- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4819dd70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009e50 tx=0x7fcb40009630 comp rx=0 tx=0).stop 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.621+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 shutdown_connections 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.621+0000 7fcb4ffa7700 1 --2- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb3003e470 0x7fcb30040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.621+0000 7fcb4ffa7700 1 --2- 172.21.15.39:0/313028998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb4810a5f0 0x7fcb4819dd70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:21.250 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.621+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 >> 172.21.15.39:0/313028998 conn(0x7fcb48104010 msgr2=0x7fcb481048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:21.251 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.621+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 shutdown_connections 2024-05-04T03:00:21.251 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:20.621+0000 7fcb4ffa7700 1 -- 172.21.15.39:0/313028998 wait complete. 2024-05-04T03:00:21.256 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-04T03:00:21.256 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-05-04T03:00:21.263 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-04T03:00:21.264 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-05-04T03:00:21.264 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-05-04T03:00:21.264 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-04T03:00:21.265 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph Dashboard is now available at: 2024-05-04T03:00:21.265 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: URL: https://smithi039:8443/ 2024-05-04T03:00:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: User: admin 2024-05-04T03:00:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: Password: ql7ev5gikd 2024-05-04T03:00:21.266 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:21.266 INFO:teuthology.orchestra.run.smithi039.stdout:Saving cluster configuration to /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config directory 2024-05-04T03:00:21.266 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling autotune for osd_memory_target 2024-05-04T03:00:21.494 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:21 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/313028998' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-04T03:00:22.495 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:22 smithi039 ceph-mon[29715]: mgrmap e13: x(active, since 6s) 2024-05-04T03:00:22.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.703+0000 7fe1b3d89700 1 Processor -- start 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.703+0000 7fe1b3d89700 1 -- start start 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.703+0000 7fe1b3d89700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac10ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.703+0000 7fe1b3d89700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac07d6c0 con 0x7fe1ac10a650 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.704+0000 7fe1b1b25700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac10ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.704+0000 7fe1b1b25700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac10ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38542/0 (socket says 172.21.15.39:38542) 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.704+0000 7fe1b1b25700 1 -- 172.21.15.39:0/1667981496 learned_addr learned my addr 172.21.15.39:0/1667981496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.704+0000 7fe1b1b25700 1 -- 172.21.15.39:0/1667981496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac10cfb0 con 0x7fe1ac10a650 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.704+0000 7fe1b1b25700 1 --2- 172.21.15.39:0/1667981496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac10ca70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80098a0 tx=0x7fe1a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d83440333bfd370 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.704+0000 7fe1b0b23700 1 -- 172.21.15.39:0/1667981496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1a8004160 con 0x7fe1ac10a650 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.704+0000 7fe1b0b23700 1 -- 172.21.15.39:0/1667981496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1a8004300 con 0x7fe1ac10a650 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.704+0000 7fe1b0b23700 1 -- 172.21.15.39:0/1667981496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1a8011960 con 0x7fe1ac10a650 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.705+0000 7fe1b3d89700 1 -- 172.21.15.39:0/1667981496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 msgr2=0x7fe1ac10ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:22.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.705+0000 7fe1b3d89700 1 --2- 172.21.15.39:0/1667981496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac10ca70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80098a0 tx=0x7fe1a8009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.705+0000 7fe1b3d89700 1 -- 172.21.15.39:0/1667981496 shutdown_connections 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.705+0000 7fe1b3d89700 1 --2- 172.21.15.39:0/1667981496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac10ca70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.705+0000 7fe1b3d89700 1 -- 172.21.15.39:0/1667981496 >> 172.21.15.39:0/1667981496 conn(0x7fe1ac104050 msgr2=0x7fe1ac106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.705+0000 7fe1b3d89700 1 -- 172.21.15.39:0/1667981496 shutdown_connections 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.705+0000 7fe1b3d89700 1 -- 172.21.15.39:0/1667981496 wait complete. 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.706+0000 7fe1b3d89700 1 Processor -- start 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.706+0000 7fe1b3d89700 1 -- start start 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.706+0000 7fe1b3d89700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.706+0000 7fe1b3d89700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10def0 con 0x7fe1ac10a650 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.706+0000 7fe1b1b25700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.706+0000 7fe1b1b25700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac19ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38546/0 (socket says 172.21.15.39:38546) 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.706+0000 7fe1b1b25700 1 -- 172.21.15.39:0/2505687342 learned_addr learned my addr 172.21.15.39:0/2505687342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.706+0000 7fe1b1b25700 1 -- 172.21.15.39:0/2505687342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac19e310 con 0x7fe1ac10a650 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.707+0000 7fe1b1b25700 1 --2- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac19ddd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8011670 tx=0x7fe1a8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:22.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.707+0000 7fe1a2ffd700 1 -- 172.21.15.39:0/2505687342 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1a8003a60 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.707+0000 7fe1a2ffd700 1 -- 172.21.15.39:0/2505687342 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1a801b8d0 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.707+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac19e5a0 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.707+0000 7fe1a2ffd700 1 -- 172.21.15.39:0/2505687342 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1a8023790 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.707+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac1a1170 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.708+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ac064e20 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.708+0000 7fe1a2ffd700 1 -- 172.21.15.39:0/2505687342 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe1a8023a40 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.708+0000 7fe1a2ffd700 1 --2- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe19803e5c0 0x7fe198040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.708+0000 7fe1a2ffd700 1 -- 172.21.15.39:0/2505687342 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe1a8054170 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.708+0000 7fe1b1324700 1 --2- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe19803e5c0 0x7fe198040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.709+0000 7fe1b1324700 1 --2- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe19803e5c0 0x7fe198040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe19c006fd0 tx=0x7fe19c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.713+0000 7fe1a2ffd700 1 -- 172.21.15.39:0/2505687342 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1a801f030 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.927+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe1ac1a1870 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.927+0000 7fe1a2ffd700 1 -- 172.21.15.39:0/2505687342 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe1a800b3e0 con 0x7fe1ac10a650 2024-05-04T03:00:22.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.929+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe19803e5c0 msgr2=0x7fe198040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.929+0000 7fe1b3d89700 1 --2- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe19803e5c0 0x7fe198040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe19c006fd0 tx=0x7fe19c006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.929+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 msgr2=0x7fe1ac19ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.929+0000 7fe1b3d89700 1 --2- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac19ddd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8011670 tx=0x7fe1a8011a30 comp rx=0 tx=0).stop 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.930+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 shutdown_connections 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.930+0000 7fe1b3d89700 1 --2- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe19803e5c0 0x7fe198040ad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.930+0000 7fe1b3d89700 1 --2- 172.21.15.39:0/2505687342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1ac10a650 0x7fe1ac19ddd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.930+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 >> 172.21.15.39:0/2505687342 conn(0x7fe1ac104050 msgr2=0x7fe1ac104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.930+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 shutdown_connections 2024-05-04T03:00:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:21.930+0000 7fe1b3d89700 1 -- 172.21.15.39:0/2505687342 wait complete. 2024-05-04T03:00:23.819 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.896+0000 7f72d2446700 1 Processor -- start 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.896+0000 7f72d2446700 1 -- start start 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72d2446700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72d2446700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1093b0 con 0x7f72cc1089b0 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72cbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72cbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc108dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38550/0 (socket says 172.21.15.39:38550) 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72cbfff700 1 -- 172.21.15.39:0/627495617 learned_addr learned my addr 172.21.15.39:0/627495617 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72cbfff700 1 -- 172.21.15.39:0/627495617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc109b80 con 0x7f72cc1089b0 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72cbfff700 1 --2- 172.21.15.39:0/627495617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc108dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0098a0 tx=0x7f72bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda0079394267104 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72caffd700 1 -- 172.21.15.39:0/627495617 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72bc004160 con 0x7f72cc1089b0 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72caffd700 1 -- 172.21.15.39:0/627495617 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72bc004300 con 0x7f72cc1089b0 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.897+0000 7f72caffd700 1 -- 172.21.15.39:0/627495617 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72bc011960 con 0x7f72cc1089b0 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.898+0000 7f72d2446700 1 -- 172.21.15.39:0/627495617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 msgr2=0x7f72cc108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.898+0000 7f72d2446700 1 --2- 172.21.15.39:0/627495617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc108dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0098a0 tx=0x7f72bc009bb0 comp rx=0 tx=0).stop 2024-05-04T03:00:23.820 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.898+0000 7f72d2446700 1 -- 172.21.15.39:0/627495617 shutdown_connections 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.898+0000 7f72d2446700 1 --2- 172.21.15.39:0/627495617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc108dc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.898+0000 7f72d2446700 1 -- 172.21.15.39:0/627495617 >> 172.21.15.39:0/627495617 conn(0x7f72cc104010 msgr2=0x7f72cc1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.898+0000 7f72d2446700 1 -- 172.21.15.39:0/627495617 shutdown_connections 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.898+0000 7f72d2446700 1 -- 172.21.15.39:0/627495617 wait complete. 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.899+0000 7f72d2446700 1 Processor -- start 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.899+0000 7f72d2446700 1 -- start start 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.899+0000 7f72d2446700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.899+0000 7f72d2446700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc10cf30 con 0x7f72cc1089b0 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72cbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72cbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38560/0 (socket says 172.21.15.39:38560) 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72cbfff700 1 -- 172.21.15.39:0/2750457674 learned_addr learned my addr 172.21.15.39:0/2750457674 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72cbfff700 1 -- 172.21.15.39:0/2750457674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc19d1e0 con 0x7f72cc1089b0 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72cbfff700 1 --2- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc19cca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f72bc011980 tx=0x7f72bc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72c97fa700 1 -- 172.21.15.39:0/2750457674 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72bc01f070 con 0x7f72cc1089b0 2024-05-04T03:00:23.821 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72cc19d470 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72c97fa700 1 -- 172.21.15.39:0/2750457674 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72bc003aa0 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72c97fa700 1 -- 172.21.15.39:0/2750457674 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72bc019560 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.900+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72cc1a0080 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.901+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72cc10d0b0 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.901+0000 7f72c97fa700 1 -- 172.21.15.39:0/2750457674 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f72bc022070 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.902+0000 7f72c97fa700 1 --2- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72b403e560 0x7f72b4040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.902+0000 7f72c97fa700 1 -- 172.21.15.39:0/2750457674 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f72bc053c00 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.902+0000 7f72cb7fe700 1 --2- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72b403e560 0x7f72b4040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.902+0000 7f72cb7fe700 1 --2- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72b403e560 0x7f72b4040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006fd0 tx=0x7f72c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:22.907+0000 7f72c97fa700 1 -- 172.21.15.39:0/2750457674 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72bc0172b0 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.241+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f72cc1a0680 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.244+0000 7f72c97fa700 1 -- 172.21.15.39:0/2750457674 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f72bc018e60 con 0x7f72cc1089b0 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72b403e560 msgr2=0x7f72b4040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:23.822 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 --2- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72b403e560 0x7f72b4040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006fd0 tx=0x7f72c0006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 msgr2=0x7f72cc19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 --2- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc19cca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f72bc011980 tx=0x7f72bc011a60 comp rx=0 tx=0).stop 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 shutdown_connections 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 --2- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72b403e560 0x7f72b4040a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 --2- 172.21.15.39:0/2750457674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72cc1089b0 0x7f72cc19cca0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 >> 172.21.15.39:0/2750457674 conn(0x7f72cc104010 msgr2=0x7f72cc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 shutdown_connections 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-05-04T03:00:23.247+0000 7f72d2446700 1 -- 172.21.15.39:0/2750457674 wait complete. 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout:Or, if you are only running a single cluster on this host: 2024-05-04T03:00:23.823 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout: ceph telemetry on 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout:For more information see: 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:23.824 INFO:teuthology.orchestra.run.smithi039.stdout:Bootstrap complete. 2024-05-04T03:00:23.858 INFO:tasks.cephadm:Fetching config... 2024-05-04T03:00:23.858 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:00:23.858 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-04T03:00:23.874 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-04T03:00:23.874 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:00:23.874 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-04T03:00:23.930 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-04T03:00:23.930 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:00:23.930 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-04T03:00:23.994 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-04T03:00:23.994 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:00:23.994 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-04T03:00:24.050 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-04T03:00:24.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS2jk0AYNqqUa9VytIdCOlrdvOzBQigMpHBQjA5Pd3R083U+v/xHXVYxu8B1sjJ8maoUB4twM8jAxF2pHacZBbA/yvYMy3lZraAD+MD/s0mx4iJX8uZ64TM8F/EZZ6jdTJzuFXNkw30Rv4CfFxD2Ae947qZHXp3Q154AW1KQoyFUsj0ixmKAgqxw//GWU7cQsSVM3vp6K+vyhN41qxbU5BNCDkpJXmgRaHYcZyVjUTdTv5GY/dFWGKmy00LmcH72XY6RUczEOBodiOgaJYrEEERlRmbWON/1GLFWTClDcJqZpaBUev8HE8k5bZ567L/R6BQGfV4XOnYl294CAkCSelqX4BpOtELLvR39POl+3+FVQOMLsWnLOrrxAHtfBGuxAQdS+5DzCvbJHy6bqlTLZXMTcUyVw1cOoPD6GA8zSejS5kFEPl29BzYWh2hd4jR0PRxRWPoigYrn3iMxq33EtAlFj32ArBPmtfbXDG72HKXYhFNqtXJSgQnjIUiE9OMD0= ceph-1e913ae0-09c2-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-04T03:00:24.122 INFO:teuthology.orchestra.run.smithi039.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS2jk0AYNqqUa9VytIdCOlrdvOzBQigMpHBQjA5Pd3R083U+v/xHXVYxu8B1sjJ8maoUB4twM8jAxF2pHacZBbA/yvYMy3lZraAD+MD/s0mx4iJX8uZ64TM8F/EZZ6jdTJzuFXNkw30Rv4CfFxD2Ae947qZHXp3Q154AW1KQoyFUsj0ixmKAgqxw//GWU7cQsSVM3vp6K+vyhN41qxbU5BNCDkpJXmgRaHYcZyVjUTdTv5GY/dFWGKmy00LmcH72XY6RUczEOBodiOgaJYrEEERlRmbWON/1GLFWTClDcJqZpaBUev8HE8k5bZ567L/R6BQGfV4XOnYl294CAkCSelqX4BpOtELLvR39POl+3+FVQOMLsWnLOrrxAHtfBGuxAQdS+5DzCvbJHy6bqlTLZXMTcUyVw1cOoPD6GA8zSejS5kFEPl29BzYWh2hd4jR0PRxRWPoigYrn3iMxq33EtAlFj32ArBPmtfbXDG72HKXYhFNqtXJSgQnjIUiE9OMD0= ceph-1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:00:24.134 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS2jk0AYNqqUa9VytIdCOlrdvOzBQigMpHBQjA5Pd3R083U+v/xHXVYxu8B1sjJ8maoUB4twM8jAxF2pHacZBbA/yvYMy3lZraAD+MD/s0mx4iJX8uZ64TM8F/EZZ6jdTJzuFXNkw30Rv4CfFxD2Ae947qZHXp3Q154AW1KQoyFUsj0ixmKAgqxw//GWU7cQsSVM3vp6K+vyhN41qxbU5BNCDkpJXmgRaHYcZyVjUTdTv5GY/dFWGKmy00LmcH72XY6RUczEOBodiOgaJYrEEERlRmbWON/1GLFWTClDcJqZpaBUev8HE8k5bZ567L/R6BQGfV4XOnYl294CAkCSelqX4BpOtELLvR39POl+3+FVQOMLsWnLOrrxAHtfBGuxAQdS+5DzCvbJHy6bqlTLZXMTcUyVw1cOoPD6GA8zSejS5kFEPl29BzYWh2hd4jR0PRxRWPoigYrn3iMxq33EtAlFj32ArBPmtfbXDG72HKXYhFNqtXJSgQnjIUiE9OMD0= ceph-1e913ae0-09c2-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-04T03:00:24.185 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS2jk0AYNqqUa9VytIdCOlrdvOzBQigMpHBQjA5Pd3R083U+v/xHXVYxu8B1sjJ8maoUB4twM8jAxF2pHacZBbA/yvYMy3lZraAD+MD/s0mx4iJX8uZ64TM8F/EZZ6jdTJzuFXNkw30Rv4CfFxD2Ae947qZHXp3Q154AW1KQoyFUsj0ixmKAgqxw//GWU7cQsSVM3vp6K+vyhN41qxbU5BNCDkpJXmgRaHYcZyVjUTdTv5GY/dFWGKmy00LmcH72XY6RUczEOBodiOgaJYrEEERlRmbWON/1GLFWTClDcJqZpaBUev8HE8k5bZ567L/R6BQGfV4XOnYl294CAkCSelqX4BpOtELLvR39POl+3+FVQOMLsWnLOrrxAHtfBGuxAQdS+5DzCvbJHy6bqlTLZXMTcUyVw1cOoPD6GA8zSejS5kFEPl29BzYWh2hd4jR0PRxRWPoigYrn3iMxq33EtAlFj32ArBPmtfbXDG72HKXYhFNqtXJSgQnjIUiE9OMD0= ceph-1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:00:24.199 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS2jk0AYNqqUa9VytIdCOlrdvOzBQigMpHBQjA5Pd3R083U+v/xHXVYxu8B1sjJ8maoUB4twM8jAxF2pHacZBbA/yvYMy3lZraAD+MD/s0mx4iJX8uZ64TM8F/EZZ6jdTJzuFXNkw30Rv4CfFxD2Ae947qZHXp3Q154AW1KQoyFUsj0ixmKAgqxw//GWU7cQsSVM3vp6K+vyhN41qxbU5BNCDkpJXmgRaHYcZyVjUTdTv5GY/dFWGKmy00LmcH72XY6RUczEOBodiOgaJYrEEERlRmbWON/1GLFWTClDcJqZpaBUev8HE8k5bZ567L/R6BQGfV4XOnYl294CAkCSelqX4BpOtELLvR39POl+3+FVQOMLsWnLOrrxAHtfBGuxAQdS+5DzCvbJHy6bqlTLZXMTcUyVw1cOoPD6GA8zSejS5kFEPl29BzYWh2hd4jR0PRxRWPoigYrn3iMxq33EtAlFj32ArBPmtfbXDG72HKXYhFNqtXJSgQnjIUiE9OMD0= ceph-1e913ae0-09c2-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-04T03:00:24.248 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDS2jk0AYNqqUa9VytIdCOlrdvOzBQigMpHBQjA5Pd3R083U+v/xHXVYxu8B1sjJ8maoUB4twM8jAxF2pHacZBbA/yvYMy3lZraAD+MD/s0mx4iJX8uZ64TM8F/EZZ6jdTJzuFXNkw30Rv4CfFxD2Ae947qZHXp3Q154AW1KQoyFUsj0ixmKAgqxw//GWU7cQsSVM3vp6K+vyhN41qxbU5BNCDkpJXmgRaHYcZyVjUTdTv5GY/dFWGKmy00LmcH72XY6RUczEOBodiOgaJYrEEERlRmbWON/1GLFWTClDcJqZpaBUev8HE8k5bZ567L/R6BQGfV4XOnYl294CAkCSelqX4BpOtELLvR39POl+3+FVQOMLsWnLOrrxAHtfBGuxAQdS+5DzCvbJHy6bqlTLZXMTcUyVw1cOoPD6GA8zSejS5kFEPl29BzYWh2hd4jR0PRxRWPoigYrn3iMxq33EtAlFj32ArBPmtfbXDG72HKXYhFNqtXJSgQnjIUiE9OMD0= ceph-1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:00:24.261 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-04T03:00:24.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:24 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2750457674' entity='client.admin' 2024-05-04T03:00:24.509 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:00:26.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 -- 172.21.15.39:0/2804083377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 msgr2=0x7efc881034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:26.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 --2- 172.21.15.39:0/2804083377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 0x7efc881034c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009910 tx=0x7efc7c009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 -- 172.21.15.39:0/2804083377 shutdown_connections 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 --2- 172.21.15.39:0/2804083377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 0x7efc881034c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 -- 172.21.15.39:0/2804083377 >> 172.21.15.39:0/2804083377 conn(0x7efc880fd8c0 msgr2=0x7efc880ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 -- 172.21.15.39:0/2804083377 shutdown_connections 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 -- 172.21.15.39:0/2804083377 wait complete. 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 Processor -- start 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.648+0000 7efc8ecc5700 1 -- start start 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc8ecc5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 0x7efc88198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:26.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc8ecc5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc88108950 con 0x7efc881030b0 2024-05-04T03:00:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc8ca61700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 0x7efc88198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc8ca61700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 0x7efc88198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60404/0 (socket says 172.21.15.39:60404) 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc8ca61700 1 -- 172.21.15.39:0/441827569 learned_addr learned my addr 172.21.15.39:0/441827569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc8ca61700 1 -- 172.21.15.39:0/441827569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc88198aa0 con 0x7efc881030b0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc8ca61700 1 --2- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 0x7efc88198560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc7c005760 tx=0x7efc7c004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc85ffb700 1 -- 172.21.15.39:0/441827569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc7c027070 con 0x7efc881030b0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc88198d30 con 0x7efc881030b0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc85ffb700 1 -- 172.21.15.39:0/441827569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc7c00bdf0 con 0x7efc881030b0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.649+0000 7efc85ffb700 1 -- 172.21.15.39:0/441827569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc7c00fd70 con 0x7efc881030b0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.650+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8819b940 con 0x7efc881030b0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.651+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc881917a0 con 0x7efc881030b0 2024-05-04T03:00:26.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.651+0000 7efc85ffb700 1 -- 172.21.15.39:0/441827569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7efc7c02a070 con 0x7efc881030b0 2024-05-04T03:00:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.651+0000 7efc85ffb700 1 --2- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc7803e510 0x7efc78040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.651+0000 7efc85ffb700 1 -- 172.21.15.39:0/441827569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efc7c0538b0 con 0x7efc881030b0 2024-05-04T03:00:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.651+0000 7efc87fff700 1 --2- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc7803e510 0x7efc78040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:26.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.652+0000 7efc87fff700 1 --2- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc7803e510 0x7efc78040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efc74006fd0 tx=0x7efc74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:26.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.657+0000 7efc85ffb700 1 -- 172.21.15.39:0/441827569 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc7c013070 con 0x7efc881030b0 2024-05-04T03:00:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.869+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7efc8802e700 con 0x7efc881030b0 2024-05-04T03:00:26.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.872+0000 7efc85ffb700 1 -- 172.21.15.39:0/441827569 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7efc7c0310c0 con 0x7efc881030b0 2024-05-04T03:00:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.874+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc7803e510 msgr2=0x7efc78040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.874+0000 7efc8ecc5700 1 --2- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc7803e510 0x7efc78040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efc74006fd0 tx=0x7efc74006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.874+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 msgr2=0x7efc88198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.874+0000 7efc8ecc5700 1 --2- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 0x7efc88198560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc7c005760 tx=0x7efc7c004f60 comp rx=0 tx=0).stop 2024-05-04T03:00:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.874+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 shutdown_connections 2024-05-04T03:00:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.874+0000 7efc8ecc5700 1 --2- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc7803e510 0x7efc78040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.874+0000 7efc8ecc5700 1 --2- 172.21.15.39:0/441827569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc881030b0 0x7efc88198560 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.875+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 >> 172.21.15.39:0/441827569 conn(0x7efc880fd8c0 msgr2=0x7efc880fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.875+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 shutdown_connections 2024-05-04T03:00:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:26.875+0000 7efc8ecc5700 1 -- 172.21.15.39:0/441827569 wait complete. 2024-05-04T03:00:27.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:27.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:27.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:00:27.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:27.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:00:27.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:27.444 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-04T03:00:27.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-04T03:00:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:00:28.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:27 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/441827569' entity='client.admin' 2024-05-04T03:00:28.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:00:28.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:29.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.155+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/2522745735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 msgr2=0x7fa2c8103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:29.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.155+0000 7fa2cf4b1700 1 --2- 172.21.15.39:0/2522745735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 0x7fa2c8103ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009910 tx=0x7fa2c4009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:29.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.155+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/2522745735 shutdown_connections 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.155+0000 7fa2cf4b1700 1 --2- 172.21.15.39:0/2522745735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 0x7fa2c8103ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.155+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/2522745735 >> 172.21.15.39:0/2522745735 conn(0x7fa2c80ff220 msgr2=0x7fa2c81016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.155+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/2522745735 shutdown_connections 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.155+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/2522745735 wait complete. 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cf4b1700 1 Processor -- start 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cf4b1700 1 -- start start 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cf4b1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 0x7fa2c8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cf4b1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c8109a50 con 0x7fa2c8103be0 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cd24d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 0x7fa2c8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cd24d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 0x7fa2c8199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60426/0 (socket says 172.21.15.39:60426) 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cd24d700 1 -- 172.21.15.39:0/1859674318 learned_addr learned my addr 172.21.15.39:0/1859674318 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:29.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cd24d700 1 -- 172.21.15.39:0/1859674318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c8199b80 con 0x7fa2c8103be0 2024-05-04T03:00:29.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.156+0000 7fa2cd24d700 1 --2- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 0x7fa2c8199640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4005950 tx=0x7fa2c4005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:29.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.157+0000 7fa2be7fc700 1 -- 172.21.15.39:0/1859674318 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c4027070 con 0x7fa2c8103be0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.157+0000 7fa2be7fc700 1 -- 172.21.15.39:0/1859674318 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2c400bdf0 con 0x7fa2c8103be0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.157+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c8199e10 con 0x7fa2c8103be0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.157+0000 7fa2be7fc700 1 -- 172.21.15.39:0/1859674318 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c400fdb0 con 0x7fa2c8103be0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.157+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c819ca20 con 0x7fa2c8103be0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.158+0000 7fa2be7fc700 1 -- 172.21.15.39:0/1859674318 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa2c402a070 con 0x7fa2c8103be0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.158+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c810a1d0 con 0x7fa2c8103be0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.158+0000 7fa2be7fc700 1 --2- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2b403e510 0x7fa2b4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.158+0000 7fa2be7fc700 1 -- 172.21.15.39:0/1859674318 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa2c4053970 con 0x7fa2c8103be0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.158+0000 7fa2cca4c700 1 --2- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2b403e510 0x7fa2b4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:29.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.159+0000 7fa2cca4c700 1 --2- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2b403e510 0x7fa2b4040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8006fd0 tx=0x7fa2b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:29.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.163+0000 7fa2be7fc700 1 -- 172.21.15.39:0/1859674318 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2c4013070 con 0x7fa2c8103be0 2024-05-04T03:00:29.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.383+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa2c802e700 con 0x7fa2b403e510 2024-05-04T03:00:29.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.387+0000 7fa2be7fc700 1 -- 172.21.15.39:0/1859674318 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa2c802e700 con 0x7fa2b403e510 2024-05-04T03:00:29.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2b403e510 msgr2=0x7fa2b4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:29.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 --2- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2b403e510 0x7fa2b4040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8006fd0 tx=0x7fa2b8006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:29.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 msgr2=0x7fa2c8199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:29.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 --2- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 0x7fa2c8199640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4005950 tx=0x7fa2c4005810 comp rx=0 tx=0).stop 2024-05-04T03:00:29.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 shutdown_connections 2024-05-04T03:00:29.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 --2- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2b403e510 0x7fa2b4040a20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:29.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 --2- 172.21.15.39:0/1859674318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c8103be0 0x7fa2c8199640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:29.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 >> 172.21.15.39:0/1859674318 conn(0x7fa2c80ff220 msgr2=0x7fa2c80fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:29.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 shutdown_connections 2024-05-04T03:00:29.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:29.390+0000 7fa2cf4b1700 1 -- 172.21.15.39:0/1859674318 wait complete. 2024-05-04T03:00:30.148 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi066 2024-05-04T03:00:30.148 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T03:00:30.148 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.conf 2024-05-04T03:00:30.168 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T03:00:30.168 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:00:30.227 INFO:tasks.cephadm:Adding host smithi066 to orchestrator... 2024-05-04T03:00:30.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch host add smithi066 2024-05-04T03:00:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:00:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:30 smithi039 ceph-mon[29715]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:00:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:00:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:00:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:00:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:31.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:31 smithi039 ceph-mon[29715]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-04T03:00:31.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:31 smithi039 ceph-mon[29715]: Updating smithi039:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:00:31.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:31 smithi039 ceph-mon[29715]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:00:31.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:31 smithi039 ceph-mon[29715]: Updating smithi039:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-04T03:00:31.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.929+0000 7efdb422a700 1 -- 172.21.15.39:0/1000299847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 msgr2=0x7efdac105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:31.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.929+0000 7efdb422a700 1 --2- 172.21.15.39:0/1000299847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 0x7efdac105100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efda8009910 tx=0x7efda8009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.929+0000 7efdb422a700 1 -- 172.21.15.39:0/1000299847 shutdown_connections 2024-05-04T03:00:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.929+0000 7efdb422a700 1 --2- 172.21.15.39:0/1000299847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 0x7efdac105100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.929+0000 7efdb422a700 1 -- 172.21.15.39:0/1000299847 >> 172.21.15.39:0/1000299847 conn(0x7efdac0785b0 msgr2=0x7efdac078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.930+0000 7efdb422a700 1 -- 172.21.15.39:0/1000299847 shutdown_connections 2024-05-04T03:00:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.930+0000 7efdb422a700 1 -- 172.21.15.39:0/1000299847 wait complete. 2024-05-04T03:00:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.930+0000 7efdb422a700 1 Processor -- start 2024-05-04T03:00:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.930+0000 7efdb422a700 1 -- start start 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb422a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 0x7efdac19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb422a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10e000 con 0x7efdac104cf0 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb1fc6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 0x7efdac19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb1fc6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 0x7efdac19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60452/0 (socket says 172.21.15.39:60452) 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb1fc6700 1 -- 172.21.15.39:0/823838169 learned_addr learned my addr 172.21.15.39:0/823838169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb1fc6700 1 -- 172.21.15.39:0/823838169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac19faa0 con 0x7efdac104cf0 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb1fc6700 1 --2- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 0x7efdac19f560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efda8005060 tx=0x7efda8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efda2ffd700 1 -- 172.21.15.39:0/823838169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efda8026070 con 0x7efdac104cf0 2024-05-04T03:00:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efda2ffd700 1 -- 172.21.15.39:0/823838169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efda800b750 con 0x7efdac104cf0 2024-05-04T03:00:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac19fd30 con 0x7efdac104cf0 2024-05-04T03:00:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efda2ffd700 1 -- 172.21.15.39:0/823838169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efda8003850 con 0x7efdac104cf0 2024-05-04T03:00:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.931+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac1996f0 con 0x7efdac104cf0 2024-05-04T03:00:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.932+0000 7efda2ffd700 1 -- 172.21.15.39:0/823838169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7efda8029070 con 0x7efdac104cf0 2024-05-04T03:00:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.932+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdac06b670 con 0x7efdac104cf0 2024-05-04T03:00:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.933+0000 7efda2ffd700 1 --2- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd9803e510 0x7efd98040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.933+0000 7efda2ffd700 1 -- 172.21.15.39:0/823838169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efda801e070 con 0x7efdac104cf0 2024-05-04T03:00:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.933+0000 7efdb17c5700 1 --2- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd9803e510 0x7efd98040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.933+0000 7efdb17c5700 1 --2- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd9803e510 0x7efd98040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006fd0 tx=0x7efd9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:31.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:31.938+0000 7efda2ffd700 1 -- 172.21.15.39:0/823838169 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efda8013070 con 0x7efdac104cf0 2024-05-04T03:00:32.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:32.156+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}) -- 0x7efdac19ac00 con 0x7efd9803e510 2024-05-04T03:00:32.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:32 smithi039 ceph-mon[29715]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:00:33.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:33 smithi039 ceph-mon[29715]: Deploying cephadm binary to smithi066 2024-05-04T03:00:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:35 smithi039 ceph-mon[29715]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:36.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.454+0000 7efda2ffd700 1 -- 172.21.15.39:0/823838169 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7efdac19ac00 con 0x7efd9803e510 2024-05-04T03:00:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi066' with addr '172.21.15.66' 2024-05-04T03:00:36.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd9803e510 msgr2=0x7efd98040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:36.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 --2- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd9803e510 0x7efd98040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006fd0 tx=0x7efd9c006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:36.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 msgr2=0x7efdac19f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 --2- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 0x7efdac19f560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efda8005060 tx=0x7efda8005710 comp rx=0 tx=0).stop 2024-05-04T03:00:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 shutdown_connections 2024-05-04T03:00:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 --2- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd9803e510 0x7efd98040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 --2- 172.21.15.39:0/823838169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac104cf0 0x7efdac19f560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 >> 172.21.15.39:0/823838169 conn(0x7efdac0785b0 msgr2=0x7efdac105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 shutdown_connections 2024-05-04T03:00:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:36.457+0000 7efdb422a700 1 -- 172.21.15.39:0/823838169 wait complete. 2024-05-04T03:00:37.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-04T03:00:37.331 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:00:37.751 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:37 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:37.751 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:37 smithi039 ceph-mon[29715]: Added host smithi066 2024-05-04T03:00:37.751 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:37 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:00:37.752 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:37 smithi039 ceph-mon[29715]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:37.752 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:37 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:38.841 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.898+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/3190909630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 msgr2=0x7ff4c81041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.898+0000 7ff4cf4d8700 1 --2- 172.21.15.39:0/3190909630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 0x7ff4c81041e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009910 tx=0x7ff4b8009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.898+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/3190909630 shutdown_connections 2024-05-04T03:00:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.898+0000 7ff4cf4d8700 1 --2- 172.21.15.39:0/3190909630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 0x7ff4c81041e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.898+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/3190909630 >> 172.21.15.39:0/3190909630 conn(0x7ff4c80ff660 msgr2=0x7ff4c8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.898+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/3190909630 shutdown_connections 2024-05-04T03:00:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.898+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/3190909630 wait complete. 2024-05-04T03:00:38.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.899+0000 7ff4cf4d8700 1 Processor -- start 2024-05-04T03:00:38.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.899+0000 7ff4cf4d8700 1 -- start start 2024-05-04T03:00:38.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.899+0000 7ff4cf4d8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 0x7ff4c819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:38.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.899+0000 7ff4cf4d8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c810df70 con 0x7ff4c8103dd0 2024-05-04T03:00:38.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.899+0000 7ff4cd274700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 0x7ff4c819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:38.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.899+0000 7ff4cd274700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 0x7ff4c819f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55754/0 (socket says 172.21.15.39:55754) 2024-05-04T03:00:38.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.899+0000 7ff4cd274700 1 -- 172.21.15.39:0/2503416328 learned_addr learned my addr 172.21.15.39:0/2503416328 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.900+0000 7ff4cd274700 1 -- 172.21.15.39:0/2503416328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c819fab0 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.900+0000 7ff4cd274700 1 --2- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 0x7ff4c819f570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8005660 tx=0x7ff4b8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.900+0000 7ff4be7fc700 1 -- 172.21.15.39:0/2503416328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4b8026070 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.900+0000 7ff4be7fc700 1 -- 172.21.15.39:0/2503416328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4b800b6f0 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.900+0000 7ff4be7fc700 1 -- 172.21.15.39:0/2503416328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4b80037b0 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.900+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c819fda0 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.900+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c81998d0 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.901+0000 7ff4be7fc700 1 -- 172.21.15.39:0/2503416328 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff4b8029070 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.901+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c8105280 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.901+0000 7ff4be7fc700 1 --2- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff4b403e570 0x7ff4b4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.901+0000 7ff4be7fc700 1 -- 172.21.15.39:0/2503416328 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff4b801e070 con 0x7ff4c8103dd0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.901+0000 7ff4cca73700 1 --2- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff4b403e570 0x7ff4b4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:38.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.902+0000 7ff4cca73700 1 --2- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff4b403e570 0x7ff4b4040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4006fd0 tx=0x7ff4c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:38.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:38.907+0000 7ff4be7fc700 1 -- 172.21.15.39:0/2503416328 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4b8013070 con 0x7ff4c8103dd0 2024-05-04T03:00:39.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.126+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4c819ffd0 con 0x7ff4b403e570 2024-05-04T03:00:39.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.127+0000 7ff4be7fc700 1 -- 172.21.15.39:0/2503416328 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7ff4c819ffd0 con 0x7ff4b403e570 2024-05-04T03:00:39.128 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:39.128 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}] 2024-05-04T03:00:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff4b403e570 msgr2=0x7ff4b4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 --2- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff4b403e570 0x7ff4b4040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4006fd0 tx=0x7ff4c4006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 msgr2=0x7ff4c819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 --2- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 0x7ff4c819f570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8005660 tx=0x7ff4b8005740 comp rx=0 tx=0).stop 2024-05-04T03:00:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 shutdown_connections 2024-05-04T03:00:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 --2- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff4b403e570 0x7ff4b4040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 --2- 172.21.15.39:0/2503416328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8103dd0 0x7ff4c819f570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 >> 172.21.15.39:0/2503416328 conn(0x7ff4c80ff660 msgr2=0x7ff4c8109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:39.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 shutdown_connections 2024-05-04T03:00:39.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:39.130+0000 7ff4cf4d8700 1 -- 172.21.15.39:0/2503416328 wait complete. 2024-05-04T03:00:39.722 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-05-04T03:00:39.722 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T03:00:39.722 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-05-04T03:00:39.741 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T03:00:39.741 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:00:39.799 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-05-04T03:00:39.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch host add smithi167 2024-05-04T03:00:40.040 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:00:40.423 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:40 smithi039 ceph-mon[29715]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:40.423 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:40 smithi039 ceph-mon[29715]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:00:40.423 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.533+0000 7f0bd0f80700 1 -- 172.21.15.39:0/4009785263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 msgr2=0x7f0bcc108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.533+0000 7f0bd0f80700 1 --2- 172.21.15.39:0/4009785263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 0x7f0bcc108ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009910 tx=0x7f0bc0009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.533+0000 7f0bd0f80700 1 -- 172.21.15.39:0/4009785263 shutdown_connections 2024-05-04T03:00:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.533+0000 7f0bd0f80700 1 --2- 172.21.15.39:0/4009785263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 0x7f0bcc108ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.533+0000 7f0bd0f80700 1 -- 172.21.15.39:0/4009785263 >> 172.21.15.39:0/4009785263 conn(0x7f0bcc0fe5b0 msgr2=0x7f0bcc100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.533+0000 7f0bd0f80700 1 -- 172.21.15.39:0/4009785263 shutdown_connections 2024-05-04T03:00:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.533+0000 7f0bd0f80700 1 -- 172.21.15.39:0/4009785263 wait complete. 2024-05-04T03:00:41.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bd0f80700 1 Processor -- start 2024-05-04T03:00:41.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bd0f80700 1 -- start start 2024-05-04T03:00:41.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bd0f80700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 0x7f0bcc19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:41.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bd0f80700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc10cec0 con 0x7f0bcc108ac0 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 0x7f0bcc19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 0x7f0bcc19f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55766/0 (socket says 172.21.15.39:55766) 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bca59c700 1 -- 172.21.15.39:0/1699371575 learned_addr learned my addr 172.21.15.39:0/1699371575 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bca59c700 1 -- 172.21.15.39:0/1699371575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc19fa90 con 0x7f0bcc108ac0 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.534+0000 7f0bca59c700 1 --2- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 0x7f0bcc19f550 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0005660 tx=0x7f0bc0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.535+0000 7f0bbb7fe700 1 -- 172.21.15.39:0/1699371575 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bc0026070 con 0x7f0bcc108ac0 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.535+0000 7f0bbb7fe700 1 -- 172.21.15.39:0/1699371575 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc000b6f0 con 0x7f0bcc108ac0 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.535+0000 7f0bbb7fe700 1 -- 172.21.15.39:0/1699371575 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bc00037d0 con 0x7f0bcc108ac0 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.535+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc19fd80 con 0x7f0bcc108ac0 2024-05-04T03:00:41.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.535+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc1998b0 con 0x7f0bcc108ac0 2024-05-04T03:00:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.536+0000 7f0bbb7fe700 1 -- 172.21.15.39:0/1699371575 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0bc0029070 con 0x7f0bcc108ac0 2024-05-04T03:00:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.536+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc1041d0 con 0x7f0bcc108ac0 2024-05-04T03:00:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.536+0000 7f0bbb7fe700 1 --2- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0bb403e510 0x7f0bb4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.536+0000 7f0bbb7fe700 1 -- 172.21.15.39:0/1699371575 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0bc001e070 con 0x7f0bcc108ac0 2024-05-04T03:00:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.536+0000 7f0bc9d9b700 1 --2- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0bb403e510 0x7f0bb4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.537+0000 7f0bc9d9b700 1 --2- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0bb403e510 0x7f0bb4040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006fd0 tx=0x7f0bbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.541+0000 7f0bbb7fe700 1 -- 172.21.15.39:0/1699371575 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0bc0013070 con 0x7f0bcc108ac0 2024-05-04T03:00:41.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:41.760+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) -- 0x7f0bcc19ac20 con 0x7f0bb403e510 2024-05-04T03:00:42.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:42 smithi039 ceph-mon[29715]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:43.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:43 smithi039 ceph-mon[29715]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:00:43.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:43 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:43.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:43 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:43.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:43 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:43.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:43 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:43.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:43 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:00:43.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:43 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:00:43.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:43 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:00:44.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:44 smithi039 ceph-mon[29715]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-04T03:00:44.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:44 smithi039 ceph-mon[29715]: Deploying cephadm binary to smithi167 2024-05-04T03:00:44.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:44 smithi039 ceph-mon[29715]: Updating smithi066:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:00:44.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:44 smithi039 ceph-mon[29715]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:44.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:44 smithi039 ceph-mon[29715]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:00:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:45 smithi039 ceph-mon[29715]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.169+0000 7f0bbb7fe700 1 -- 172.21.15.39:0/1699371575 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0bcc19ac20 con 0x7f0bb403e510 2024-05-04T03:00:46.171 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0bb403e510 msgr2=0x7f0bb4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 --2- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0bb403e510 0x7f0bb4040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006fd0 tx=0x7f0bbc006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 msgr2=0x7f0bcc19f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 --2- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 0x7f0bcc19f550 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0005660 tx=0x7f0bc0005740 comp rx=0 tx=0).stop 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 shutdown_connections 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 --2- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0bb403e510 0x7f0bb4040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 --2- 172.21.15.39:0/1699371575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bcc108ac0 0x7f0bcc19f550 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 >> 172.21.15.39:0/1699371575 conn(0x7f0bcc0fe5b0 msgr2=0x7f0bcc1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 shutdown_connections 2024-05-04T03:00:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:46.172+0000 7f0bd0f80700 1 -- 172.21.15.39:0/1699371575 wait complete. 2024-05-04T03:00:46.821 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-04T03:00:47.064 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:00:47.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:46 smithi039 ceph-mon[29715]: Updating smithi066:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-04T03:00:47.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:47.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:47.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:47.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:47.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:46 smithi039 ceph-mon[29715]: Added host smithi167 2024-05-04T03:00:47.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:00:47.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:48.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:48 smithi039 ceph-mon[29715]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:48.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:48 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:48.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.666+0000 7f491f817700 1 -- 172.21.15.39:0/701504455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 msgr2=0x7f4918108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:48.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.666+0000 7f491f817700 1 --2- 172.21.15.39:0/701504455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 0x7f4918108f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4914009910 tx=0x7f4914009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:48.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.666+0000 7f491f817700 1 -- 172.21.15.39:0/701504455 shutdown_connections 2024-05-04T03:00:48.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.666+0000 7f491f817700 1 --2- 172.21.15.39:0/701504455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 0x7f4918108f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:48.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.666+0000 7f491f817700 1 -- 172.21.15.39:0/701504455 >> 172.21.15.39:0/701504455 conn(0x7f49180fde30 msgr2=0x7f4918100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:48.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.666+0000 7f491f817700 1 -- 172.21.15.39:0/701504455 shutdown_connections 2024-05-04T03:00:48.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.666+0000 7f491f817700 1 -- 172.21.15.39:0/701504455 wait complete. 2024-05-04T03:00:48.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.667+0000 7f491f817700 1 Processor -- start 2024-05-04T03:00:48.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.667+0000 7f491f817700 1 -- start start 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.667+0000 7f491f817700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 0x7f491819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.667+0000 7f491f817700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f491810e0c0 con 0x7f4918108b70 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.667+0000 7f491d5b3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 0x7f491819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.667+0000 7f491d5b3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 0x7f491819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35442/0 (socket says 172.21.15.39:35442) 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.667+0000 7f491d5b3700 1 -- 172.21.15.39:0/3022468039 learned_addr learned my addr 172.21.15.39:0/3022468039 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.667+0000 7f491d5b3700 1 -- 172.21.15.39:0/3022468039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491819fad0 con 0x7f4918108b70 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.668+0000 7f491d5b3700 1 --2- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 0x7f491819f590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4914005660 tx=0x7f4914005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.668+0000 7f490e7fc700 1 -- 172.21.15.39:0/3022468039 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4914026070 con 0x7f4918108b70 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.668+0000 7f490e7fc700 1 -- 172.21.15.39:0/3022468039 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f491400b750 con 0x7f4918108b70 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.668+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491819fd60 con 0x7f4918108b70 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.668+0000 7f490e7fc700 1 -- 172.21.15.39:0/3022468039 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4914003810 con 0x7f4918108b70 2024-05-04T03:00:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.668+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4918199720 con 0x7f4918108b70 2024-05-04T03:00:48.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.669+0000 7f490e7fc700 1 -- 172.21.15.39:0/3022468039 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4914029070 con 0x7f4918108b70 2024-05-04T03:00:48.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.669+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4918105390 con 0x7f4918108b70 2024-05-04T03:00:48.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.669+0000 7f490e7fc700 1 --2- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f490403e510 0x7f4904040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:48.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.669+0000 7f490e7fc700 1 -- 172.21.15.39:0/3022468039 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f491401e070 con 0x7f4918108b70 2024-05-04T03:00:48.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.669+0000 7f491cdb2700 1 --2- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f490403e510 0x7f4904040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:48.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.670+0000 7f491cdb2700 1 --2- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f490403e510 0x7f4904040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4908006fd0 tx=0x7f4908006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:48.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.674+0000 7f490e7fc700 1 -- 172.21.15.39:0/3022468039 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4914013070 con 0x7f4918108b70 2024-05-04T03:00:48.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.911+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f491819ac30 con 0x7f490403e510 2024-05-04T03:00:48.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.912+0000 7f490e7fc700 1 -- 172.21.15.39:0/3022468039 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f491819ac30 con 0x7f490403e510 2024-05-04T03:00:48.913 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:00:48.913 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-05-04T03:00:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f490403e510 msgr2=0x7f4904040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 --2- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f490403e510 0x7f4904040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4908006fd0 tx=0x7f4908006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 msgr2=0x7f491819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 --2- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 0x7f491819f590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4914005660 tx=0x7f4914005740 comp rx=0 tx=0).stop 2024-05-04T03:00:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 shutdown_connections 2024-05-04T03:00:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 --2- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f490403e510 0x7f4904040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:48.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 --2- 172.21.15.39:0/3022468039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4918108b70 0x7f491819f590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:48.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 >> 172.21.15.39:0/3022468039 conn(0x7f49180fde30 msgr2=0x7f4918105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:48.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 shutdown_connections 2024-05-04T03:00:48.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:48.914+0000 7f491f817700 1 -- 172.21.15.39:0/3022468039 wait complete. 2024-05-04T03:00:49.561 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-04T03:00:49.561 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-04T03:00:49.806 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:00:49.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:49.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:49 smithi039 ceph-mon[29715]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:49.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:49 smithi039 ceph-mon[29715]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:00:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.359+0000 7fb5f031a700 1 -- 172.21.15.39:0/3070225689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 msgr2=0x7fb5e8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.359+0000 7fb5f031a700 1 --2- 172.21.15.39:0/3070225689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 0x7fb5e8108e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009910 tx=0x7fb5e4009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.359+0000 7fb5f031a700 1 -- 172.21.15.39:0/3070225689 shutdown_connections 2024-05-04T03:00:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.359+0000 7fb5f031a700 1 --2- 172.21.15.39:0/3070225689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 0x7fb5e8108e70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.359+0000 7fb5f031a700 1 -- 172.21.15.39:0/3070225689 >> 172.21.15.39:0/3070225689 conn(0x7fb5e80fe5b0 msgr2=0x7fb5e8100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.359+0000 7fb5f031a700 1 -- 172.21.15.39:0/3070225689 shutdown_connections 2024-05-04T03:00:51.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.359+0000 7fb5f031a700 1 -- 172.21.15.39:0/3070225689 wait complete. 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.360+0000 7fb5f031a700 1 Processor -- start 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.360+0000 7fb5f031a700 1 -- start start 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.360+0000 7fb5f031a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 0x7fb5e819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.360+0000 7fb5f031a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e810ce60 con 0x7fb5e8108a60 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.360+0000 7fb5ee0b6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 0x7fb5e819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.360+0000 7fb5ee0b6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 0x7fb5e819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35458/0 (socket says 172.21.15.39:35458) 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.360+0000 7fb5ee0b6700 1 -- 172.21.15.39:0/3275324883 learned_addr learned my addr 172.21.15.39:0/3275324883 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.361+0000 7fb5ee0b6700 1 -- 172.21.15.39:0/3275324883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e819faf0 con 0x7fb5e8108a60 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.361+0000 7fb5ee0b6700 1 --2- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 0x7fb5e819f5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4005660 tx=0x7fb5e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:51.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.361+0000 7fb5df7fe700 1 -- 172.21.15.39:0/3275324883 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5e4026070 con 0x7fb5e8108a60 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.361+0000 7fb5df7fe700 1 -- 172.21.15.39:0/3275324883 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5e400b6f0 con 0x7fb5e8108a60 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.361+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e819fde0 con 0x7fb5e8108a60 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.361+0000 7fb5df7fe700 1 -- 172.21.15.39:0/3275324883 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5e40037d0 con 0x7fb5e8108a60 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.361+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e8199790 con 0x7fb5e8108a60 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.362+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e8104170 con 0x7fb5e8108a60 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.362+0000 7fb5df7fe700 1 -- 172.21.15.39:0/3275324883 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb5e4029070 con 0x7fb5e8108a60 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.362+0000 7fb5df7fe700 1 --2- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5d403e510 0x7fb5d4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.362+0000 7fb5df7fe700 1 -- 172.21.15.39:0/3275324883 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb5e401e070 con 0x7fb5e8108a60 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.363+0000 7fb5ed8b5700 1 --2- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5d403e510 0x7fb5d4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:51.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.363+0000 7fb5ed8b5700 1 --2- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5d403e510 0x7fb5d4040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8006fd0 tx=0x7fb5d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:51.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.368+0000 7fb5df7fe700 1 -- 172.21.15.39:0/3275324883 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5e4013070 con 0x7fb5e8108a60 2024-05-04T03:00:51.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.589+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb5e8064e50 con 0x7fb5e8108a60 2024-05-04T03:00:51.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.945+0000 7fb5df7fe700 1 -- 172.21.15.39:0/3275324883 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb5e8064e50 con 0x7fb5e8108a60 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5d403e510 msgr2=0x7fb5d4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 --2- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5d403e510 0x7fb5d4040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8006fd0 tx=0x7fb5d8006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 msgr2=0x7fb5e819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 --2- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 0x7fb5e819f5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4005660 tx=0x7fb5e4005740 comp rx=0 tx=0).stop 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 shutdown_connections 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 --2- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5d403e510 0x7fb5d4040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 --2- 172.21.15.39:0/3275324883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5e8108a60 0x7fb5e819f5b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 >> 172.21.15.39:0/3275324883 conn(0x7fb5e80fe5b0 msgr2=0x7fb5e8106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 shutdown_connections 2024-05-04T03:00:51.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:00:51.947+0000 7fb5f031a700 1 -- 172.21.15.39:0/3275324883 wait complete. 2024-05-04T03:00:51.949 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-05-04T03:00:52.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:52.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3275324883' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-04T03:00:52.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:52.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:52.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:52.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:52.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:00:52.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:00:52.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:00:52.638 INFO:tasks.cephadm:Adding mon.a on smithi039 2024-05-04T03:00:52.638 INFO:tasks.cephadm:Adding mon.b on smithi066 2024-05-04T03:00:52.638 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-05-04T03:00:52.639 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi039:172.21.15.39=a;smithi066:172.21.15.66=b;smithi167:172.21.15.167=c' 2024-05-04T03:00:52.869 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:00:53.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:52 smithi039 ceph-mon[29715]: Updating smithi167:/etc/ceph/ceph.conf 2024-05-04T03:00:53.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:52 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3275324883' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-04T03:00:53.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:52 smithi039 ceph-mon[29715]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:00:53.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:52 smithi039 ceph-mon[29715]: Updating smithi167:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:00:53.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:52 smithi039 ceph-mon[29715]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:00:53.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:53.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:53.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:54.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.294+0000 7fedf49d9700 1 -- 172.21.15.167:0/2742774815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 msgr2=0x7fedf01041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:54.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.294+0000 7fedf49d9700 1 --2- 172.21.15.167:0/2742774815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 0x7fedf01041e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fede4009910 tx=0x7fede4009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:54.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.294+0000 7fedf49d9700 1 -- 172.21.15.167:0/2742774815 shutdown_connections 2024-05-04T03:00:54.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.294+0000 7fedf49d9700 1 --2- 172.21.15.167:0/2742774815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 0x7fedf01041e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:54.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.294+0000 7fedf49d9700 1 -- 172.21.15.167:0/2742774815 >> 172.21.15.167:0/2742774815 conn(0x7fedf00ff660 msgr2=0x7fedf0101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:54.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.294+0000 7fedf49d9700 1 -- 172.21.15.167:0/2742774815 shutdown_connections 2024-05-04T03:00:54.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.294+0000 7fedf49d9700 1 -- 172.21.15.167:0/2742774815 wait complete. 2024-05-04T03:00:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.295+0000 7fedf49d9700 1 Processor -- start 2024-05-04T03:00:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.295+0000 7fedf49d9700 1 -- start start 2024-05-04T03:00:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.295+0000 7fedf49d9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 0x7fedf019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.295+0000 7fedf49d9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fedf010df70 con 0x7fedf0103dd0 2024-05-04T03:00:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.296+0000 7fedee59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 0x7fedf019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:54.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.296+0000 7fedee59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 0x7fedf019f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.167:46700/0 (socket says 172.21.15.167:46700) 2024-05-04T03:00:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.296+0000 7fedee59c700 1 -- 172.21.15.167:0/2615684787 learned_addr learned my addr 172.21.15.167:0/2615684787 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:00:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.296+0000 7fedee59c700 1 -- 172.21.15.167:0/2615684787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf019fac0 con 0x7fedf0103dd0 2024-05-04T03:00:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.296+0000 7fedee59c700 1 --2- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 0x7fedf019f580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fede4005660 tx=0x7fede4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.297+0000 7feddf7fe700 1 -- 172.21.15.167:0/2615684787 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fede4026070 con 0x7fedf0103dd0 2024-05-04T03:00:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.297+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf019fdb0 con 0x7fedf0103dd0 2024-05-04T03:00:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.297+0000 7feddf7fe700 1 -- 172.21.15.167:0/2615684787 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fede400b6f0 con 0x7fedf0103dd0 2024-05-04T03:00:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.297+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01998e0 con 0x7fedf0103dd0 2024-05-04T03:00:54.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.297+0000 7feddf7fe700 1 -- 172.21.15.167:0/2615684787 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fede40037d0 con 0x7fedf0103dd0 2024-05-04T03:00:54.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.298+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf0105280 con 0x7fedf0103dd0 2024-05-04T03:00:54.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.298+0000 7feddf7fe700 1 -- 172.21.15.167:0/2615684787 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fede4029070 con 0x7fedf0103dd0 2024-05-04T03:00:54.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.298+0000 7feddf7fe700 1 --2- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fedd803e510 0x7fedd8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:54.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.298+0000 7feddf7fe700 1 -- 172.21.15.167:0/2615684787 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fede401e070 con 0x7fedf0103dd0 2024-05-04T03:00:54.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.298+0000 7fededd9b700 1 --2- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fedd803e510 0x7fedd8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:54.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.299+0000 7fededd9b700 1 --2- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fedd803e510 0x7fedd8040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fede0006fd0 tx=0x7fede0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:54.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.304+0000 7feddf7fe700 1 -- 172.21.15.167:0/2615684787 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fede4013070 con 0x7fedf0103dd0 2024-05-04T03:00:54.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:53 smithi039 ceph-mon[29715]: Updating smithi167:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-04T03:00:54.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:53 smithi039 ceph-mon[29715]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:54.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.517+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi066:172.21.15.66=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) -- 0x7fedf01a00a0 con 0x7fedd803e510 2024-05-04T03:00:54.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.526+0000 7feddf7fe700 1 -- 172.21.15.167:0/2615684787 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fedf01a00a0 con 0x7fedd803e510 2024-05-04T03:00:54.526 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-05-04T03:00:54.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fedd803e510 msgr2=0x7fedd8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:54.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 --2- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fedd803e510 0x7fedd8040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fede0006fd0 tx=0x7fede0006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:54.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 msgr2=0x7fedf019f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:54.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 --2- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 0x7fedf019f580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fede4005660 tx=0x7fede4005740 comp rx=0 tx=0).stop 2024-05-04T03:00:54.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 shutdown_connections 2024-05-04T03:00:54.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 --2- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fedd803e510 0x7fedd8040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:54.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 --2- 172.21.15.167:0/2615684787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedf0103dd0 0x7fedf019f580 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:54.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 >> 172.21.15.167:0/2615684787 conn(0x7fedf00ff660 msgr2=0x7fedf0109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:54.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 shutdown_connections 2024-05-04T03:00:54.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:54.528+0000 7fedf49d9700 1 -- 172.21.15.167:0/2615684787 wait complete. 2024-05-04T03:00:55.138 DEBUG:teuthology.orchestra.run.smithi066:mon.b> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.b.service 2024-05-04T03:00:55.141 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.c.service 2024-05-04T03:00:55.143 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-04T03:00:55.143 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-04T03:00:55.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi066:172.21.15.66=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi066:172.21.15.66=b;smithi167:172.21.15.167=c;count:3 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: Deploying daemon mon.c on smithi167 2024-05-04T03:00:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:55 smithi039 ceph-mon[29715]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:00:58.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:57 smithi039 ceph-mon[29715]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.747+0000 7f300239a700 1 -- 172.21.15.167:0/2996921531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 msgr2=0x7f2ffc105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.747+0000 7f300239a700 1 --2- 172.21.15.167:0/2996921531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 0x7f2ffc105100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009910 tx=0x7f2fec009c20 comp rx=0 tx=0).stop 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.747+0000 7f300239a700 1 -- 172.21.15.167:0/2996921531 shutdown_connections 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.747+0000 7f300239a700 1 --2- 172.21.15.167:0/2996921531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 0x7f2ffc105100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.747+0000 7f300239a700 1 -- 172.21.15.167:0/2996921531 >> 172.21.15.167:0/2996921531 conn(0x7f2ffc0785b0 msgr2=0x7f2ffc078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.747+0000 7f300239a700 1 -- 172.21.15.167:0/2996921531 shutdown_connections 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.747+0000 7f300239a700 1 -- 172.21.15.167:0/2996921531 wait complete. 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.748+0000 7f300239a700 1 Processor -- start 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.748+0000 7f300239a700 1 -- start start 2024-05-04T03:00:58.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.748+0000 7f300239a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 0x7f2ffc19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:58.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.748+0000 7f300239a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10e000 con 0x7f2ffc104cf0 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f2ffbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 0x7f2ffc19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f2ffbfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 0x7f2ffc19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.167:49332/0 (socket says 172.21.15.167:49332) 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f2ffbfff700 1 -- 172.21.15.167:0/2905926208 learned_addr learned my addr 172.21.15.167:0/2905926208 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f2ffbfff700 1 -- 172.21.15.167:0/2905926208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc19faa0 con 0x7f2ffc104cf0 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f2ffbfff700 1 --2- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 0x7f2ffc19f560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2fec005060 tx=0x7f2fec005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f2ff97fa700 1 -- 172.21.15.167:0/2905926208 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fec026070 con 0x7f2ffc104cf0 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc19fd30 con 0x7f2ffc104cf0 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f2ff97fa700 1 -- 172.21.15.167:0/2905926208 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2fec00b750 con 0x7f2ffc104cf0 2024-05-04T03:00:58.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.749+0000 7f2ff97fa700 1 -- 172.21.15.167:0/2905926208 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fec003850 con 0x7f2ffc104cf0 2024-05-04T03:00:58.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.750+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1996f0 con 0x7f2ffc104cf0 2024-05-04T03:00:58.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.751+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc06b670 con 0x7f2ffc104cf0 2024-05-04T03:00:58.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.751+0000 7f2ff97fa700 1 -- 172.21.15.167:0/2905926208 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2fec029070 con 0x7f2ffc104cf0 2024-05-04T03:00:58.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.751+0000 7f2ff97fa700 1 --2- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2fe403e510 0x7f2fe4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:00:58.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.751+0000 7f2ff97fa700 1 -- 172.21.15.167:0/2905926208 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2fec01e070 con 0x7f2ffc104cf0 2024-05-04T03:00:58.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.751+0000 7f2ffb7fe700 1 --2- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2fe403e510 0x7f2fe4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:00:58.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.752+0000 7f2ffb7fe700 1 --2- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2fe403e510 0x7f2fe4040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0006fd0 tx=0x7f2ff0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:00:58.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:58.757+0000 7f2ff97fa700 1 -- 172.21.15.167:0/2905926208 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2fec013070 con 0x7f2ffc104cf0 2024-05-04T03:00:59.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.069+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2ffc19ac00 con 0x7f2ffc104cf0 2024-05-04T03:00:59.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.069+0000 7f2ff97fa700 1 -- 172.21.15.167:0/2905926208 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f2fec030300 con 0x7f2ffc104cf0 2024-05-04T03:00:59.070 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T03:00:59.070 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","modified":"2024-05-04T02:59:17.356054Z","created":"2024-05-04T02:59:17.356054Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-04T03:00:59.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2fe403e510 msgr2=0x7f2fe4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 --2- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2fe403e510 0x7f2fe4040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0006fd0 tx=0x7f2ff0006e40 comp rx=0 tx=0).stop 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 msgr2=0x7f2ffc19f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 --2- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 0x7f2ffc19f560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2fec005060 tx=0x7f2fec005710 comp rx=0 tx=0).stop 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 shutdown_connections 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 --2- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2fe403e510 0x7f2fe4040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 --2- 172.21.15.167:0/2905926208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc104cf0 0x7f2ffc19f560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 >> 172.21.15.167:0/2905926208 conn(0x7f2ffc0785b0 msgr2=0x7f2ffc105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 shutdown_connections 2024-05-04T03:00:59.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:00:59.072+0000 7f300239a700 1 -- 172.21.15.167:0/2905926208 wait complete. 2024-05-04T03:00:59.074 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-05-04T03:00:59.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:00:59 smithi167 systemd[1]: Starting Ceph mon.c for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:01:00.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:00:59 smithi167 podman[33594]: 2024-05-04 03:00:59.753071141 +0000 UTC m=+0.017718798 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:01:00.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:00:59 smithi167 podman[33594]: 2024-05-04 03:00:59.86451768 +0000 UTC m=+0.129165324 container create e7adc71bf1ee23361ef06e2bedf82b4c1552dee8ad7ea73b0331450d0be6446f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-04T03:01:00.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 podman[33594]: 2024-05-04 03:01:00.041869672 +0000 UTC m=+0.306517316 container init e7adc71bf1ee23361ef06e2bedf82b4c1552dee8ad7ea73b0331450d0be6446f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-05-04T03:01:00.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 podman[33594]: 2024-05-04 03:01:00.044815968 +0000 UTC m=+0.309463615 container start e7adc71bf1ee23361ef06e2bedf82b4c1552dee8ad7ea73b0331450d0be6446f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-04T03:01:00.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: set uid:gid to 167:167 (ceph:ceph) 2024-05-04T03:01:00.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev), process ceph-mon, pid 2 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pidfile_write: ignore empty --pid-file 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: load: jerasure load: lrc 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: RocksDB version: 7.9.2 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Git sha 0 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Compile date 2024-05-03 18:51:23 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: DB SUMMARY 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: DB Session ID: WAMWT61R38QIFDPI2W3X 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: CURRENT file: CURRENT 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: IDENTITY file: IDENTITY 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.error_if_exists: 0 2024-05-04T03:01:00.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.create_if_missing: 0 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.paranoid_checks: 1 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.env: 0x558a3b3d0500 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.fs: PosixFileSystem 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.info_log: 0x558a3d22a020 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_file_opening_threads: 16 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.statistics: (nil) 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.use_fsync: 0 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_log_file_size: 0 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.keep_log_file_num: 1000 2024-05-04T03:01:00.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.recycle_log_file_num: 0 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.allow_fallocate: 1 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.allow_mmap_reads: 0 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.allow_mmap_writes: 0 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.use_direct_reads: 0 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.create_missing_column_families: 0 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.db_log_dir: 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.wal_dir: 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-04T03:01:00.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-04T03:01:00.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.advise_random_on_open: 1 2024-05-04T03:01:00.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.db_write_buffer_size: 0 2024-05-04T03:01:00.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.write_buffer_manager: 0x558a3d0f48c0 2024-05-04T03:01:00.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-04T03:01:00.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-04T03:01:00.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-04T03:01:00.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.rate_limiter: (nil) 2024-05-04T03:01:00.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-04T03:01:00.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.wal_recovery_mode: 2 2024-05-04T03:01:00.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.enable_thread_tracking: 0 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.enable_pipelined_write: 0 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.unordered_write: 0 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.row_cache: None 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.wal_filter: None 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.allow_ingest_behind: 0 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.two_write_queues: 0 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.manual_wal_flush: 0 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.wal_compression: 0 2024-05-04T03:01:00.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.atomic_flush: 0 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.log_readahead_size: 0 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.best_efforts_recovery: 0 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.allow_data_in_errors: 0 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.db_host_id: __hostname__ 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_background_jobs: 2 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_background_compactions: -1 2024-05-04T03:01:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_subcompactions: 1 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_total_wal_size: 0 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_open_files: -1 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bytes_per_sync: 0 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_readahead_size: 0 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_background_flushes: -1 2024-05-04T03:01:00.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Compression algorithms supported: 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: kZSTD supported: 0 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: kXpressCompression supported: 0 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: kLZ4HCCompression supported: 1 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: kZlibCompression supported: 1 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: kSnappyCompression supported: 1 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: kLZ4Compression supported: 1 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: kBZip2Compression supported: 0 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.merge_operator: 2024-05-04T03:01:00.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_filter: None 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_filter_factory: None 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.sst_partitioner_factory: None 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558a3d22a1e0) 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks: 1 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_top_level_index_and_filter: 1 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_type: 0 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_index_type: 0 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_shortening: 1 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: checksum: 4 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: no_block_cache: 0 2024-05-04T03:01:00.112 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache: 0x558a3d1271f0 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_name: BinnedLRUCache 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_options: 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: capacity : 536870912 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_shard_bits : 4 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: strict_capacity_limit : 0 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: high_pri_pool_ratio: 0.000 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_compressed: (nil) 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: persistent_cache: (nil) 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size: 4096 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size_deviation: 10 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_restart_interval: 16 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_block_restart_interval: 1 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: metadata_block_size: 4096 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: partition_filters: 0 2024-05-04T03:01:00.113 INFO:journalctl@ceph.mon.c.smithi167.stdout: use_delta_encoding: 1 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: filter_policy: bloomfilter 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: whole_key_filtering: 1 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: verify_compression: 0 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: read_amp_bytes_per_bit: 0 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: format_version: 5 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_index_compression: 1 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_align: 0 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: max_auto_readahead_size: 262144 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: prepopulate_block_cache: 0 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: initial_auto_readahead_size: 8192 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_file_reads_for_auto_readahead: 2 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.write_buffer_size: 33554432 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_write_buffer_number: 2 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression: NoCompression 2024-05-04T03:01:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression: Disabled 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.prefix_extractor: nullptr 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.num_levels: 7 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-04T03:01:00.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.level: 32767 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.strategy: 0 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.enabled: false 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.target_file_size_base: 67108864 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-04T03:01:00.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.arena_block_size: 1048576 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.disable_auto_compactions: 0 2024-05-04T03:01:00.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.inplace_update_support: 0 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-04T03:01:00.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.bloom_locality: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.max_successive_merges: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.paranoid_file_checks: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.force_consistency_checks: 1 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.report_bg_io_stats: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.ttl: 2592000 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.enable_blob_files: false 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.min_blob_size: 0 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.blob_file_size: 268435456 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-04T03:01:00.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.blob_file_starting_level: 0 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 747658a0-444c-4af1-aa1d-a00346ef7661 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714791660085788, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714791660086383, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714791660, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "747658a0-444c-4af1-aa1d-a00346ef7661", "db_session_id": "WAMWT61R38QIFDPI2W3X", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714791660086488, "job": 1, "event": "recovery_finished"} 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-04T03:01:00.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558a3d10bc00 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: DB pointer 0x558a41efe000 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** DB Stats ** 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-04T03:01:00.121 INFO:journalctl@ceph.mon.c.smithi167.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-04T03:01:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache BinnedLRUCache@0x558a3d1271f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: using public_addr v2:172.21.15.167:0/0 -> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: starting mon.c rank -1 at public addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] at bind addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(???) e0 preinit fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 bash[33594]: e7adc71bf1ee23361ef06e2bedf82b4c1552dee8ad7ea73b0331450d0be6446f 2024-05-04T03:01:00.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 systemd[1]: Started Ceph mon.c for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:01:00.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).mds e1 new map 2024-05-04T03:01:00.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-04T03:01:00.427 INFO:journalctl@ceph.mon.c.smithi167.stdout: e1 2024-05-04T03:01:00.427 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout: legacy client fscid: -1 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout: No filesystems configured 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T03:01:00.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-04T03:01:00.429 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Deploying cephadm binary to smithi167 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Updating smithi066:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Updating smithi066:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Added host smithi167 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3275324883' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Updating smithi167:/etc/ceph/ceph.conf 2024-05-04T03:01:00.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3275324883' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Updating smithi167:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Updating smithi167:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi066:172.21.15.66=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi066:172.21.15.66=b;smithi167:172.21.15.167=c;count:3 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:00.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:00.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:00.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:00.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: Deploying daemon mon.c on smithi167 2024-05-04T03:01:00.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/2905926208' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:00.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:00 smithi167 ceph-mon[33614]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-04T03:01:00.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:59 smithi039 ceph-mon[29715]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:00.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:00:59 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/2905926208' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:00.608 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-04T03:01:00.608 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-04T03:01:00.843 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:01:05.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.282+0000 7f18f8839700 1 -- 172.21.15.167:0/3316139516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0035c0 msgr2=0x7f18dc005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:05.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.282+0000 7f18f8839700 1 --2- 172.21.15.167:0/3316139516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0035c0 0x7f18dc005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18f406afd0 tx=0x7f18ec00fe60 comp rx=0 tx=0).stop 2024-05-04T03:01:05.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.282+0000 7f18f8839700 1 -- 172.21.15.167:0/3316139516 shutdown_connections 2024-05-04T03:01:05.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.282+0000 7f18f8839700 1 --2- 172.21.15.167:0/3316139516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0035c0 0x7f18dc005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:05.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.282+0000 7f18f8839700 1 -- 172.21.15.167:0/3316139516 >> 172.21.15.167:0/3316139516 conn(0x7f18f40fdfa0 msgr2=0x7f18f41003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:05.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.283+0000 7f18f8839700 1 -- 172.21.15.167:0/3316139516 shutdown_connections 2024-05-04T03:01:05.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.283+0000 7f18f8839700 1 -- 172.21.15.167:0/3316139516 wait complete. 2024-05-04T03:01:05.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f8839700 1 Processor -- start 2024-05-04T03:01:05.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f8839700 1 -- start start 2024-05-04T03:01:05.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f8839700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 0x7f18f419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:05.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f8839700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18f419fb20 0x7f18f4199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:05.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f8839700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18f410aa20 con 0x7f18f419fb20 2024-05-04T03:01:05.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f8839700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18f410a8a0 con 0x7f18dc0035c0 2024-05-04T03:01:05.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 0x7f18f419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:05.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 0x7f18f419f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57024/0 (socket says 172.21.15.167:57024) 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f259c700 1 -- 172.21.15.167:0/780820076 learned_addr learned my addr 172.21.15.167:0/780820076 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.284+0000 7f18f1d9b700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18f419fb20 0x7f18f4199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f259c700 1 -- 172.21.15.167:0/780820076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 msgr2=0x7f18f419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f259c700 1 -- 172.21.15.167:0/780820076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 msgr2=0x7f18f419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f259c700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 0x7f18f419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f259c700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 0x7f18f419f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f1d9b700 1 -- 172.21.15.167:0/780820076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 msgr2=0x7f18f419f5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f1d9b700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 0x7f18f419f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f1d9b700 1 -- 172.21.15.167:0/780820076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f41a0110 con 0x7f18f419fb20 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f1d9b700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18f419fb20 0x7f18f4199820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18e800b9b0 tx=0x7f18e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18e37fe700 1 -- 172.21.15.167:0/780820076 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f18e800c7b0 con 0x7f18f419fb20 2024-05-04T03:01:05.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18e37fe700 1 -- 172.21.15.167:0/780820076 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18e800cde0 con 0x7f18f419fb20 2024-05-04T03:01:05.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f4199e20 con 0x7f18f419fb20 2024-05-04T03:01:05.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.285+0000 7f18e37fe700 1 -- 172.21.15.167:0/780820076 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f18e8012340 con 0x7f18f419fb20 2024-05-04T03:01:05.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.286+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f419a430 con 0x7f18f419fb20 2024-05-04T03:01:05.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.287+0000 7f18e37fe700 1 -- 172.21.15.167:0/780820076 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f18e80124e0 con 0x7f18f419fb20 2024-05-04T03:01:05.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.287+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f4104270 con 0x7f18f419fb20 2024-05-04T03:01:05.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.287+0000 7f18e37fe700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18d403e2d0 0x7f18d40407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:05.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.287+0000 7f18e37fe700 1 -- 172.21.15.167:0/780820076 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f18e8051d20 con 0x7f18f419fb20 2024-05-04T03:01:05.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.287+0000 7f18f259c700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18d403e2d0 0x7f18d40407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:05.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.288+0000 7f18f259c700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18d403e2d0 0x7f18d40407e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18ec005ed0 tx=0x7f18ec001e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:05.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.292+0000 7f18e37fe700 1 -- 172.21.15.167:0/780820076 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18e80127d0 con 0x7f18f419fb20 2024-05-04T03:01:05.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:01:05.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: mon.a calling monitor election 2024-05-04T03:01:05.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:05.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: mon.c calling monitor election 2024-05-04T03:01:05.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:05.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-04T03:01:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: monmap epoch 2 2024-05-04T03:01:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: last_changed 2024-05-04T03:01:00.116457+0000 2024-05-04T03:01:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: created 2024-05-04T02:59:17.356054+0000 2024-05-04T03:01:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: min_mon_release 19 (squid) 2024-05-04T03:01:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: election_strategy: 1 2024-05-04T03:01:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-05-04T03:01:05.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-05-04T03:01:05.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: fsmap 2024-05-04T03:01:05.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:01:05.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: mgrmap e13: x(active, since 50s) 2024-05-04T03:01:05.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: overall HEALTH_OK 2024-05-04T03:01:05.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:05.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:05.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.584+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f18f419add0 con 0x7f18f419fb20 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: mon.a calling monitor election 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: mon.c calling monitor election 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:05.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: monmap epoch 2 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: last_changed 2024-05-04T03:01:00.116457+0000 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: created 2024-05-04T02:59:17.356054+0000 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: min_mon_release 19 (squid) 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: election_strategy: 1 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: fsmap 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: mgrmap e13: x(active, since 50s) 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: overall HEALTH_OK 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:05.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:05.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.584+0000 7f18e37fe700 1 -- 172.21.15.167:0/780820076 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f18e8014440 con 0x7f18f419fb20 2024-05-04T03:01:05.587 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T03:01:05.587 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","modified":"2024-05-04T03:01:00.116457Z","created":"2024-05-04T02:59:17.356054Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18d403e2d0 msgr2=0x7f18d40407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18d403e2d0 0x7f18d40407e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18ec005ed0 tx=0x7f18ec001e70 comp rx=0 tx=0).stop 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18f419fb20 msgr2=0x7f18f4199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18f419fb20 0x7f18f4199820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18e800b9b0 tx=0x7f18e800bd70 comp rx=0 tx=0).stop 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 shutdown_connections 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18d403e2d0 0x7f18d40407e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18dc0035c0 0x7f18f419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 --2- 172.21.15.167:0/780820076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18f419fb20 0x7f18f4199820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 >> 172.21.15.167:0/780820076 conn(0x7f18f40fdfa0 msgr2=0x7f18f4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:05.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 shutdown_connections 2024-05-04T03:01:05.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:05.587+0000 7f18f8839700 1 -- 172.21.15.167:0/780820076 wait complete. 2024-05-04T03:01:05.589 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-05-04T03:01:06.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:06 smithi167 ceph-mon[33614]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:06.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:06.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:06.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:06.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:06 smithi167 ceph-mon[33614]: Deploying daemon mon.b on smithi066 2024-05-04T03:01:06.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:06 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/780820076' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:06.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:06 smithi039 ceph-mon[29715]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:06 smithi039 ceph-mon[29715]: Deploying daemon mon.b on smithi066 2024-05-04T03:01:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:06 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/780820076' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:07.200 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-04T03:01:07.201 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-04T03:01:07.439 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:01:08.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:08 smithi167 ceph-mon[33614]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:08 smithi039 ceph-mon[29715]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:08.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.800+0000 7f137dedd700 1 -- 172.21.15.167:0/4032519042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1378102550 msgr2=0x7f1378102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:08.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.800+0000 7f137dedd700 1 --2- 172.21.15.167:0/4032519042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1378102550 0x7f1378102960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f136800aa20 tx=0x7f136800ad30 comp rx=0 tx=0).stop 2024-05-04T03:01:08.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.800+0000 7f137dedd700 1 -- 172.21.15.167:0/4032519042 shutdown_connections 2024-05-04T03:01:08.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.800+0000 7f137dedd700 1 --2- 172.21.15.167:0/4032519042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1378102550 0x7f1378102960 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:08.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.800+0000 7f137dedd700 1 -- 172.21.15.167:0/4032519042 >> 172.21.15.167:0/4032519042 conn(0x7f1378077e10 msgr2=0x7f1378078270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:08.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.800+0000 7f137dedd700 1 -- 172.21.15.167:0/4032519042 shutdown_connections 2024-05-04T03:01:08.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.800+0000 7f137dedd700 1 -- 172.21.15.167:0/4032519042 wait complete. 2024-05-04T03:01:08.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.801+0000 7f137dedd700 1 Processor -- start 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.801+0000 7f137dedd700 1 -- start start 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f137dedd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 0x7f137819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f137dedd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f137819daa0 0x7f13781977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f137dedd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f137810be50 con 0x7f137819daa0 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f137dedd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f137810bcd0 con 0x7f1378102550 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f13777fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 0x7f137819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f13777fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 0x7f137819d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:57258/0 (socket says 172.21.15.167:57258) 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f13777fe700 1 -- 172.21.15.167:0/4172934712 learned_addr learned my addr 172.21.15.167:0/4172934712 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f1376ffd700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f137819daa0 0x7f13781977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f13777fe700 1 -- 172.21.15.167:0/4172934712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 msgr2=0x7f137819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f13777fe700 1 -- 172.21.15.167:0/4172934712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 msgr2=0x7f137819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-04T03:01:08.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f13777fe700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 0x7f137819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f13777fe700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 0x7f137819d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f1376ffd700 1 -- 172.21.15.167:0/4172934712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 msgr2=0x7f137819d560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f1376ffd700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 0x7f137819d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.802+0000 7f1376ffd700 1 -- 172.21.15.167:0/4172934712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137819e090 con 0x7f137819daa0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.803+0000 7f1376ffd700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f137819daa0 0x7f13781977a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f136c00b4d0 tx=0x7f136c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.803+0000 7f1374ff9700 1 -- 172.21.15.167:0/4172934712 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f136c011630 con 0x7f137819daa0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.803+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1378197da0 con 0x7f137819daa0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.803+0000 7f1374ff9700 1 -- 172.21.15.167:0/4172934712 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f136c011c60 con 0x7f137819daa0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.803+0000 7f1374ff9700 1 -- 172.21.15.167:0/4172934712 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f136c010c90 con 0x7f137819daa0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.803+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1378198330 con 0x7f137819daa0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.804+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1378198640 con 0x7f137819daa0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.804+0000 7f1374ff9700 1 -- 172.21.15.167:0/4172934712 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f136c018020 con 0x7f137819daa0 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.804+0000 7f1374ff9700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f136003e2b0 0x7f13600407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:08.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.804+0000 7f1374ff9700 1 -- 172.21.15.167:0/4172934712 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f136c0522f0 con 0x7f137819daa0 2024-05-04T03:01:08.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.804+0000 7f13777fe700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f136003e2b0 0x7f13600407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:08.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.805+0000 7f13777fe700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f136003e2b0 0x7f13600407c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f136800c6d0 tx=0x7f136801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:08.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:08.810+0000 7f1374ff9700 1 -- 172.21.15.167:0/4172934712 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f136c01f020 con 0x7f137819daa0 2024-05-04T03:01:09.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.101+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1378198d50 con 0x7f137819daa0 2024-05-04T03:01:09.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.102+0000 7f1374ff9700 1 -- 172.21.15.167:0/4172934712 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f136c0187b0 con 0x7f137819daa0 2024-05-04T03:01:09.103 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T03:01:09.103 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","modified":"2024-05-04T03:01:00.116457Z","created":"2024-05-04T02:59:17.356054Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-04T03:01:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f136003e2b0 msgr2=0x7f13600407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f136003e2b0 0x7f13600407c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f136800c6d0 tx=0x7f136801a040 comp rx=0 tx=0).stop 2024-05-04T03:01:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f137819daa0 msgr2=0x7f13781977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f137819daa0 0x7f13781977a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f136c00b4d0 tx=0x7f136c00b890 comp rx=0 tx=0).stop 2024-05-04T03:01:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 shutdown_connections 2024-05-04T03:01:09.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f136003e2b0 0x7f13600407c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:09.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1378102550 0x7f137819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:09.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 --2- 172.21.15.167:0/4172934712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f137819daa0 0x7f13781977a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:09.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 >> 172.21.15.167:0/4172934712 conn(0x7f1378077e10 msgr2=0x7f1378103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:09.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 shutdown_connections 2024-05-04T03:01:09.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:09.104+0000 7f137dedd700 1 -- 172.21.15.167:0/4172934712 wait complete. 2024-05-04T03:01:09.106 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-05-04T03:01:09.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:09 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/4172934712' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:09.466 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:09 smithi066 systemd[1]: Started Ceph mon.b for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:01:09.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:09 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/4172934712' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:10.628 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-04T03:01:10.628 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-04T03:01:10.877 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:01:14.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: mon.a calling monitor election 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: mon.c calling monitor election 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: mon.b calling monitor election 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: monmap epoch 3 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: last_changed 2024-05-04T03:01:09.361504+0000 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: created 2024-05-04T02:59:17.356054+0000 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: min_mon_release 19 (squid) 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: election_strategy: 1 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: fsmap 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:01:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: mgrmap e13: x(active, since 59s) 2024-05-04T03:01:14.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: overall HEALTH_OK 2024-05-04T03:01:14.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:14.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:14.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:01:14.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: mon.a calling monitor election 2024-05-04T03:01:14.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: mon.c calling monitor election 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: mon.b calling monitor election 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: monmap epoch 3 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: last_changed 2024-05-04T03:01:09.361504+0000 2024-05-04T03:01:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: created 2024-05-04T02:59:17.356054+0000 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: min_mon_release 19 (squid) 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: election_strategy: 1 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: fsmap 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: mgrmap e13: x(active, since 59s) 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: overall HEALTH_OK 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: mon.a calling monitor election 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: mon.c calling monitor election 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: mon.b calling monitor election 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: monmap epoch 3 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: last_changed 2024-05-04T03:01:09.361504+0000 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: created 2024-05-04T02:59:17.356054+0000 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: min_mon_release 19 (squid) 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: election_strategy: 1 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: fsmap 2024-05-04T03:01:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:01:14.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: mgrmap e13: x(active, since 59s) 2024-05-04T03:01:14.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: overall HEALTH_OK 2024-05-04T03:01:14.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:14.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.452+0000 7f8240afa700 1 -- 172.21.15.167:0/1365973191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 msgr2=0x7f8228005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:15.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.452+0000 7f8240afa700 1 --2- 172.21.15.167:0/1365973191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 0x7f8228005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f822c009bf0 tx=0x7f822c009f00 comp rx=0 tx=0).stop 2024-05-04T03:01:15.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.453+0000 7f8240afa700 1 -- 172.21.15.167:0/1365973191 shutdown_connections 2024-05-04T03:01:15.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.453+0000 7f8240afa700 1 --2- 172.21.15.167:0/1365973191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 0x7f8228005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:15.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.453+0000 7f8240afa700 1 -- 172.21.15.167:0/1365973191 >> 172.21.15.167:0/1365973191 conn(0x7f823c0797e0 msgr2=0x7f823c079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:15.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.453+0000 7f8240afa700 1 -- 172.21.15.167:0/1365973191 shutdown_connections 2024-05-04T03:01:15.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.453+0000 7f8240afa700 1 -- 172.21.15.167:0/1365973191 wait complete. 2024-05-04T03:01:15.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8240afa700 1 Processor -- start 2024-05-04T03:01:15.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8240afa700 1 -- start start 2024-05-04T03:01:15.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8240afa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 0x7f823c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:15.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8240afa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c19ec00 0x7f823c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:15.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8240afa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823c198e70 0x7f823c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:15.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8240afa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f823c10a9c0 con 0x7f823c102dd0 2024-05-04T03:01:15.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8240afa700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f823c10a840 con 0x7f823c19ec00 2024-05-04T03:01:15.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8240afa700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f823c10ab40 con 0x7f823c198e70 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f823ad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823c198e70 0x7f823c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f823ad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823c198e70 0x7f823c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52936/0 (socket says 172.21.15.167:52936) 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f823ad9d700 1 -- 172.21.15.167:0/3401876134 learned_addr learned my addr 172.21.15.167:0/3401876134 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f8239d9b700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c19ec00 0x7f823c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.454+0000 7f823a59c700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 0x7f823c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.455+0000 7f823ad9d700 1 -- 172.21.15.167:0/3401876134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c19ec00 msgr2=0x7f823c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.455+0000 7f823ad9d700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c19ec00 0x7f823c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.455+0000 7f823ad9d700 1 -- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 msgr2=0x7f823c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.455+0000 7f823ad9d700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 0x7f823c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.455+0000 7f823ad9d700 1 -- 172.21.15.167:0/3401876134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c1a5550 con 0x7f823c198e70 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.455+0000 7f823a59c700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 0x7f823c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:01:15.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.455+0000 7f823ad9d700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823c198e70 0x7f823c199280 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f823400b550 tx=0x7f823400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.457+0000 7f82337fe700 1 -- 172.21.15.167:0/3401876134 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234015020 con 0x7f823c198e70 2024-05-04T03:01:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.457+0000 7f82337fe700 1 -- 172.21.15.167:0/3401876134 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8234011cc0 con 0x7f823c198e70 2024-05-04T03:01:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.457+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c1a57e0 con 0x7f823c198e70 2024-05-04T03:01:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.457+0000 7f82337fe700 1 -- 172.21.15.167:0/3401876134 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234010c90 con 0x7f823c198e70 2024-05-04T03:01:15.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.457+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c1a5cf0 con 0x7f823c198e70 2024-05-04T03:01:15.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.458+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c064e20 con 0x7f823c198e70 2024-05-04T03:01:15.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.458+0000 7f82337fe700 1 -- 172.21.15.167:0/3401876134 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8234018020 con 0x7f823c198e70 2024-05-04T03:01:15.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.458+0000 7f82337fe700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f821c03e9a0 0x7f821c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:15.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.459+0000 7f82337fe700 1 -- 172.21.15.167:0/3401876134 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8234052010 con 0x7f823c198e70 2024-05-04T03:01:15.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.459+0000 7f823a59c700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f821c03e9a0 0x7f821c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:15.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.459+0000 7f823a59c700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f821c03e9a0 0x7f821c040eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f822c009720 tx=0x7f822c003db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.468+0000 7f82337fe700 1 -- 172.21.15.167:0/3401876134 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f823401f020 con 0x7f823c198e70 2024-05-04T03:01:15.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:15.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:15 smithi167 ceph-mon[33614]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:15.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:15.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.704 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:15.704 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:15 smithi039 ceph-mon[29715]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:15.704 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.704 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:15.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.762+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f823c1a5fb0 con 0x7f823c198e70 2024-05-04T03:01:15.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.762+0000 7f82337fe700 1 -- 172.21.15.167:0/3401876134 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f823400cd80 con 0x7f823c198e70 2024-05-04T03:01:15.763 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-04T03:01:15.763 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","modified":"2024-05-04T03:01:09.361504Z","created":"2024-05-04T02:59:17.356054Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-04T03:01:15.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f821c03e9a0 msgr2=0x7f821c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:15.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f821c03e9a0 0x7f821c040eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f822c009720 tx=0x7f822c003db0 comp rx=0 tx=0).stop 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823c198e70 msgr2=0x7f823c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823c198e70 0x7f823c199280 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f823400b550 tx=0x7f823400b860 comp rx=0 tx=0).stop 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 shutdown_connections 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f821c03e9a0 0x7f821c040eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f823c102dd0 0x7f823c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823c198e70 0x7f823c199280 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 --2- 172.21.15.167:0/3401876134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f823c19ec00 0x7f823c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 >> 172.21.15.167:0/3401876134 conn(0x7f823c0797e0 msgr2=0x7f823c105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.765+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 shutdown_connections 2024-05-04T03:01:15.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:15.766+0000 7f8240afa700 1 -- 172.21.15.167:0/3401876134 wait complete. 2024-05-04T03:01:15.770 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-05-04T03:01:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:15 smithi066 ceph-mon[33264]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:01:16.448 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-04T03:01:16.449 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-04T03:01:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/3401876134' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: Updating smithi167:/etc/ceph/ceph.conf 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: Updating smithi066:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: Updating smithi167:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: Updating smithi039:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.699 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:01:16.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/3401876134' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:16.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: Updating smithi167:/etc/ceph/ceph.conf 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: Updating smithi066:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: Updating smithi167:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: Updating smithi039:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/3401876134' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:01:16.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:16.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: Updating smithi039:/etc/ceph/ceph.conf 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: Updating smithi066:/etc/ceph/ceph.conf 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: Updating smithi167:/etc/ceph/ceph.conf 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: Updating smithi066:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: Updating smithi167:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: Updating smithi039:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:16.827 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: Reconfiguring mon.a (unknown last config time)... 2024-05-04T03:01:17.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:17.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:17.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:17.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: Reconfiguring daemon mon.a on smithi039 2024-05-04T03:01:17.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:17 smithi039 ceph-mon[29715]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:17.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: Reconfiguring mon.a (unknown last config time)... 2024-05-04T03:01:17.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:17.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:17.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:17.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: Reconfiguring daemon mon.a on smithi039 2024-05-04T03:01:17.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:17 smithi066 ceph-mon[33264]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:17.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: Reconfiguring mon.a (unknown last config time)... 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: Reconfiguring daemon mon.a on smithi039 2024-05-04T03:01:17.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:17 smithi167 ceph-mon[33614]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.708+0000 7fe7094e8700 1 -- 172.21.15.39:0/4236397270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704103330 msgr2=0x7fe704103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.708+0000 7fe7094e8700 1 --2- 172.21.15.39:0/4236397270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704103330 0x7fe704103740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009910 tx=0x7fe6ec009c20 comp rx=0 tx=0).stop 2024-05-04T03:01:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.708+0000 7fe7094e8700 1 -- 172.21.15.39:0/4236397270 shutdown_connections 2024-05-04T03:01:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.708+0000 7fe7094e8700 1 --2- 172.21.15.39:0/4236397270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704103330 0x7fe704103740 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.708+0000 7fe7094e8700 1 -- 172.21.15.39:0/4236397270 >> 172.21.15.39:0/4236397270 conn(0x7fe7040fde30 msgr2=0x7fe704100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.708+0000 7fe7094e8700 1 -- 172.21.15.39:0/4236397270 shutdown_connections 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.708+0000 7fe7094e8700 1 -- 172.21.15.39:0/4236397270 wait complete. 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.709+0000 7fe7094e8700 1 Processor -- start 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.709+0000 7fe7094e8700 1 -- start start 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7094e8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe704103330 0x7fe70419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7094e8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70419fd80 0x7fe704199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7094e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704199fa0 0x7fe70419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7094e8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe70410cea0 con 0x7fe704199fa0 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7094e8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe70410cd20 con 0x7fe704103330 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7094e8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe70410d020 con 0x7fe70419fd80 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704199fa0 0x7fe70419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704199fa0 0x7fe70419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53508/0 (socket says 172.21.15.39:53508) 2024-05-04T03:01:18.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 -- 172.21.15.39:0/2409665380 learned_addr learned my addr 172.21.15.39:0/2409665380 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:01:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe702ffd700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe704103330 0x7fe70419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7027fc700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70419fd80 0x7fe704199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 -- 172.21.15.39:0/2409665380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70419fd80 msgr2=0x7fe704199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70419fd80 0x7fe704199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 -- 172.21.15.39:0/2409665380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe704103330 msgr2=0x7fe70419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe704103330 0x7fe70419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 -- 172.21.15.39:0/2409665380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7041a66d0 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7037fe700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704199fa0 0x7fe70419a3b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00ba50 tx=0x7fe6fc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe6fbfff700 1 -- 172.21.15.39:0/2409665380 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc019070 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe6fbfff700 1 -- 172.21.15.39:0/2409665380 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6fc014440 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7041a6960 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.710+0000 7fe6fbfff700 1 -- 172.21.15.39:0/2409665380 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0075b0 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.711+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7041a6e70 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.712+0000 7fe6fbfff700 1 -- 172.21.15.39:0/2409665380 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe6fc01c070 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.712+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe704064e20 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.712+0000 7fe6fbfff700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6f003e9a0 0x7fe6f0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.712+0000 7fe6fbfff700 1 -- 172.21.15.39:0/2409665380 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe6fc051de0 con 0x7fe704199fa0 2024-05-04T03:01:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.712+0000 7fe702ffd700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6f003e9a0 0x7fe6f0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.712+0000 7fe702ffd700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6f003e9a0 0x7fe6f0040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00b5c0 tx=0x7fe6ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.717+0000 7fe6fbfff700 1 -- 172.21.15.39:0/2409665380 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6fc022170 con 0x7fe704199fa0 2024-05-04T03:01:18.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.934+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe7041a7100 con 0x7fe704199fa0 2024-05-04T03:01:18.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.934+0000 7fe6fbfff700 1 -- 172.21.15.39:0/2409665380 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fe6fc012c40 con 0x7fe704199fa0 2024-05-04T03:01:18.935 INFO:teuthology.orchestra.run.smithi039.stdout:# minimal ceph.conf for 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:18.935 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-05-04T03:01:18.935 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:01:18.935 INFO:teuthology.orchestra.run.smithi039.stdout: mon_host = [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6f003e9a0 msgr2=0x7fe6f0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6f003e9a0 0x7fe6f0040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00b5c0 tx=0x7fe6ec005fb0 comp rx=0 tx=0).stop 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704199fa0 msgr2=0x7fe70419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704199fa0 0x7fe70419a3b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00ba50 tx=0x7fe6fc00be10 comp rx=0 tx=0).stop 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 shutdown_connections 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6f003e9a0 0x7fe6f0040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe704199fa0 0x7fe70419a3b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe704103330 0x7fe70419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 --2- 172.21.15.39:0/2409665380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe70419fd80 0x7fe704199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 >> 172.21.15.39:0/2409665380 conn(0x7fe7040fde30 msgr2=0x7fe704102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:18.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.936+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 shutdown_connections 2024-05-04T03:01:18.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:18.937+0000 7fe7094e8700 1 -- 172.21.15.39:0/2409665380 wait complete. 2024-05-04T03:01:19.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:18 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2409665380' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:19.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:18 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2409665380' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:19.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:18 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2409665380' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:19.685 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-04T03:01:19.685 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:01:19.685 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.conf 2024-05-04T03:01:19.713 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:01:19.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:01:19.777 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T03:01:19.778 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2024-05-04T03:01:19.804 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T03:01:19.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:01:19.868 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T03:01:19.869 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-05-04T03:01:19.900 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T03:01:19.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:01:19.968 INFO:tasks.cephadm:Adding mgr.x on smithi039 2024-05-04T03:01:19.968 INFO:tasks.cephadm:Adding mgr.y on smithi066 2024-05-04T03:01:19.968 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-05-04T03:01:19.968 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi039=x;smithi066=y;smithi167=z' 2024-05-04T03:01:20.237 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:01:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:20 smithi066 ceph-mon[33264]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:20 smithi066 ceph-mon[33264]: Reconfiguring mon.b (monmap changed)... 2024-05-04T03:01:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:20.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:20.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:20.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:20 smithi066 ceph-mon[33264]: Reconfiguring daemon mon.b on smithi066 2024-05-04T03:01:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:20 smithi167 ceph-mon[33614]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:20 smithi167 ceph-mon[33614]: Reconfiguring mon.b (monmap changed)... 2024-05-04T03:01:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:20.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:20.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:20 smithi167 ceph-mon[33614]: Reconfiguring daemon mon.b on smithi066 2024-05-04T03:01:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:20 smithi039 ceph-mon[29715]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:20 smithi039 ceph-mon[29715]: Reconfiguring mon.b (monmap changed)... 2024-05-04T03:01:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:20.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:20.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:20.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:20 smithi039 ceph-mon[29715]: Reconfiguring daemon mon.b on smithi066 2024-05-04T03:01:21.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.750+0000 7fd5820c0700 1 -- 172.21.15.167:0/762609645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c102bf0 msgr2=0x7fd57c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:21.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.750+0000 7fd5820c0700 1 --2- 172.21.15.167:0/762609645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c102bf0 0x7fd57c103000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009910 tx=0x7fd56c009c20 comp rx=0 tx=0).stop 2024-05-04T03:01:21.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.751+0000 7fd5820c0700 1 -- 172.21.15.167:0/762609645 shutdown_connections 2024-05-04T03:01:21.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.751+0000 7fd5820c0700 1 --2- 172.21.15.167:0/762609645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c102bf0 0x7fd57c103000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:21.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.751+0000 7fd5820c0700 1 -- 172.21.15.167:0/762609645 >> 172.21.15.167:0/762609645 conn(0x7fd57c0fe250 msgr2=0x7fd57c1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:21.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.751+0000 7fd5820c0700 1 -- 172.21.15.167:0/762609645 shutdown_connections 2024-05-04T03:01:21.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.751+0000 7fd5820c0700 1 -- 172.21.15.167:0/762609645 wait complete. 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.751+0000 7fd5820c0700 1 Processor -- start 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.751+0000 7fd5820c0700 1 -- start start 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd5820c0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57c102bf0 0x7fd57c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd5820c0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c199e10 0x7fd57c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd5820c0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57c19f2e0 0x7fd57c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd5820c0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c108a20 con 0x7fd57c199e10 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd5820c0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c1088a0 con 0x7fd57c102bf0 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd5820c0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c108ba0 con 0x7fd57c19f2e0 2024-05-04T03:01:21.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57c19f2e0 0x7fd57c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57c19f2e0 0x7fd57c1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52952/0 (socket says 172.21.15.167:52952) 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 -- 172.21.15.167:0/4215965012 learned_addr learned my addr 172.21.15.167:0/4215965012 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57affd700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c199e10 0x7fd57c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57b7fe700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57c102bf0 0x7fd57c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 -- 172.21.15.167:0/4215965012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57c102bf0 msgr2=0x7fd57c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57c102bf0 0x7fd57c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 -- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c199e10 msgr2=0x7fd57c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c199e10 0x7fd57c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:21.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 -- 172.21.15.167:0/4215965012 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c1a1c70 con 0x7fd57c19f2e0 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57b7fe700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57c102bf0 0x7fd57c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57affd700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c199e10 0x7fd57c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.752+0000 7fd57bfff700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57c19f2e0 0x7fd57c1a1700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd57400ba50 tx=0x7fd57400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.753+0000 7fd578ff9700 1 -- 172.21.15.167:0/4215965012 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574019070 con 0x7fd57c19f2e0 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.753+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c1a1f60 con 0x7fd57c19f2e0 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.753+0000 7fd578ff9700 1 -- 172.21.15.167:0/4215965012 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd574014440 con 0x7fd57c19f2e0 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.753+0000 7fd578ff9700 1 -- 172.21.15.167:0/4215965012 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5740075b0 con 0x7fd57c19f2e0 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.753+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c1a2440 con 0x7fd57c19f2e0 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.754+0000 7fd578ff9700 1 -- 172.21.15.167:0/4215965012 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd57401c070 con 0x7fd57c19f2e0 2024-05-04T03:01:21.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.753+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57c064e20 con 0x7fd57c19f2e0 2024-05-04T03:01:21.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.754+0000 7fd578ff9700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56803e9a0 0x7fd568040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:21.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.754+0000 7fd578ff9700 1 -- 172.21.15.167:0/4215965012 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd574051e60 con 0x7fd57c19f2e0 2024-05-04T03:01:21.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.754+0000 7fd57b7fe700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56803e9a0 0x7fd568040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:21.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.755+0000 7fd57b7fe700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56803e9a0 0x7fd568040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00b5c0 tx=0x7fd56c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:21.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.759+0000 7fd578ff9700 1 -- 172.21.15.167:0/4215965012 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd574022170 con 0x7fd57c19f2e0 2024-05-04T03:01:21.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.984+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi066=y;smithi167=z", "target": ["mon-mgr", ""]}) -- 0x7fd57c063c90 con 0x7fd56803e9a0 2024-05-04T03:01:21.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.992+0000 7fd578ff9700 1 -- 172.21.15.167:0/4215965012 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd57c063c90 con 0x7fd56803e9a0 2024-05-04T03:01:21.991 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56803e9a0 msgr2=0x7fd568040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56803e9a0 0x7fd568040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00b5c0 tx=0x7fd56c005fb0 comp rx=0 tx=0).stop 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57c19f2e0 msgr2=0x7fd57c1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57c19f2e0 0x7fd57c1a1700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd57400ba50 tx=0x7fd57400be10 comp rx=0 tx=0).stop 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 shutdown_connections 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56803e9a0 0x7fd568040eb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57c19f2e0 0x7fd57c1a1700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd57c102bf0 0x7fd57c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 --2- 172.21.15.167:0/4215965012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd57c199e10 0x7fd57c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 >> 172.21.15.167:0/4215965012 conn(0x7fd57c0fe250 msgr2=0x7fd57c06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:21.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 shutdown_connections 2024-05-04T03:01:21.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:01:21.994+0000 7fd5820c0700 1 -- 172.21.15.167:0/4215965012 wait complete. 2024-05-04T03:01:22.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:22.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:22.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:21 smithi167 ceph-mon[33614]: Reconfiguring mon.c (monmap changed)... 2024-05-04T03:01:22.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:22.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:21 smithi167 ceph-mon[33614]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:22.006 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:22.006 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:22.006 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:21 smithi167 ceph-mon[33614]: Reconfiguring daemon mon.c on smithi167 2024-05-04T03:01:22.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:22.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:22.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:21 smithi039 ceph-mon[29715]: Reconfiguring mon.c (monmap changed)... 2024-05-04T03:01:22.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:22.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:21 smithi039 ceph-mon[29715]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:22.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:22.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:22.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:21 smithi039 ceph-mon[29715]: Reconfiguring daemon mon.c on smithi167 2024-05-04T03:01:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:21 smithi066 ceph-mon[33264]: Reconfiguring mon.c (monmap changed)... 2024-05-04T03:01:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:01:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:21 smithi066 ceph-mon[33264]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:01:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:21 smithi066 ceph-mon[33264]: Reconfiguring daemon mon.c on smithi167 2024-05-04T03:01:23.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:22 smithi167 ceph-mon[33614]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi066=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:23.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:22 smithi167 ceph-mon[33614]: Saving service mgr spec with placement smithi039=x;smithi066=y;smithi167=z;count:3 2024-05-04T03:01:23.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:22 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:22 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:22 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:22 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:23.260 DEBUG:teuthology.orchestra.run.smithi066:mgr.y> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.y.service 2024-05-04T03:01:23.263 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.z.service 2024-05-04T03:01:23.265 INFO:tasks.cephadm:Deploying OSDs... 2024-05-04T03:01:23.265 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:01:23.266 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-05-04T03:01:23.282 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-04T03:01:23.282 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-05-04T03:01:23.287 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:22 smithi066 ceph-mon[33264]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi066=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:23.287 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:22 smithi066 ceph-mon[33264]: Saving service mgr spec with placement smithi039=x;smithi066=y;smithi167=z;count:3 2024-05-04T03:01:23.287 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:22 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.287 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:22 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.287 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:22 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.287 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:22 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:23.337 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-04T03:01:23.337 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:23.338 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-04T03:01:23.338 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:23.338 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:23.338 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-04 03:00:24.798522362 +0000 2024-05-04T03:01:23.338 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-04 02:55:48.773306747 +0000 2024-05-04T03:01:23.338 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-04 02:55:48.773306747 +0000 2024-05-04T03:01:23.338 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-05-04 02:55:48.773306747 +0000 2024-05-04T03:01:23.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-04T03:01:23.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:22 smithi039 ceph-mon[29715]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi066=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:23.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:22 smithi039 ceph-mon[29715]: Saving service mgr spec with placement smithi039=x;smithi066=y;smithi167=z;count:3 2024-05-04T03:01:23.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:22 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:22 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:22 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:23.400 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:22 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:23.402 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-04T03:01:23.402 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-04T03:01:23.402 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.00020347 s, 2.5 MB/s 2024-05-04T03:01:23.404 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-04T03:01:23.461 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-04 03:00:24.831521764 +0000 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-04 02:55:49.127299908 +0000 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-04 02:55:49.127299908 +0000 2024-05-04T03:01:23.517 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-05-04 02:55:49.127299908 +0000 2024-05-04T03:01:23.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-04T03:01:23.581 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-04T03:01:23.581 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-04T03:01:23.581 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000312198 s, 1.6 MB/s 2024-05-04T03:01:23.582 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-04T03:01:23.639 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-05-04T03:01:23.694 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-04T03:01:23.694 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:23.694 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-04T03:01:23.694 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:23.694 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:23.694 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-04 03:00:24.863521185 +0000 2024-05-04T03:01:23.695 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-04 02:55:49.470293282 +0000 2024-05-04T03:01:23.696 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-04 02:55:49.470293282 +0000 2024-05-04T03:01:23.696 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-05-04 02:55:49.470293282 +0000 2024-05-04T03:01:23.696 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-04T03:01:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-04T03:01:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-04T03:01:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000182935 s, 2.8 MB/s 2024-05-04T03:01:23.758 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-04T03:01:23.814 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-05-04T03:01:23.869 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-04T03:01:23.870 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:23.870 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-04T03:01:23.870 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:23.870 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:23.870 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-05-04 03:00:24.897520569 +0000 2024-05-04T03:01:23.871 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-05-04 02:55:49.816286598 +0000 2024-05-04T03:01:23.871 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-05-04 02:55:49.816286598 +0000 2024-05-04T03:01:23.871 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-05-04 02:55:49.816286598 +0000 2024-05-04T03:01:23.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-04T03:01:23.933 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-05-04T03:01:23.934 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-05-04T03:01:23.934 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000368962 s, 1.4 MB/s 2024-05-04T03:01:23.935 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-04T03:01:23.991 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-05-04T03:01:23.991 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-05-04T03:01:24.011 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-04T03:01:24.011 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-05-04T03:01:24.069 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-04T03:01:24.069 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:24.070 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-04T03:01:24.070 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:24.070 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:24.070 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-05-04 03:01:08.770503252 +0000 2024-05-04T03:01:24.070 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-05-04 02:55:46.989075336 +0000 2024-05-04T03:01:24.070 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-05-04 02:55:46.989075336 +0000 2024-05-04T03:01:24.070 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-05-04 02:55:46.989075336 +0000 2024-05-04T03:01:24.071 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-04T03:01:24.138 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-05-04T03:01:24.138 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-05-04T03:01:24.138 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000269421 s, 1.9 MB/s 2024-05-04T03:01:24.140 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-04T03:01:24.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:23 smithi039 ceph-mon[29715]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:24.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:24.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:24.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:24.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:24.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:01:24.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:24.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:24.199 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-05-04T03:01:24.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:23 smithi167 ceph-mon[33614]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:24.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:24.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:24.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:24.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:24.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:01:24.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:24.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:24.257 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-04T03:01:24.257 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:24.257 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-04T03:01:24.257 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:24.258 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:24.258 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-05-04 03:01:08.771503246 +0000 2024-05-04T03:01:24.258 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-05-04 02:55:47.326073630 +0000 2024-05-04T03:01:24.258 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-05-04 02:55:47.326073630 +0000 2024-05-04T03:01:24.258 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-05-04 02:55:47.326073630 +0000 2024-05-04T03:01:24.259 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-04T03:01:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:23 smithi066 ceph-mon[33264]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:24.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:24.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:01:24.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:24.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-05-04T03:01:24.323 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-05-04T03:01:24.324 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000223041 s, 2.3 MB/s 2024-05-04T03:01:24.325 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-04T03:01:24.383 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-05-04T03:01:24.440 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-04T03:01:24.440 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:24.440 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-04T03:01:24.440 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:24.440 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:24.441 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-05-04 03:01:08.771503246 +0000 2024-05-04T03:01:24.441 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-05-04 02:55:47.670071890 +0000 2024-05-04T03:01:24.441 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-05-04 02:55:47.670071890 +0000 2024-05-04T03:01:24.441 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-05-04 02:55:47.670071890 +0000 2024-05-04T03:01:24.441 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-04T03:01:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-05-04T03:01:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-05-04T03:01:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000416689 s, 1.2 MB/s 2024-05-04T03:01:24.507 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-04T03:01:24.565 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-05-04 03:01:08.771503246 +0000 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-05-04 02:55:48.016070140 +0000 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-05-04 02:55:48.016070140 +0000 2024-05-04T03:01:24.621 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-05-04 02:55:48.016070140 +0000 2024-05-04T03:01:24.621 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-04T03:01:24.684 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-05-04T03:01:24.684 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-05-04T03:01:24.684 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000192775 s, 2.7 MB/s 2024-05-04T03:01:24.685 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-04T03:01:24.742 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-04T03:01:24.742 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-05-04T03:01:24.759 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-04T03:01:24.759 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-04 03:00:59.117412718 +0000 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-04 02:55:44.997131584 +0000 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-04 02:55:44.997131584 +0000 2024-05-04T03:01:24.815 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-04 02:55:44.997131584 +0000 2024-05-04T03:01:24.816 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-04T03:01:24.880 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-04T03:01:24.880 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-04T03:01:24.880 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000218828 s, 2.3 MB/s 2024-05-04T03:01:24.881 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-04T03:01:24.938 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-05-04T03:01:24.998 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-04T03:01:24.998 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:24.998 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-04T03:01:24.998 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:24.998 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:24.998 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-04 03:00:59.117412718 +0000 2024-05-04T03:01:24.999 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-04 02:55:45.347124825 +0000 2024-05-04T03:01:24.999 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-04 02:55:45.347124825 +0000 2024-05-04T03:01:24.999 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-04 02:55:45.347124825 +0000 2024-05-04T03:01:24.999 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-04T03:01:25.062 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-04T03:01:25.062 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-04T03:01:25.062 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.00020351 s, 2.5 MB/s 2024-05-04T03:01:25.063 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-04T03:01:25.122 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-05-04T03:01:25.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:24 smithi167 ceph-mon[33614]: Deploying daemon mgr.z on smithi167 2024-05-04T03:01:25.177 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-04T03:01:25.177 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:25.177 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-04T03:01:25.177 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:25.177 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:25.178 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-04 03:00:59.118412700 +0000 2024-05-04T03:01:25.178 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-04 02:55:45.687118257 +0000 2024-05-04T03:01:25.178 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-04 02:55:45.687118257 +0000 2024-05-04T03:01:25.178 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-04 02:55:45.687118257 +0000 2024-05-04T03:01:25.178 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-04T03:01:25.243 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-04T03:01:25.243 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-04T03:01:25.243 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000183235 s, 2.8 MB/s 2024-05-04T03:01:25.245 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-04T03:01:25.308 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-05-04T03:01:25.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:24 smithi066 ceph-mon[33264]: Deploying daemon mgr.z on smithi167 2024-05-04T03:01:25.363 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-04T03:01:25.363 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:01:25.363 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-04T03:01:25.363 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:01:25.364 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:01:25.364 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-04 03:01:25.178943748 +0000 2024-05-04T03:01:25.364 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-04 02:55:46.029111651 +0000 2024-05-04T03:01:25.364 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-04 02:55:46.029111651 +0000 2024-05-04T03:01:25.364 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-04 02:55:46.029111651 +0000 2024-05-04T03:01:25.364 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-04T03:01:25.430 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-04T03:01:25.430 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-04T03:01:25.430 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000204741 s, 2.5 MB/s 2024-05-04T03:01:25.432 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-04T03:01:25.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:24 smithi039 ceph-mon[29715]: Deploying daemon mgr.z on smithi167 2024-05-04T03:01:25.489 INFO:tasks.cephadm:Deploying osd.0 on smithi039 with /dev/vg_nvme/lv_4... 2024-05-04T03:01:25.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-04T03:01:25.742 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:01:26.029 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:25 smithi167 ceph-mon[33614]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:26.119 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:25 smithi039 ceph-mon[29715]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:26.316 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:26.213+0000 7f14b48d2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:01:26.316 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:26.315+0000 7f14b48d2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T03:01:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:25 smithi066 ceph-mon[33264]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:26.926 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:26.622+0000 7f14b48d2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T03:01:26.926 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:26.706+0000 7f14b48d2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T03:01:27.082 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:27.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:01:27.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:27.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:27.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:27 smithi039 ceph-mon[29715]: Deploying daemon mgr.y on smithi066 2024-05-04T03:01:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:01:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:27.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:27.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:27 smithi066 ceph-mon[33264]: Deploying daemon mgr.y on smithi066 2024-05-04T03:01:27.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:27.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:27.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:01:27.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:27.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:27.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:27 smithi167 ceph-mon[33614]: Deploying daemon mgr.y on smithi066 2024-05-04T03:01:27.685 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-04T03:01:27.685 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-04T03:01:27.686 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-04T03:01:27.686 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-04T03:01:27.686 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00707828 s, 1.5 GB/s 2024-05-04T03:01:27.686 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-04T03:01:28.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:28 smithi167 ceph-mon[33614]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:28.176 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:27 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:27.916+0000 7f14b48d2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T03:01:28.177 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:28.087+0000 7f14b48d2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T03:01:28.263 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:28 smithi039 ceph-mon[29715]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:28.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_4 2024-05-04T03:01:28.315 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:28 smithi066 ceph-mon[33264]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:28.315 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 systemd[1]: Starting Ceph mgr.y for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:01:28.507 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:01:28.546 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:28.259+0000 7f14b48d2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T03:01:28.546 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:28.545+0000 7f14b48d2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T03:01:28.567 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 podman[34644]: 2024-05-04 03:01:28.21860037 +0000 UTC m=+0.018377527 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:01:28.567 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 podman[34644]: 2024-05-04 03:01:28.421540743 +0000 UTC m=+0.221317931 container create d3214c33ee95aa15ffa415b2abd652edd04a4ae629bcfe2a0aceb9e9c5ce4046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-04T03:01:28.847 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:28.637+0000 7f14b48d2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T03:01:28.848 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:28.846+0000 7f14b48d2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T03:01:28.963 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 podman[34644]: 2024-05-04 03:01:28.599253669 +0000 UTC m=+0.399030812 container init d3214c33ee95aa15ffa415b2abd652edd04a4ae629bcfe2a0aceb9e9c5ce4046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-04T03:01:28.963 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 podman[34644]: 2024-05-04 03:01:28.601998597 +0000 UTC m=+0.401775751 container start d3214c33ee95aa15ffa415b2abd652edd04a4ae629bcfe2a0aceb9e9c5ce4046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-04T03:01:28.963 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 ceph-mgr[34664]: -- 172.21.15.66:0/3642783290 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f55fb0c5a0 con 0x55f564896900 2024-05-04T03:01:28.963 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 bash[34644]: d3214c33ee95aa15ffa415b2abd652edd04a4ae629bcfe2a0aceb9e9c5ce4046 2024-05-04T03:01:28.963 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 systemd[1]: Started Ceph mgr.y for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:01:28.964 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:28.768+0000 7f01fd69f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T03:01:28.964 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:28.961+0000 7f01fd69f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:01:29.176 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:29.042+0000 7f14b48d2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:01:29.317 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:29.074+0000 7f01fd69f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T03:01:29.752 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:29.405+0000 7f01fd69f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T03:01:29.752 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:29.501+0000 7f01fd69f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T03:01:29.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:29.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:29.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:29.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:29.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:29 smithi039 ceph-mon[29715]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:29.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:30.022 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:30.022 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:30.022 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:30.022 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:30.022 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:29 smithi066 ceph-mon[33264]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:30.022 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:30.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 -- 172.21.15.39:0/3841580615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728103250 msgr2=0x7f7728103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:30.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 --2- 172.21.15.39:0/3841580615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728103250 0x7f7728103660 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7718009990 tx=0x7f7718009ca0 comp rx=0 tx=0).stop 2024-05-04T03:01:30.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 -- 172.21.15.39:0/3841580615 shutdown_connections 2024-05-04T03:01:30.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 --2- 172.21.15.39:0/3841580615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728103250 0x7f7728103660 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:30.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 --2- 172.21.15.39:0/3841580615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7728104d20 0x7f772810b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:30.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 --2- 172.21.15.39:0/3841580615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7728104380 0x7f7728104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:30.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 -- 172.21.15.39:0/3841580615 >> 172.21.15.39:0/3841580615 conn(0x7f77280fe890 msgr2=0x7f7728100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:30.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 -- 172.21.15.39:0/3841580615 shutdown_connections 2024-05-04T03:01:30.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.106+0000 7f772e01c700 1 -- 172.21.15.39:0/3841580615 wait complete. 2024-05-04T03:01:30.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f772e01c700 1 Processor -- start 2024-05-04T03:01:30.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f772e01c700 1 -- start start 2024-05-04T03:01:30.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f772e01c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7728103250 0x7f7728076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:30.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f772e01c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7728104380 0x7f772819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f772e01c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728104d20 0x7f77281a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f772e01c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f772810eb60 con 0x7f7728104d20 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f772e01c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f772810e9e0 con 0x7f7728103250 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f772e01c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f772810ece0 con 0x7f7728104380 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f77277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7728103250 0x7f7728076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f7727fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728104d20 0x7f77281a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f7726ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7728104380 0x7f772819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f77277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7728103250 0x7f7728076450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:40398/0 (socket says 172.21.15.39:40398) 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.107+0000 7f77277fe700 1 -- 172.21.15.39:0/2198186505 learned_addr learned my addr 172.21.15.39:0/2198186505 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f77277fe700 1 -- 172.21.15.39:0/2198186505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7728104380 msgr2=0x7f772819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f77277fe700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7728104380 0x7f772819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f77277fe700 1 -- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728104d20 msgr2=0x7f77281a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:30.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f77277fe700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728104d20 0x7f77281a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f77277fe700 1 -- 172.21.15.39:0/2198186505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77281a2420 con 0x7f7728103250 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f7726ffd700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7728104380 0x7f772819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f7727fff700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728104d20 0x7f77281a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f77277fe700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7728103250 0x7f7728076450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77180098e0 tx=0x7f771800b9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718026070 con 0x7f7728103250 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7718003d20 con 0x7f7728103250 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771800fbf0 con 0x7f7728103250 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77281a26b0 con 0x7f7728103250 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.108+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77281a2b40 con 0x7f7728103250 2024-05-04T03:01:30.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.109+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7718029070 con 0x7f7728103250 2024-05-04T03:01:30.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.110+0000 7f7724ff9700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f771003e750 0x7f7710040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:30.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.110+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7718053680 con 0x7f7728103250 2024-05-04T03:01:30.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.110+0000 7f7726ffd700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f771003e750 0x7f7710040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:30.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.110+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77280512c0 con 0x7f7728103250 2024-05-04T03:01:30.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.110+0000 7f7726ffd700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f771003e750 0x7f7710040c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7728075e00 tx=0x7f771c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:30.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.116+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7718013070 con 0x7f7728103250 2024-05-04T03:01:30.176 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:29.684+0000 7f14b48d2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T03:01:30.176 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:29.919+0000 7f14b48d2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T03:01:30.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:30.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:30.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:30.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:30.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:29 smithi167 ceph-mon[33614]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:30.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:30.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:30.342+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f772802e9f0 con 0x7f771003e750 2024-05-04T03:01:30.676 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:30.358+0000 7f14b48d2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T03:01:30.676 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:30.460+0000 7f14b48d2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T03:01:31.067 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:30 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:30.796+0000 7f01fd69f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T03:01:31.067 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:30 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:30.971+0000 7f01fd69f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T03:01:31.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:31.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:31.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.094 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.094 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:31.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.176 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:30.834+0000 7f14b48d2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T03:01:31.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:01:31.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:31.445 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:31.147+0000 7f01fd69f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T03:01:31.445 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:31.444+0000 7f01fd69f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T03:01:31.526 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:31.191+0000 7f14b48d2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T03:01:31.526 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:31.283+0000 7f14b48d2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T03:01:31.749 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:31.539+0000 7f01fd69f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T03:01:31.749 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:31.748+0000 7f01fd69f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T03:01:31.926 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:31.525+0000 7f14b48d2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T03:01:31.926 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:31.657+0000 7f14b48d2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T03:01:32.067 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:31.945+0000 7f01fd69f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:01:32.399 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:31.972+0000 7f14b48d2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T03:01:32.399 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:32.087+0000 7f14b48d2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T03:01:32.399 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:32.176+0000 7f14b48d2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T03:01:32.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:32 smithi167 ceph-mon[33614]: Reconfiguring mgr.x (unknown last config time)... 2024-05-04T03:01:32.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:32 smithi167 ceph-mon[33614]: Reconfiguring daemon mgr.x on smithi039 2024-05-04T03:01:32.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:32 smithi167 ceph-mon[33614]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:32.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:32 smithi039 ceph-mon[29715]: Reconfiguring mgr.x (unknown last config time)... 2024-05-04T03:01:32.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:32 smithi039 ceph-mon[29715]: Reconfiguring daemon mgr.x on smithi039 2024-05-04T03:01:32.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:32 smithi039 ceph-mon[29715]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:32.796 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:32 smithi066 ceph-mon[33264]: Reconfiguring mgr.x (unknown last config time)... 2024-05-04T03:01:32.796 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:32 smithi066 ceph-mon[33264]: Reconfiguring daemon mgr.x on smithi039 2024-05-04T03:01:32.796 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:32 smithi066 ceph-mon[33264]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:32.796 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:32.569+0000 7f01fd69f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T03:01:32.796 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:32.796+0000 7f01fd69f200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T03:01:33.176 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:32.776+0000 7f14b48d2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T03:01:33.176 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:01:33 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z[35649]: 2024-05-04T03:01:32.998+0000 7f14b48d2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T03:01:33.567 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:33.232+0000 7f01fd69f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T03:01:33.567 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:33.331+0000 7f01fd69f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T03:01:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:33.845+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f7718009d10 con 0x7f7728103250 2024-05-04T03:01:34.067 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:33.712+0000 7f01fd69f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T03:01:34.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:34.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:34.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:34.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: Standby manager daemon z started 2024-05-04T03:01:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-04T03:01:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:01:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-04T03:01:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:33 smithi066 ceph-mon[33264]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:01:34.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:34.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:34.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:34.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:34.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: Standby manager daemon z started 2024-05-04T03:01:34.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-04T03:01:34.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:01:34.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-04T03:01:34.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:33 smithi167 ceph-mon[33614]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:01:34.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: Standby manager daemon z started 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-04T03:01:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:33 smithi039 ceph-mon[29715]: from='mgr.? 172.21.15.167:0/2383140542' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:01:34.428 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:34.084+0000 7f01fd69f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T03:01:34.428 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:34.178+0000 7f01fd69f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T03:01:34.817 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:34.428+0000 7f01fd69f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T03:01:34.817 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:34.565+0000 7f01fd69f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T03:01:35.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:34 smithi167 ceph-mon[33614]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-04T03:01:35.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:34 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-04T03:01:35.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:34 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1314769153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9940446-c2ae-4eb5-b6fe-313baf8466d2"}]: dispatch 2024-05-04T03:01:35.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:34 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1314769153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9940446-c2ae-4eb5-b6fe-313baf8466d2"}]': finished 2024-05-04T03:01:35.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:34 smithi167 ceph-mon[33614]: osdmap e5: 1 total, 0 up, 1 in 2024-05-04T03:01:35.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:34 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:35.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:34 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1446127536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:01:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:34 smithi039 ceph-mon[29715]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-04T03:01:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:34 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-04T03:01:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:34 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1314769153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9940446-c2ae-4eb5-b6fe-313baf8466d2"}]: dispatch 2024-05-04T03:01:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:34 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1314769153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9940446-c2ae-4eb5-b6fe-313baf8466d2"}]': finished 2024-05-04T03:01:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:34 smithi039 ceph-mon[29715]: osdmap e5: 1 total, 0 up, 1 in 2024-05-04T03:01:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:34 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:35.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:34 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1446127536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:01:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:34 smithi066 ceph-mon[33264]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-04T03:01:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:34 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-04T03:01:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:34 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1314769153' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9940446-c2ae-4eb5-b6fe-313baf8466d2"}]: dispatch 2024-05-04T03:01:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:34 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1314769153' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9940446-c2ae-4eb5-b6fe-313baf8466d2"}]': finished 2024-05-04T03:01:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:34 smithi066 ceph-mon[33264]: osdmap e5: 1 total, 0 up, 1 in 2024-05-04T03:01:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:34 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:34 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1446127536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:01:35.318 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:34.879+0000 7f01fd69f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T03:01:35.318 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:34.998+0000 7f01fd69f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T03:01:35.318 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:35.091+0000 7f01fd69f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T03:01:36.044 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:35 smithi066 ceph-mon[33264]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:36.045 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:35.713+0000 7f01fd69f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T03:01:36.045 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:01:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y[34660]: 2024-05-04T03:01:35.936+0000 7f01fd69f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T03:01:36.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:35 smithi167 ceph-mon[33614]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:35 smithi039 ceph-mon[29715]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:36.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:36.861+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mon.2 v2:172.21.15.66:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f7718027bd0 con 0x7f7728103250 2024-05-04T03:01:37.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:36 smithi066 ceph-mon[33264]: Standby manager daemon y started 2024-05-04T03:01:37.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:36 smithi066 ceph-mon[33264]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-04T03:01:37.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:36 smithi066 ceph-mon[33264]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:01:37.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:36 smithi066 ceph-mon[33264]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-04T03:01:37.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:36 smithi066 ceph-mon[33264]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:01:37.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:36 smithi167 ceph-mon[33614]: Standby manager daemon y started 2024-05-04T03:01:37.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:36 smithi167 ceph-mon[33614]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-04T03:01:37.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:36 smithi167 ceph-mon[33614]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:01:37.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:36 smithi167 ceph-mon[33614]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-04T03:01:37.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:36 smithi167 ceph-mon[33614]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:01:37.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:36 smithi039 ceph-mon[29715]: Standby manager daemon y started 2024-05-04T03:01:37.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:36 smithi039 ceph-mon[29715]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-04T03:01:37.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:36 smithi039 ceph-mon[29715]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:01:37.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:36 smithi039 ceph-mon[29715]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-04T03:01:37.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:36 smithi039 ceph-mon[29715]: from='mgr.? 172.21.15.66:0/34193350' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:01:38.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:37 smithi167 ceph-mon[33614]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-04T03:01:38.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:37 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-04T03:01:38.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:37 smithi167 ceph-mon[33614]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:38.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:37 smithi039 ceph-mon[29715]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-04T03:01:38.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:37 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-04T03:01:38.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:37 smithi039 ceph-mon[29715]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:38.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:37 smithi066 ceph-mon[33264]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-04T03:01:38.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:37 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-04T03:01:38.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:37 smithi066 ceph-mon[33264]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:40.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:39 smithi039 ceph-mon[29715]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:40.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:39 smithi066 ceph-mon[33264]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:40.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:39 smithi167 ceph-mon[33614]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:42.232 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:41 smithi039 ceph-mon[29715]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:42.232 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-04T03:01:42.232 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:42.232 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:41 smithi039 ceph-mon[29715]: Deploying daemon osd.0 on smithi039 2024-05-04T03:01:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:41 smithi066 ceph-mon[33264]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-04T03:01:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:41 smithi066 ceph-mon[33264]: Deploying daemon osd.0 on smithi039 2024-05-04T03:01:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:41 smithi167 ceph-mon[33614]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-04T03:01:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:41 smithi167 ceph-mon[33614]: Deploying daemon osd.0 on smithi039 2024-05-04T03:01:44.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:43 smithi039 ceph-mon[29715]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:43 smithi066 ceph-mon[33264]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:43 smithi167 ceph-mon[33614]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:46.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:45 smithi039 ceph-mon[29715]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:46.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:45 smithi066 ceph-mon[33264]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:46.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:45 smithi167 ceph-mon[33614]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:47 smithi167 ceph-mon[33614]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:48.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:48.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:48.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:48.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:47 smithi039 ceph-mon[29715]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:48.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:48.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:48.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:48.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:47 smithi066 ceph-mon[33264]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:49 smithi066 ceph-mon[33264]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:49 smithi167 ceph-mon[33614]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:50.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:50.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:50.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:49 smithi039 ceph-mon[29715]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:50.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.797+0000 7f7724ff9700 1 -- 172.21.15.39:0/2198186505 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f772802e9f0 con 0x7f771003e750 2024-05-04T03:01:50.799 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 0 on host 'smithi039' 2024-05-04T03:01:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f771003e750 msgr2=0x7f7710040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f771003e750 0x7f7710040c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7728075e00 tx=0x7f771c009260 comp rx=0 tx=0).stop 2024-05-04T03:01:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7728103250 msgr2=0x7f7728076450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7728103250 0x7f7728076450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77180098e0 tx=0x7f771800b9e0 comp rx=0 tx=0).stop 2024-05-04T03:01:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 shutdown_connections 2024-05-04T03:01:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f771003e750 0x7f7710040c60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7728104d20 0x7f77281a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7728103250 0x7f7728076450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:50.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 --2- 172.21.15.39:0/2198186505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7728104380 0x7f772819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:50.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.801+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 >> 172.21.15.39:0/2198186505 conn(0x7f77280fe890 msgr2=0x7f7728107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:50.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.802+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 shutdown_connections 2024-05-04T03:01:50.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:01:50.802+0000 7f772e01c700 1 -- 172.21.15.39:0/2198186505 wait complete. 2024-05-04T03:01:51.429 DEBUG:teuthology.orchestra.run.smithi039:osd.0> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.0.service 2024-05-04T03:01:51.431 INFO:tasks.cephadm:Deploying osd.1 on smithi066 with /dev/vg_nvme/lv_4... 2024-05-04T03:01:51.432 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-04T03:01:51.693 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:01:52.053 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:51 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:52.053 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:51 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:52.053 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:51 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:52.053 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:51 smithi066 ceph-mon[33264]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-04T03:01:52.054 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:51 smithi066 ceph-mon[33264]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:52.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:51 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:52.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:51 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:52.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:51 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:52.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:51 smithi167 ceph-mon[33614]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-04T03:01:52.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:51 smithi167 ceph-mon[33614]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:01:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:51 smithi039 ceph-mon[29715]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-04T03:01:52.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:51 smithi039 ceph-mon[29715]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:52.870 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:52 smithi039 ceph-mon[29715]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-04T03:01:52.870 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:52 smithi039 ceph-mon[29715]: osdmap e6: 1 total, 0 up, 1 in 2024-05-04T03:01:52.870 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:52 smithi039 ceph-mon[29715]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:01:52.871 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:52.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:52 smithi066 ceph-mon[33264]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-04T03:01:52.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:52 smithi066 ceph-mon[33264]: osdmap e6: 1 total, 0 up, 1 in 2024-05-04T03:01:52.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:52 smithi066 ceph-mon[33264]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:01:52.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:53.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:52 smithi167 ceph-mon[33614]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-04T03:01:53.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:52 smithi167 ceph-mon[33614]: osdmap e6: 1 total, 0 up, 1 in 2024-05-04T03:01:53.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:52 smithi167 ceph-mon[33614]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:01:53.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-04T03:01:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-04T03:01:53.482 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-05-04T03:01:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-05-04T03:01:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113037 s, 928 MB/s 2024-05-04T03:01:53.482 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-05-04T03:01:53.691 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:01:53 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0[42157]: 2024-05-04T03:01:53.242+0000 7fce37816700 -1 osd.0 0 waiting for initial osdmap 2024-05-04T03:01:53.691 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:01:53 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0[42157]: 2024-05-04T03:01:53.247+0000 7fce2f5c3700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:01:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:53 smithi066 ceph-mon[33264]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:01:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:53 smithi066 ceph-mon[33264]: osdmap e7: 1 total, 0 up, 1 in 2024-05-04T03:01:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:53 smithi066 ceph-mon[33264]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:53 smithi066 ceph-mon[33264]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' 2024-05-04T03:01:54.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:53 smithi167 ceph-mon[33614]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:01:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:53 smithi167 ceph-mon[33614]: osdmap e7: 1 total, 0 up, 1 in 2024-05-04T03:01:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:53 smithi167 ceph-mon[33614]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:53 smithi167 ceph-mon[33614]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' 2024-05-04T03:01:54.098 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_4 2024-05-04T03:01:54.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:53 smithi039 ceph-mon[29715]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:01:54.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:53 smithi039 ceph-mon[29715]: osdmap e7: 1 total, 0 up, 1 in 2024-05-04T03:01:54.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:54.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:54.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:53 smithi039 ceph-mon[29715]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:01:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:53 smithi039 ceph-mon[29715]: from='osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170]' entity='osd.0' 2024-05-04T03:01:54.345 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:01:55.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:01:55.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:01:55.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] boot 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: osdmap e8: 1 total, 1 up, 1 in 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:55.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:01:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:01:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] boot 2024-05-04T03:01:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: osdmap e8: 1 total, 1 up, 1 in 2024-05-04T03:01:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:01:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:01:55.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: osd.0 [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] boot 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: osdmap e8: 1 total, 1 up, 1 in 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:01:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 -- 172.21.15.66:0/4200504640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 msgr2=0x7f2bac105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/4200504640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 0x7f2bac105dc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009990 tx=0x7f2b9c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 -- 172.21.15.66:0/4200504640 shutdown_connections 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/4200504640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bac1049e0 0x7f2bac10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/4200504640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 0x7f2bac105dc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/4200504640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac103f60 0x7f2bac104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 -- 172.21.15.66:0/4200504640 >> 172.21.15.66:0/4200504640 conn(0x7f2bac0ff9d0 msgr2=0x7f2bac101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 -- 172.21.15.66:0/4200504640 shutdown_connections 2024-05-04T03:01:55.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.883+0000 7f2bb19b8700 1 -- 172.21.15.66:0/4200504640 wait complete. 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bb19b8700 1 Processor -- start 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bb19b8700 1 -- start start 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bb19b8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bac103f60 0x7f2bac078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bb19b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac1049e0 0x7f2bac076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bb19b8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 0x7f2bac077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bb19b8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10fdd0 con 0x7f2bac1049e0 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bb19b8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10fc50 con 0x7f2bac1059b0 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bb19b8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10ff50 con 0x7f2bac103f60 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bab7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 0x7f2bac077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bab7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 0x7f2bac077160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43600/0 (socket says 172.21.15.66:43600) 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2bab7fe700 1 -- 172.21.15.66:0/2814186164 learned_addr learned my addr 172.21.15.66:0/2814186164 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2baa7fc700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac1049e0 0x7f2bac076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:55.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.884+0000 7f2baaffd700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bac103f60 0x7f2bac078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bab7fe700 1 -- 172.21.15.66:0/2814186164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bac103f60 msgr2=0x7f2bac078570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bab7fe700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bac103f60 0x7f2bac078570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bab7fe700 1 -- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac1049e0 msgr2=0x7f2bac076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bab7fe700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac1049e0 0x7f2bac076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bab7fe700 1 -- 172.21.15.66:0/2814186164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac0778c0 con 0x7f2bac1059b0 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2baaffd700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bac103f60 0x7f2bac078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2baa7fc700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac1049e0 0x7f2bac076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bab7fe700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 0x7f2bac077160 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4007ee0 tx=0x7f2ba400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bb09b6700 1 -- 172.21.15.66:0/2814186164 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400dbd0 con 0x7f2bac1059b0 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bb09b6700 1 -- 172.21.15.66:0/2814186164 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ba400dd70 con 0x7f2bac1059b0 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac1a4a50 con 0x7f2bac1059b0 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bb09b6700 1 -- 172.21.15.66:0/2814186164 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40058d0 con 0x7f2bac1059b0 2024-05-04T03:01:55.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.885+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac1a4ee0 con 0x7f2bac1059b0 2024-05-04T03:01:55.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.886+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bac0512c0 con 0x7f2bac1059b0 2024-05-04T03:01:55.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.887+0000 7f2bb09b6700 1 -- 172.21.15.66:0/2814186164 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f2ba401a020 con 0x7f2bac1059b0 2024-05-04T03:01:55.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.888+0000 7f2bb09b6700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2b940b27f0 0x7f2b940b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:01:55.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.888+0000 7f2bb09b6700 1 -- 172.21.15.66:0/2814186164 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f2ba40e0000 con 0x7f2bac1059b0 2024-05-04T03:01:55.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.888+0000 7f2baaffd700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2b940b27f0 0x7f2b940b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:01:55.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.889+0000 7f2baaffd700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2b940b27f0 0x7f2b940b4d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006010 tx=0x7f2b9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:01:55.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:55.893+0000 7f2bb09b6700 1 -- 172.21.15.66:0/2814186164 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ba4010070 con 0x7f2bac1059b0 2024-05-04T03:01:56.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:55 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi039 2024-05-04T03:01:56.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:55 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi039 to 17088M 2024-05-04T03:01:56.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:55 smithi066 ceph-mon[33264]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:01:56.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:01:56.117+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2bac063c00 con 0x7f2b940b27f0 2024-05-04T03:01:56.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:55 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi039 2024-05-04T03:01:56.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:55 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi039 to 17088M 2024-05-04T03:01:56.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:55 smithi167 ceph-mon[33614]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:01:56.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:55 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi039 2024-05-04T03:01:56.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:55 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi039 to 17088M 2024-05-04T03:01:56.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:55 smithi039 ceph-mon[29715]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:01:57.058 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:56 smithi066 ceph-mon[33264]: osdmap e9: 1 total, 1 up, 1 in 2024-05-04T03:01:57.058 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:56 smithi066 ceph-mon[33264]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:57.058 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:01:57.058 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:01:57.059 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:57.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:56 smithi167 ceph-mon[33614]: osdmap e9: 1 total, 1 up, 1 in 2024-05-04T03:01:57.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:56 smithi167 ceph-mon[33614]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:57.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:01:57.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:01:57.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:57.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:56 smithi039 ceph-mon[29715]: osdmap e9: 1 total, 1 up, 1 in 2024-05-04T03:01:57.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:56 smithi039 ceph-mon[29715]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:01:57.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:01:57.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:01:57.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:01:58.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:57 smithi066 ceph-mon[33264]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:01:58.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:57 smithi167 ceph-mon[33614]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:01:58.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:57 smithi039 ceph-mon[29715]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:01:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:58 smithi066 ceph-mon[33264]: from='client.? 172.21.15.66:0/1873160384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]: dispatch 2024-05-04T03:01:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:58 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]: dispatch 2024-05-04T03:01:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:58 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]': finished 2024-05-04T03:01:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:58 smithi066 ceph-mon[33264]: osdmap e10: 2 total, 1 up, 2 in 2024-05-04T03:01:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:01:59.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:58 smithi167 ceph-mon[33614]: from='client.? 172.21.15.66:0/1873160384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]: dispatch 2024-05-04T03:01:59.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:58 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]: dispatch 2024-05-04T03:01:59.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:58 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]': finished 2024-05-04T03:01:59.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:58 smithi167 ceph-mon[33614]: osdmap e10: 2 total, 1 up, 2 in 2024-05-04T03:01:59.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:01:59.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:58 smithi039 ceph-mon[29715]: from='client.? 172.21.15.66:0/1873160384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]: dispatch 2024-05-04T03:01:59.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:58 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]: dispatch 2024-05-04T03:01:59.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:58 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e"}]': finished 2024-05-04T03:01:59.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:58 smithi039 ceph-mon[29715]: osdmap e10: 2 total, 1 up, 2 in 2024-05-04T03:01:59.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:00.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:59 smithi167 ceph-mon[33614]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:00.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:01:59 smithi167 ceph-mon[33614]: from='client.? 172.21.15.66:0/542992611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:00.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:59 smithi039 ceph-mon[29715]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:00.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:01:59 smithi039 ceph-mon[29715]: from='client.? 172.21.15.66:0/542992611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:00.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:59 smithi066 ceph-mon[33264]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:00.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:01:59 smithi066 ceph-mon[33264]: from='client.? 172.21.15.66:0/542992611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:02.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:01 smithi066 ceph-mon[33264]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:02.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:01 smithi167 ceph-mon[33614]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:02.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:01 smithi039 ceph-mon[29715]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:04.154 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:03 smithi066 ceph-mon[33264]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:04.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:03 smithi167 ceph-mon[33614]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:04.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:03 smithi039 ceph-mon[29715]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:06.184 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:05 smithi066 ceph-mon[33264]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:06.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:05 smithi167 ceph-mon[33614]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:06.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:05 smithi039 ceph-mon[29715]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:07.236 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:06 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-04T03:02:07.236 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:06 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:07.236 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:06 smithi066 ceph-mon[33264]: Deploying daemon osd.1 on smithi066 2024-05-04T03:02:07.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-04T03:02:07.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:07.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:06 smithi167 ceph-mon[33614]: Deploying daemon osd.1 on smithi066 2024-05-04T03:02:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-04T03:02:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:06 smithi039 ceph-mon[29715]: Deploying daemon osd.1 on smithi066 2024-05-04T03:02:08.172 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:07 smithi066 ceph-mon[33264]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:07 smithi167 ceph-mon[33614]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:08.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:07 smithi039 ceph-mon[29715]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:10.276 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:09 smithi066 ceph-mon[33264]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:09 smithi167 ceph-mon[33614]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:10.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:09 smithi039 ceph-mon[29715]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:12.515 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:12 smithi066 ceph-mon[33264]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:12.515 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:12 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:12.516 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:12 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:12.516 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:12 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:12.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:12 smithi167 ceph-mon[33614]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:12.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:12 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:12.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:12 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:12.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:12 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:12 smithi039 ceph-mon[29715]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:12 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:12 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:12 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.637+0000 7f2bb09b6700 1 -- 172.21.15.66:0/2814186164 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2bac063c00 con 0x7f2b940b27f0 2024-05-04T03:02:14.638 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 1 on host 'smithi066' 2024-05-04T03:02:14.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:14 smithi066 ceph-mon[33264]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:14.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:14.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:14.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:14.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:14.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2b940b27f0 msgr2=0x7f2b940b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2b940b27f0 0x7f2b940b4d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006010 tx=0x7f2b9c01a040 comp rx=0 tx=0).stop 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 msgr2=0x7f2bac077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 0x7f2bac077160 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4007ee0 tx=0x7f2ba400d3b0 comp rx=0 tx=0).stop 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 shutdown_connections 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2b940b27f0 0x7f2b940b4d00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2bac1059b0 0x7f2bac077160 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bac103f60 0x7f2bac078570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 --2- 172.21.15.66:0/2814186164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bac1049e0 0x7f2bac076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 >> 172.21.15.66:0/2814186164 conn(0x7f2bac0ff9d0 msgr2=0x7f2bac108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:02:14.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 shutdown_connections 2024-05-04T03:02:14.641 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:02:14.640+0000 7f2bb19b8700 1 -- 172.21.15.66:0/2814186164 wait complete. 2024-05-04T03:02:14.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:14 smithi167 ceph-mon[33614]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:14.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:14.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:14.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:14 smithi039 ceph-mon[29715]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:14.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:14.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:15.230 DEBUG:teuthology.orchestra.run.smithi066:osd.1> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.1.service 2024-05-04T03:02:15.232 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-05-04T03:02:15.232 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-04T03:02:15.476 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:02:15.529 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:15 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1[37875]: 2024-05-04T03:02:15.283+0000 7fa132e1a7c0 -1 osd.1 0 log_to_monitors true 2024-05-04T03:02:15.791 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:15.791 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:15.791 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:15.791 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:15 smithi039 ceph-mon[29715]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:15.791 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:15 smithi039 ceph-mon[29715]: from='osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:02:15.791 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:15 smithi039 ceph-mon[29715]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:02:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:15 smithi066 ceph-mon[33264]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:15 smithi066 ceph-mon[33264]: from='osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:02:15.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:15 smithi066 ceph-mon[33264]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:02:15.914 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:15.915 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:15.915 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:15.915 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:15 smithi167 ceph-mon[33614]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:15.915 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:15 smithi167 ceph-mon[33614]: from='osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:02:15.915 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:15 smithi167 ceph-mon[33614]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:02:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:16 smithi167 ceph-mon[33614]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-04T03:02:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:16 smithi167 ceph-mon[33614]: osdmap e11: 2 total, 1 up, 2 in 2024-05-04T03:02:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:16 smithi167 ceph-mon[33614]: from='osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:02:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:16 smithi167 ceph-mon[33614]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:02:16.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:16 smithi039 ceph-mon[29715]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-04T03:02:16.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:16 smithi039 ceph-mon[29715]: osdmap e11: 2 total, 1 up, 2 in 2024-05-04T03:02:16.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:16 smithi039 ceph-mon[29715]: from='osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:02:16.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:16.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:16 smithi039 ceph-mon[29715]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:02:17.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:16 smithi066 ceph-mon[33264]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-04T03:02:17.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:16 smithi066 ceph-mon[33264]: osdmap e11: 2 total, 1 up, 2 in 2024-05-04T03:02:17.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:16 smithi066 ceph-mon[33264]: from='osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:02:17.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:17.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:16 smithi066 ceph-mon[33264]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:02:17.273 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-04T03:02:17.273 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-04T03:02:17.273 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-05-04T03:02:17.273 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-05-04T03:02:17.273 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137954 s, 760 MB/s 2024-05-04T03:02:17.273 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-05-04T03:02:17.567 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:17 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1[37875]: 2024-05-04T03:02:17.115+0000 7fa12847e700 -1 osd.1 0 waiting for initial osdmap 2024-05-04T03:02:17.567 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:17 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1[37875]: 2024-05-04T03:02:17.120+0000 7fa12022b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:02:17.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:17 smithi167 ceph-mon[33614]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:02:17.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:17 smithi167 ceph-mon[33614]: osdmap e12: 2 total, 1 up, 2 in 2024-05-04T03:02:17.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:17.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:17 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:17.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:17 smithi167 ceph-mon[33614]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:17.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:17 smithi167 ceph-mon[33614]: from='osd.1 ' entity='osd.1' 2024-05-04T03:02:17.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-05-04T03:02:17.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:17 smithi039 ceph-mon[29715]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:02:17.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:17 smithi039 ceph-mon[29715]: osdmap e12: 2 total, 1 up, 2 in 2024-05-04T03:02:17.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:17.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:17 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:17.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:17 smithi039 ceph-mon[29715]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:17.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:17 smithi039 ceph-mon[29715]: from='osd.1 ' entity='osd.1' 2024-05-04T03:02:17.966 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:17 smithi066 ceph-mon[33264]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:02:17.966 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:17 smithi066 ceph-mon[33264]: osdmap e12: 2 total, 1 up, 2 in 2024-05-04T03:02:17.966 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:17.966 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:17 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:17.966 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:17 smithi066 ceph-mon[33264]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:02:17.966 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:17 smithi066 ceph-mon[33264]: from='osd.1 ' entity='osd.1' 2024-05-04T03:02:18.170 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:02:18.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:02:18.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] boot 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: osdmap e13: 2 total, 2 up, 2 in 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi066 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi066 to 17088M 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:18.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:02:18.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:02:18.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] boot 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: osdmap e13: 2 total, 2 up, 2 in 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi066 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi066 to 17088M 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:19.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:02:19.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:02:19.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:19.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: osd.1 [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] boot 2024-05-04T03:02:19.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: osdmap e13: 2 total, 2 up, 2 in 2024-05-04T03:02:19.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:02:19.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi066 2024-05-04T03:02:19.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:19.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:19.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:02:19.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi066 to 17088M 2024-05-04T03:02:19.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:19.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:19.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:19.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:19.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 -- 172.21.15.167:0/252442465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f18104ce0 msgr2=0x7f7f1810b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:19.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 --2- 172.21.15.167:0/252442465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f18104ce0 0x7f7f1810b770 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b0a0 tx=0x7f7f1000b3b0 comp rx=0 tx=0).stop 2024-05-04T03:02:19.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 -- 172.21.15.167:0/252442465 shutdown_connections 2024-05-04T03:02:19.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 --2- 172.21.15.167:0/252442465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f181037a0 0x7f7f18103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:19.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 --2- 172.21.15.167:0/252442465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f18104ce0 0x7f7f1810b770 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:19.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 --2- 172.21.15.167:0/252442465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f18104180 0x7f7f18104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 -- 172.21.15.167:0/252442465 >> 172.21.15.167:0/252442465 conn(0x7f7f180fe010 msgr2=0x7f7f181004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 -- 172.21.15.167:0/252442465 shutdown_connections 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.603+0000 7f7f1e265700 1 -- 172.21.15.167:0/252442465 wait complete. 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.604+0000 7f7f1e265700 1 Processor -- start 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.604+0000 7f7f1e265700 1 -- start start 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f1e265700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f181037a0 0x7f7f181a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f1e265700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f18104180 0x7f7f181a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f1e265700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f18104ce0 0x7f7f181a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:02:19.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f1e265700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1810eba0 con 0x7f7f18104ce0 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f1e265700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1810ea20 con 0x7f7f18104180 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f1e265700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1810ed20 con 0x7f7f181037a0 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f181037a0 0x7f7f181a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f181037a0 0x7f7f181a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54212/0 (socket says 172.21.15.167:54212) 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 -- 172.21.15.167:0/599500582 learned_addr learned my addr 172.21.15.167:0/599500582 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f1c802700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f18104ce0 0x7f7f181a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f177fe700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f18104180 0x7f7f181a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 -- 172.21.15.167:0/599500582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f18104180 msgr2=0x7f7f181a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f18104180 0x7f7f181a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 -- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f18104ce0 msgr2=0x7f7f181a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f18104ce0 0x7f7f181a82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 -- 172.21.15.167:0/599500582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f181a8980 con 0x7f7f181037a0 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f177fe700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f18104180 0x7f7f181a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:02:19.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f17fff700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f181037a0 0x7f7f181a10f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800e9a0 tx=0x7f7f0800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:02:19.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f157fa700 1 -- 172.21.15.167:0/599500582 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800cbd0 con 0x7f7f181037a0 2024-05-04T03:02:19.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.605+0000 7f7f157fa700 1 -- 172.21.15.167:0/599500582 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f0800cd70 con 0x7f7f181037a0 2024-05-04T03:02:19.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.606+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f181a8c10 con 0x7f7f181037a0 2024-05-04T03:02:19.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.606+0000 7f7f157fa700 1 -- 172.21.15.167:0/599500582 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08005540 con 0x7f7f181037a0 2024-05-04T03:02:19.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.606+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f181a9150 con 0x7f7f181037a0 2024-05-04T03:02:19.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.607+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f180512c0 con 0x7f7f181037a0 2024-05-04T03:02:19.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.608+0000 7f7f157fa700 1 -- 172.21.15.167:0/599500582 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f7f08004750 con 0x7f7f181037a0 2024-05-04T03:02:19.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.609+0000 7f7f157fa700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f000b27f0 0x7f7f000b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:02:19.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.609+0000 7f7f157fa700 1 -- 172.21.15.167:0/599500582 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f7f080e0bc0 con 0x7f7f181037a0 2024-05-04T03:02:19.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.609+0000 7f7f177fe700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f000b27f0 0x7f7f000b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:02:19.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.609+0000 7f7f177fe700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f000b27f0 0x7f7f000b4d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c005fd0 tx=0x7f7f0c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:02:19.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.614+0000 7f7f157fa700 1 -- 172.21.15.167:0/599500582 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f080aa130 con 0x7f7f181037a0 2024-05-04T03:02:19.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:19.828+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7f181a93e0 con 0x7f7f000b27f0 2024-05-04T03:02:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:20 smithi167 ceph-mon[33614]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:20 smithi167 ceph-mon[33614]: osdmap e14: 2 total, 2 up, 2 in 2024-05-04T03:02:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:02:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:02:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:20 smithi039 ceph-mon[29715]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:20 smithi039 ceph-mon[29715]: osdmap e14: 2 total, 2 up, 2 in 2024-05-04T03:02:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:02:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:02:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:20.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:20 smithi066 ceph-mon[33264]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:20.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:20 smithi066 ceph-mon[33264]: osdmap e14: 2 total, 2 up, 2 in 2024-05-04T03:02:20.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:02:20.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:02:20.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:21.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:21 smithi167 ceph-mon[33614]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:02:21.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:21 smithi039 ceph-mon[29715]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:02:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:21 smithi066 ceph-mon[33264]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:02:22.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:22 smithi167 ceph-mon[33614]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:22.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:22 smithi039 ceph-mon[29715]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:22 smithi066 ceph-mon[33264]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:23.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:23 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/3093625280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]: dispatch 2024-05-04T03:02:23.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:23 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]: dispatch 2024-05-04T03:02:23.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:23 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]': finished 2024-05-04T03:02:23.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:23 smithi167 ceph-mon[33614]: osdmap e15: 3 total, 2 up, 3 in 2024-05-04T03:02:23.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:23.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:23 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/3027973265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:23.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:23 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/3093625280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]: dispatch 2024-05-04T03:02:23.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:23 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]: dispatch 2024-05-04T03:02:23.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:23 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]': finished 2024-05-04T03:02:23.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:23 smithi039 ceph-mon[29715]: osdmap e15: 3 total, 2 up, 3 in 2024-05-04T03:02:23.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:23.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:23 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/3027973265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:23.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:23 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/3093625280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]: dispatch 2024-05-04T03:02:23.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:23 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]: dispatch 2024-05-04T03:02:23.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:23 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "339eb4b8-1e7f-490b-9d54-63886da97018"}]': finished 2024-05-04T03:02:23.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:23 smithi066 ceph-mon[33264]: osdmap e15: 3 total, 2 up, 3 in 2024-05-04T03:02:23.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:23.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:23 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/3027973265' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:24 smithi167 ceph-mon[33614]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:24.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:24 smithi039 ceph-mon[29715]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:24 smithi066 ceph-mon[33264]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:26 smithi167 ceph-mon[33614]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:26.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:26 smithi039 ceph-mon[29715]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:26.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:26 smithi066 ceph-mon[33264]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:28.162 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:28 smithi167 ceph-mon[33614]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:28.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:28 smithi039 ceph-mon[29715]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:28.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:28 smithi066 ceph-mon[33264]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:30.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:30 smithi167 ceph-mon[33614]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:30.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-04T03:02:30.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:30.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:30 smithi039 ceph-mon[29715]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:30.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-04T03:02:30.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:30.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:30 smithi066 ceph-mon[33264]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:30.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-04T03:02:30.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:31.261 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:31 smithi167 ceph-mon[33614]: Deploying daemon osd.2 on smithi167 2024-05-04T03:02:31.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:31 smithi039 ceph-mon[29715]: Deploying daemon osd.2 on smithi167 2024-05-04T03:02:31.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:31 smithi066 ceph-mon[33264]: Deploying daemon osd.2 on smithi167 2024-05-04T03:02:32.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:32 smithi167 ceph-mon[33614]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:32.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:32 smithi039 ceph-mon[29715]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:32.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:32 smithi066 ceph-mon[33264]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:34.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:34 smithi167 ceph-mon[33614]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:34.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:34 smithi039 ceph-mon[29715]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:34.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:34 smithi066 ceph-mon[33264]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:35.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:35.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:35.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:35.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:35 smithi167 ceph-mon[33614]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:36.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:36.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:36.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:36.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:35 smithi066 ceph-mon[33264]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:35 smithi039 ceph-mon[29715]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:37.944 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-05-04T03:02:37.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.943+0000 7f7f157fa700 1 -- 172.21.15.167:0/599500582 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7f181a93e0 con 0x7f7f000b27f0 2024-05-04T03:02:37.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f000b27f0 msgr2=0x7f7f000b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f000b27f0 0x7f7f000b4d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c005fd0 tx=0x7f7f0c005ee0 comp rx=0 tx=0).stop 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f181037a0 msgr2=0x7f7f181a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f181037a0 0x7f7f181a10f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800e9a0 tx=0x7f7f0800ecb0 comp rx=0 tx=0).stop 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 shutdown_connections 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f000b27f0 0x7f7f000b4d00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f18104ce0 0x7f7f181a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f181037a0 0x7f7f181a10f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 --2- 172.21.15.167:0/599500582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f18104180 0x7f7f181a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 >> 172.21.15.167:0/599500582 conn(0x7f7f180fe010 msgr2=0x7f7f180ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 shutdown_connections 2024-05-04T03:02:37.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:02:37.946+0000 7f7f1e265700 1 -- 172.21.15.167:0/599500582 wait complete. 2024-05-04T03:02:38.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:38.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:38.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:38.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:38.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.572 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.2.service 2024-05-04T03:02:38.574 INFO:tasks.cephadm:Deploying osd.3 on smithi039 with /dev/vg_nvme/lv_3... 2024-05-04T03:02:38.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-04T03:02:38.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:38.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:02:38.866 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2[38939]: 2024-05-04T03:02:38.627+0000 7f01cd1d07c0 -1 osd.2 0 log_to_monitors true 2024-05-04T03:02:39.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:39 smithi167 ceph-mon[33614]: from='osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:02:39.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:39 smithi167 ceph-mon[33614]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:02:39.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:39 smithi039 ceph-mon[29715]: from='osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:02:39.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:39 smithi039 ceph-mon[29715]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:02:39.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:39 smithi066 ceph-mon[33264]: from='osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:02:39.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:39 smithi066 ceph-mon[33264]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:02:40.520 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:40 smithi039 ceph-mon[29715]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:40.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:40 smithi039 ceph-mon[29715]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-04T03:02:40.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:40 smithi039 ceph-mon[29715]: from='osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:02:40.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:40 smithi039 ceph-mon[29715]: osdmap e16: 3 total, 2 up, 3 in 2024-05-04T03:02:40.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:40.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:40 smithi039 ceph-mon[29715]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:02:40.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:40 smithi167 ceph-mon[33614]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:40.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:40 smithi167 ceph-mon[33614]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-04T03:02:40.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:40 smithi167 ceph-mon[33614]: from='osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:02:40.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:40 smithi167 ceph-mon[33614]: osdmap e16: 3 total, 2 up, 3 in 2024-05-04T03:02:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:40 smithi167 ceph-mon[33614]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:02:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:40 smithi066 ceph-mon[33264]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:40 smithi066 ceph-mon[33264]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-04T03:02:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:40 smithi066 ceph-mon[33264]: from='osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:02:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:40 smithi066 ceph-mon[33264]: osdmap e16: 3 total, 2 up, 3 in 2024-05-04T03:02:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:40 smithi066 ceph-mon[33264]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:02:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-04T03:02:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-04T03:02:40.927 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-04T03:02:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-04T03:02:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00716563 s, 1.5 GB/s 2024-05-04T03:02:40.927 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 2024-05-04T03:02:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-04T03:02:41.176 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2[38939]: 2024-05-04T03:02:40.883+0000 7f01c2834700 -1 osd.2 0 waiting for initial osdmap 2024-05-04T03:02:41.176 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2[38939]: 2024-05-04T03:02:40.888+0000 7f01ba5e1700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:02:41.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:41 smithi167 ceph-mon[33614]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:02:41.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:41 smithi167 ceph-mon[33614]: osdmap e17: 3 total, 2 up, 3 in 2024-05-04T03:02:41.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:41.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:41.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:41 smithi167 ceph-mon[33614]: from='osd.2 ' entity='osd.2' 2024-05-04T03:02:41.628 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:41 smithi039 ceph-mon[29715]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:02:41.628 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:41 smithi039 ceph-mon[29715]: osdmap e17: 3 total, 2 up, 3 in 2024-05-04T03:02:41.628 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:41.628 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:41.628 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:41 smithi039 ceph-mon[29715]: from='osd.2 ' entity='osd.2' 2024-05-04T03:02:41.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_3 2024-05-04T03:02:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:41 smithi066 ceph-mon[33264]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:02:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:41 smithi066 ceph-mon[33264]: osdmap e17: 3 total, 2 up, 3 in 2024-05-04T03:02:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:41 smithi066 ceph-mon[33264]: from='osd.2 ' entity='osd.2' 2024-05-04T03:02:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:42.618 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:42.619 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.619 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] boot 2024-05-04T03:02:42.619 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: osdmap e18: 3 total, 3 up, 3 in 2024-05-04T03:02:42.619 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:42.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:02:42.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:02:42.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] boot 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: osdmap e18: 3 total, 3 up, 3 in 2024-05-04T03:02:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:42.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:02:42.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:02:42.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:02:42.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: osd.2 [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] boot 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: osdmap e18: 3 total, 3 up, 3 in 2024-05-04T03:02:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:02:43.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 -- 172.21.15.39:0/3036698449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03410a0f0 msgr2=0x7fa03410c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:43.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 --2- 172.21.15.39:0/3036698449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03410a0f0 0x7fa03410c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa02c00b0a0 tx=0x7fa02c00b3b0 comp rx=0 tx=0).stop 2024-05-04T03:02:43.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 -- 172.21.15.39:0/3036698449 shutdown_connections 2024-05-04T03:02:43.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 --2- 172.21.15.39:0/3036698449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03410a0f0 0x7fa03410c510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:43.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 --2- 172.21.15.39:0/3036698449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa034104e30 0x7fa034107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:43.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 --2- 172.21.15.39:0/3036698449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa034107790 0x7fa034109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:43.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 -- 172.21.15.39:0/3036698449 >> 172.21.15.39:0/3036698449 conn(0x7fa0340fe870 msgr2=0x7fa034100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:02:43.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 -- 172.21.15.39:0/3036698449 shutdown_connections 2024-05-04T03:02:43.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.431+0000 7fa03b606700 1 -- 172.21.15.39:0/3036698449 wait complete. 2024-05-04T03:02:43.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa03b606700 1 Processor -- start 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa03b606700 1 -- start start 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa03b606700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa034104e30 0x7fa03419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa03b606700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa034107790 0x7fa03419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa03b606700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03410a0f0 0x7fa0341a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa03b606700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa03410fd00 con 0x7fa034104e30 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa03b606700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa03410fb80 con 0x7fa03410a0f0 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa03b606700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa03410fe80 con 0x7fa034107790 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa0393a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa034104e30 0x7fa03419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa0393a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa034104e30 0x7fa03419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52854/0 (socket says 172.21.15.39:52854) 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.432+0000 7fa0393a2700 1 -- 172.21.15.39:0/4173379765 learned_addr learned my addr 172.21.15.39:0/4173379765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa038ba1700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa034107790 0x7fa03419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:02:43.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa039ba3700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03410a0f0 0x7fa0341a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa0393a2700 1 -- 172.21.15.39:0/4173379765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa034107790 msgr2=0x7fa03419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa0393a2700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa034107790 0x7fa03419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa0393a2700 1 -- 172.21.15.39:0/4173379765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03410a0f0 msgr2=0x7fa0341a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa0393a2700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03410a0f0 0x7fa0341a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa0393a2700 1 -- 172.21.15.39:0/4173379765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0341a23a0 con 0x7fa034104e30 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa0393a2700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa034104e30 0x7fa03419a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa03000d980 tx=0x7fa03000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4173379765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030017070 con 0x7fa034104e30 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4173379765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa030009dc0 con 0x7fa034104e30 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0341a2690 con 0x7fa034104e30 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4173379765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03000f3c0 con 0x7fa034104e30 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.433+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0341a2c50 con 0x7fa034104e30 2024-05-04T03:02:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.434+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0340512c0 con 0x7fa034104e30 2024-05-04T03:02:43.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.436+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4173379765 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fa030009930 con 0x7fa034104e30 2024-05-04T03:02:43.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.437+0000 7fa02a7fc700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0200b27f0 0x7fa0200b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:02:43.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.437+0000 7fa038ba1700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0200b27f0 0x7fa0200b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:02:43.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.437+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4173379765 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fa0300e0830 con 0x7fa034104e30 2024-05-04T03:02:43.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.437+0000 7fa038ba1700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0200b27f0 0x7fa0200b4d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa03419b8f0 tx=0x7fa024009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:02:43.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.442+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4173379765 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0300a9a10 con 0x7fa034104e30 2024-05-04T03:02:43.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:43.668+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa034063c00 con 0x7fa0200b27f0 2024-05-04T03:02:43.670 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:43 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi167 2024-05-04T03:02:43.670 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:43 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi167 to 17088M 2024-05-04T03:02:43.670 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:43 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:02:43.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:43 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi167 2024-05-04T03:02:43.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:43 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi167 to 17088M 2024-05-04T03:02:43.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:43 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:02:43.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:43 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi167 2024-05-04T03:02:43.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:43 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi167 to 17088M 2024-05-04T03:02:43.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:43 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:02:44.640 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 ceph-mon[29715]: pgmap v82: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:44.640 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-04T03:02:44.640 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 ceph-mon[29715]: osdmap e19: 3 total, 3 up, 3 in 2024-05-04T03:02:44.640 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:02:44.640 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:02:44.640 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:02:44.640 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:44.641 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47001]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:02:44.641 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47001]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:44.641 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47001]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:44.641 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47001]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:44.641 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47004]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:02:44.641 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47004]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:44.641 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47004]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:44.641 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47004]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:44.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 ceph-mon[33614]: pgmap v82: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-04T03:02:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 ceph-mon[33614]: osdmap e19: 3 total, 3 up, 3 in 2024-05-04T03:02:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:02:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:02:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:02:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:44.677 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:44 smithi167 sudo[42997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:02:44.677 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:44 smithi167 sudo[42997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:44.677 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:44 smithi167 sudo[42997]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:44.677 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:44 smithi167 sudo[42997]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:44.677 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:02:44.678 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:44.678 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43000]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:44.678 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43000]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:44.741 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:02:44.741 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:44.742 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41986]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:44.742 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41986]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:44.742 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41989]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:02:44.742 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41989]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:44.742 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41989]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:44.742 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41989]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:44.742 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 ceph-mon[33264]: pgmap v82: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:44.742 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-04T03:02:44.742 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 ceph-mon[33264]: osdmap e19: 3 total, 3 up, 3 in 2024-05-04T03:02:44.742 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:02:44.743 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:02:44.743 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:02:44.743 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:44.743 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41992]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:02:44.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47021]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:02:44.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:44.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47021]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:44.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47021]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:44.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47024]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:02:44.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47024]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:44.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47024]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:44.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:44 smithi039 sudo[47024]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:45.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:45.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41992]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:45.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41992]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:45.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41995]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:02:45.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:45.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41995]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:45.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:44 smithi066 sudo[41995]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:45.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43003]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:02:45.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43003]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:45.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43003]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:45.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43003]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:45.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:02:45.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:02:45.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43006]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:02:45.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:44 smithi167 sudo[43006]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: osdmap e20: 3 total, 3 up, 3 in 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.522 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:45 smithi039 ceph-mon[29715]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:02:45.666+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4173379765 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa0300a55a0 con 0x7fa034104e30 2024-05-04T03:02:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:02:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:02:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: osdmap e20: 3 total, 3 up, 3 in 2024-05-04T03:02:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:45 smithi167 ceph-mon[33614]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:45.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:02:45.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: osdmap e20: 3 total, 3 up, 3 in 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:02:45.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:02:45.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:02:45.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:02:45.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:45 smithi066 ceph-mon[33264]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:02:46.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:46 smithi167 ceph-mon[33614]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:46.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:46 smithi167 ceph-mon[33614]: osdmap e21: 3 total, 3 up, 3 in 2024-05-04T03:02:46.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:46 smithi167 ceph-mon[33614]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-04T03:02:46.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:46 smithi039 ceph-mon[29715]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:46.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:46 smithi039 ceph-mon[29715]: osdmap e21: 3 total, 3 up, 3 in 2024-05-04T03:02:46.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:46 smithi039 ceph-mon[29715]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-04T03:02:46.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:46 smithi066 ceph-mon[33264]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:46.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:46 smithi066 ceph-mon[33264]: osdmap e21: 3 total, 3 up, 3 in 2024-05-04T03:02:46.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:46 smithi066 ceph-mon[33264]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-04T03:02:47.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:47 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3379978899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "118295e4-04a9-4997-bfa5-a589e4849058"}]: dispatch 2024-05-04T03:02:47.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:47 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3379978899' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "118295e4-04a9-4997-bfa5-a589e4849058"}]': finished 2024-05-04T03:02:47.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:47 smithi167 ceph-mon[33614]: osdmap e22: 4 total, 3 up, 4 in 2024-05-04T03:02:47.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:02:47.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:47 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3466503786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:47 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3379978899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "118295e4-04a9-4997-bfa5-a589e4849058"}]: dispatch 2024-05-04T03:02:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:47 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3379978899' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "118295e4-04a9-4997-bfa5-a589e4849058"}]': finished 2024-05-04T03:02:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:47 smithi039 ceph-mon[29715]: osdmap e22: 4 total, 3 up, 4 in 2024-05-04T03:02:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:02:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:47 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3466503786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:47.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:47 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3379978899' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "118295e4-04a9-4997-bfa5-a589e4849058"}]: dispatch 2024-05-04T03:02:47.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:47 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3379978899' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "118295e4-04a9-4997-bfa5-a589e4849058"}]': finished 2024-05-04T03:02:47.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:47 smithi066 ceph-mon[33264]: osdmap e22: 4 total, 3 up, 4 in 2024-05-04T03:02:47.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:02:47.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:47 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3466503786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:02:48.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:48 smithi167 ceph-mon[33614]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:48 smithi039 ceph-mon[29715]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:48.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:48 smithi066 ceph-mon[33264]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:49.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:49 smithi039 ceph-mon[29715]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:49.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:49 smithi066 ceph-mon[33264]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:49.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:49 smithi167 ceph-mon[33614]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:51 smithi039 ceph-mon[29715]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:51 smithi066 ceph-mon[33264]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:51 smithi167 ceph-mon[33614]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:54.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:53 smithi167 ceph-mon[33614]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:54.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:53 smithi066 ceph-mon[33264]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:54.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:53 smithi039 ceph-mon[29715]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:55.307 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-04T03:02:55.307 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:55.307 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:54 smithi039 ceph-mon[29715]: Deploying daemon osd.3 on smithi039 2024-05-04T03:02:55.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-04T03:02:55.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:55.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:54 smithi066 ceph-mon[33264]: Deploying daemon osd.3 on smithi039 2024-05-04T03:02:55.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-04T03:02:55.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:02:55.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:54 smithi167 ceph-mon[33614]: Deploying daemon osd.3 on smithi039 2024-05-04T03:02:56.294 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:55 smithi039 ceph-mon[29715]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:55 smithi066 ceph-mon[33264]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:55 smithi167 ceph-mon[33614]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:58.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:57 smithi039 ceph-mon[29715]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:58.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:57 smithi066 ceph-mon[33264]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:02:58.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:57 smithi167 ceph-mon[33614]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:00.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:59 smithi066 ceph-mon[33264]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:00.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:00.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:00.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:02:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:00.376 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:59 smithi039 ceph-mon[29715]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:00.376 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:00.376 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:00.376 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:02:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:00.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:59 smithi167 ceph-mon[33614]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:00.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:00.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:00.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:02:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:02.311 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:01 smithi039 ceph-mon[29715]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:02.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:01 smithi066 ceph-mon[33264]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:02.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:01 smithi167 ceph-mon[33614]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.837+0000 7fa02a7fc700 1 -- 172.21.15.39:0/4173379765 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa034063c00 con 0x7fa0200b27f0 2024-05-04T03:03:03.838 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 3 on host 'smithi039' 2024-05-04T03:03:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.839+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0200b27f0 msgr2=0x7fa0200b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.839+0000 7fa03b606700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0200b27f0 0x7fa0200b4d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa03419b8f0 tx=0x7fa024009190 comp rx=0 tx=0).stop 2024-05-04T03:03:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa034104e30 msgr2=0x7fa03419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa034104e30 0x7fa03419a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa03000d980 tx=0x7fa03000dc90 comp rx=0 tx=0).stop 2024-05-04T03:03:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 shutdown_connections 2024-05-04T03:03:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0200b27f0 0x7fa0200b4d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa034104e30 0x7fa03419a840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03410a0f0 0x7fa0341a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:03.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 --2- 172.21.15.39:0/4173379765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa034107790 0x7fa03419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:03.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 >> 172.21.15.39:0/4173379765 conn(0x7fa0340fe870 msgr2=0x7fa034105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:03:03.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 shutdown_connections 2024-05-04T03:03:03.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:03.840+0000 7fa03b606700 1 -- 172.21.15.39:0/4173379765 wait complete. 2024-05-04T03:03:04.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:04.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:04.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:03 smithi066 ceph-mon[33264]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:04.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:03 smithi066 ceph-mon[33264]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-04T03:03:04.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:04.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:04.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:03 smithi039 ceph-mon[29715]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:04.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:03 smithi039 ceph-mon[29715]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-04T03:03:04.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:04.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:04.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:03 smithi167 ceph-mon[33614]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:04.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:03 smithi167 ceph-mon[33614]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-04T03:03:04.497 DEBUG:teuthology.orchestra.run.smithi039:osd.3> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.3.service 2024-05-04T03:03:04.500 INFO:tasks.cephadm:Deploying osd.4 on smithi066 with /dev/vg_nvme/lv_3... 2024-05-04T03:03:04.500 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-04T03:03:04.754 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:03:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:04 smithi066 ceph-mon[33264]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-04T03:03:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:04 smithi066 ceph-mon[33264]: osdmap e23: 4 total, 3 up, 4 in 2024-05-04T03:03:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:04 smithi066 ceph-mon[33264]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:03:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:05.040 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:04 smithi039 ceph-mon[29715]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-04T03:03:05.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:04 smithi039 ceph-mon[29715]: osdmap e23: 4 total, 3 up, 4 in 2024-05-04T03:03:05.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:04 smithi039 ceph-mon[29715]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:03:05.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:05.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:05.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:05.041 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:05.041 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:03:04 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3[49136]: 2024-05-04T03:03:04.771+0000 7f2f07ca5700 -1 osd.3 0 waiting for initial osdmap 2024-05-04T03:03:05.041 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:03:04 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3[49136]: 2024-05-04T03:03:04.780+0000 7f2f03a5a700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:03:05.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:04 smithi167 ceph-mon[33614]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-04T03:03:05.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:04 smithi167 ceph-mon[33614]: osdmap e23: 4 total, 3 up, 4 in 2024-05-04T03:03:05.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:04 smithi167 ceph-mon[33614]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:03:05.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:05.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:05.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:05.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:05.976 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:05 smithi066 ceph-mon[33264]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:03:05.977 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:05 smithi066 ceph-mon[33264]: osdmap e24: 4 total, 3 up, 4 in 2024-05-04T03:03:05.977 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:05.977 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:05.977 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:05 smithi066 ceph-mon[33264]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:05.977 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:05 smithi066 ceph-mon[33264]: osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] boot 2024-05-04T03:03:05.977 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:05 smithi066 ceph-mon[33264]: osdmap e25: 4 total, 4 up, 4 in 2024-05-04T03:03:05.977 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:06.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:05 smithi039 ceph-mon[29715]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:03:06.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:05 smithi039 ceph-mon[29715]: osdmap e24: 4 total, 3 up, 4 in 2024-05-04T03:03:06.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:06.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:06.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:05 smithi039 ceph-mon[29715]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:06.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:05 smithi039 ceph-mon[29715]: osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] boot 2024-05-04T03:03:06.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:05 smithi039 ceph-mon[29715]: osdmap e25: 4 total, 4 up, 4 in 2024-05-04T03:03:06.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:06.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:05 smithi167 ceph-mon[33614]: from='osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:03:06.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:05 smithi167 ceph-mon[33614]: osdmap e24: 4 total, 3 up, 4 in 2024-05-04T03:03:06.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:06.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:06.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:05 smithi167 ceph-mon[33614]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:03:06.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:05 smithi167 ceph-mon[33614]: osd.3 [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] boot 2024-05-04T03:03:06.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:05 smithi167 ceph-mon[33614]: osdmap e25: 4 total, 4 up, 4 in 2024-05-04T03:03:06.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:03:06.584 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-04T03:03:06.585 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-04T03:03:06.585 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-05-04T03:03:06.585 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-05-04T03:03:06.585 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011403 s, 920 MB/s 2024-05-04T03:03:06.585 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-05-04T03:03:07.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_3 2024-05-04T03:03:07.440 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:03:07.478 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:07 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:03:07.478 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:07 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:03:07.478 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:07 smithi066 ceph-mon[33264]: osdmap e26: 4 total, 4 up, 4 in 2024-05-04T03:03:07.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:07 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:03:07.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:07 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:03:07.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:07 smithi167 ceph-mon[33614]: osdmap e26: 4 total, 4 up, 4 in 2024-05-04T03:03:07.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:07 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:03:07.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:07 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:03:07.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:07 smithi039 ceph-mon[29715]: osdmap e26: 4 total, 4 up, 4 in 2024-05-04T03:03:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: osdmap e27: 4 total, 4 up, 4 in 2024-05-04T03:03:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: osdmap e27: 4 total, 4 up, 4 in 2024-05-04T03:03:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: osdmap e27: 4 total, 4 up, 4 in 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:08.729 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:08.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 -- 172.21.15.66:0/1281437545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 msgr2=0x7f70a410b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:08.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 --2- 172.21.15.66:0/1281437545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 0x7f70a410b5d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b0a0 tx=0x7f709c00b3b0 comp rx=0 tx=0).stop 2024-05-04T03:03:08.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 -- 172.21.15.66:0/1281437545 shutdown_connections 2024-05-04T03:03:08.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 --2- 172.21.15.66:0/1281437545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70a4103250 0x7f70a4103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:08.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 --2- 172.21.15.66:0/1281437545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 0x7f70a410b5d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:08.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 --2- 172.21.15.66:0/1281437545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a4104380 0x7f70a4104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:08.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 -- 172.21.15.66:0/1281437545 >> 172.21.15.66:0/1281437545 conn(0x7f70a40fe890 msgr2=0x7f70a4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:03:08.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 -- 172.21.15.66:0/1281437545 shutdown_connections 2024-05-04T03:03:08.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.846+0000 7f70a942d700 1 -- 172.21.15.66:0/1281437545 wait complete. 2024-05-04T03:03:08.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a942d700 1 Processor -- start 2024-05-04T03:03:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a942d700 1 -- start start 2024-05-04T03:03:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a942d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a4103250 0x7f70a40764b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a942d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70a4104380 0x7f70a419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a942d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 0x7f70a41a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a942d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f70a410ebe0 con 0x7f70a4104380 2024-05-04T03:03:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a942d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f70a410ea60 con 0x7f70a4104d20 2024-05-04T03:03:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a942d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70a410ed60 con 0x7f70a4103250 2024-05-04T03:03:08.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 0x7f70a41a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 0x7f70a41a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45340/0 (socket says 172.21.15.66:45340) 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a37fe700 1 -- 172.21.15.66:0/1708798750 learned_addr learned my addr 172.21.15.66:0/1708798750 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a27fc700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70a4104380 0x7f70a419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a2ffd700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a4103250 0x7f70a40764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a37fe700 1 -- 172.21.15.66:0/1708798750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a4103250 msgr2=0x7f70a40764b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a37fe700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a4103250 0x7f70a40764b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a37fe700 1 -- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70a4104380 msgr2=0x7f70a419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.847+0000 7f70a37fe700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70a4104380 0x7f70a419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.848+0000 7f70a37fe700 1 -- 172.21.15.66:0/1708798750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a41a2380 con 0x7f70a4104d20 2024-05-04T03:03:08.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.848+0000 7f70a37fe700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 0x7f70a41a1be0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f709c0062a0 tx=0x7f709c004f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:03:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.848+0000 7f708bfff700 1 -- 172.21.15.66:0/1708798750 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c01f030 con 0x7f70a4104d20 2024-05-04T03:03:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.848+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a41a2610 con 0x7f70a4104d20 2024-05-04T03:03:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.848+0000 7f708bfff700 1 -- 172.21.15.66:0/1708798750 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f709c007a40 con 0x7f70a4104d20 2024-05-04T03:03:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.848+0000 7f708bfff700 1 -- 172.21.15.66:0/1708798750 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c004710 con 0x7f70a4104d20 2024-05-04T03:03:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.848+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a41a2b10 con 0x7f70a4104d20 2024-05-04T03:03:08.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.849+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a40512c0 con 0x7f70a4104d20 2024-05-04T03:03:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.851+0000 7f708bfff700 1 -- 172.21.15.66:0/1708798750 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f709c003a00 con 0x7f70a4104d20 2024-05-04T03:03:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.851+0000 7f708bfff700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f708c0b2880 0x7f708c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.852+0000 7f708bfff700 1 -- 172.21.15.66:0/1708798750 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f709c00f020 con 0x7f70a4104d20 2024-05-04T03:03:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.852+0000 7f70a2ffd700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f708c0b2880 0x7f708c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:08.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.852+0000 7f70a2ffd700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f708c0b2880 0x7f708c0b4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70a4104280 tx=0x7f709400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:03:08.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:08.855+0000 7f708bfff700 1 -- 172.21.15.66:0/1708798750 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f709c029020 con 0x7f70a4104d20 2024-05-04T03:03:09.082 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:09.082+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f70a4063c00 con 0x7f708c0b2880 2024-05-04T03:03:09.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:09 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi039 2024-05-04T03:03:09.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:09 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi039 to 8544M 2024-05-04T03:03:09.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:09.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:09.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:09.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:09 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi039 2024-05-04T03:03:09.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:09 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi039 to 8544M 2024-05-04T03:03:09.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:09.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:09.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:09.721 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:09 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi039 2024-05-04T03:03:09.721 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:09 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi039 to 8544M 2024-05-04T03:03:09.721 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:09.721 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:09.721 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:10.536 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:10 smithi066 ceph-mon[33264]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-05-04T03:03:10.536 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:10 smithi066 ceph-mon[33264]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:03:10.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:10 smithi167 ceph-mon[33614]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-05-04T03:03:10.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:10 smithi167 ceph-mon[33614]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:03:10.728 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:10 smithi039 ceph-mon[29715]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-05-04T03:03:10.728 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:10 smithi039 ceph-mon[29715]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:03:12.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:12 smithi167 ceph-mon[33614]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:03:12.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:12 smithi167 ceph-mon[33614]: from='client.? 172.21.15.66:0/2043997366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]: dispatch 2024-05-04T03:03:12.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:12 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]: dispatch 2024-05-04T03:03:12.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:12 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]': finished 2024-05-04T03:03:12.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:12 smithi167 ceph-mon[33614]: osdmap e28: 5 total, 4 up, 5 in 2024-05-04T03:03:12.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:12 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:12.679 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:12 smithi167 ceph-mon[33614]: from='client.? 172.21.15.66:0/1192618965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:03:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:12 smithi039 ceph-mon[29715]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:03:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:12 smithi039 ceph-mon[29715]: from='client.? 172.21.15.66:0/2043997366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]: dispatch 2024-05-04T03:03:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:12 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]: dispatch 2024-05-04T03:03:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:12 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]': finished 2024-05-04T03:03:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:12 smithi039 ceph-mon[29715]: osdmap e28: 5 total, 4 up, 5 in 2024-05-04T03:03:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:12 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:12 smithi039 ceph-mon[29715]: from='client.? 172.21.15.66:0/1192618965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:03:12.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:12 smithi066 ceph-mon[33264]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:03:12.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:12 smithi066 ceph-mon[33264]: from='client.? 172.21.15.66:0/2043997366' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]: dispatch 2024-05-04T03:03:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:12 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]: dispatch 2024-05-04T03:03:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:12 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d54715-138c-4f72-837d-ecea4f0ef3e8"}]': finished 2024-05-04T03:03:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:12 smithi066 ceph-mon[33264]: osdmap e28: 5 total, 4 up, 5 in 2024-05-04T03:03:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:12 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:12 smithi066 ceph-mon[33264]: from='client.? 172.21.15.66:0/1192618965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:03:13.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:13 smithi039 ceph-mon[29715]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-04T03:03:13.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:13 smithi066 ceph-mon[33264]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-04T03:03:13.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:13 smithi167 ceph-mon[33614]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-04T03:03:16.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:15 smithi066 ceph-mon[33264]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:03:16.393 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:15 smithi039 ceph-mon[29715]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:03:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:15 smithi167 ceph-mon[33614]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:03:18.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:17 smithi066 ceph-mon[33264]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:03:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:17 smithi167 ceph-mon[33614]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:03:18.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:17 smithi039 ceph-mon[29715]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:03:20.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:19 smithi066 ceph-mon[33264]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:20.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-04T03:03:20.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:20.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:19 smithi066 ceph-mon[33264]: Deploying daemon osd.4 on smithi066 2024-05-04T03:03:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:19 smithi167 ceph-mon[33614]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-04T03:03:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:19 smithi167 ceph-mon[33614]: Deploying daemon osd.4 on smithi066 2024-05-04T03:03:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:19 smithi039 ceph-mon[29715]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-04T03:03:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:19 smithi039 ceph-mon[29715]: Deploying daemon osd.4 on smithi066 2024-05-04T03:03:22.253 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:21 smithi066 ceph-mon[33264]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:22.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:21 smithi167 ceph-mon[33614]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:22.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:21 smithi039 ceph-mon[29715]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:24.242 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:23 smithi167 ceph-mon[33614]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:23 smithi066 ceph-mon[33264]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:24.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:23 smithi039 ceph-mon[29715]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:26.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:26.179 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:26.179 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:26.179 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:25 smithi167 ceph-mon[33614]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:26.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:26.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:26.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:26.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:25 smithi039 ceph-mon[29715]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:26.248 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:26.248 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:26.248 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:26.248 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:25 smithi066 ceph-mon[33264]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:28.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:27 smithi039 ceph-mon[29715]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:28.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:28.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:28.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:27 smithi066 ceph-mon[33264]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:28.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:28.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:28.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:27 smithi167 ceph-mon[33614]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:28.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:28.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:28.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.800+0000 7f708bfff700 1 -- 172.21.15.66:0/1708798750 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f70a4063c00 con 0x7f708c0b2880 2024-05-04T03:03:28.801 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 4 on host 'smithi066' 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f708c0b2880 msgr2=0x7f708c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f708c0b2880 0x7f708c0b4d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70a4104280 tx=0x7f709400a3b0 comp rx=0 tx=0).stop 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 msgr2=0x7f70a41a1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 0x7f70a41a1be0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f709c0062a0 tx=0x7f709c004f40 comp rx=0 tx=0).stop 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 shutdown_connections 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f708c0b2880 0x7f708c0b4d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a4104d20 0x7f70a41a1be0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70a4103250 0x7f70a40764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 --2- 172.21.15.66:0/1708798750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70a4104380 0x7f70a419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 >> 172.21.15.66:0/1708798750 conn(0x7f70a40fe890 msgr2=0x7f70a41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 shutdown_connections 2024-05-04T03:03:28.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:03:28.803+0000 7f70a942d700 1 -- 172.21.15.66:0/1708798750 wait complete. 2024-05-04T03:03:29.455 DEBUG:teuthology.orchestra.run.smithi066:osd.4> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.4.service 2024-05-04T03:03:29.457 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-05-04T03:03:29.458 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-04T03:03:29.702 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:03:30.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:30.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:30.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:30.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:29 smithi167 ceph-mon[33614]: from='osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:03:30.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:29 smithi167 ceph-mon[33614]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:03:30.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:29 smithi167 ceph-mon[33614]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:30.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:30.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:30.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:30.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:29 smithi066 ceph-mon[33264]: from='osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:03:30.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:29 smithi066 ceph-mon[33264]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:03:30.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:29 smithi066 ceph-mon[33264]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:30.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:30.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:30.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:30.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:29 smithi039 ceph-mon[29715]: from='osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:03:30.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:29 smithi039 ceph-mon[29715]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:03:30.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:29 smithi039 ceph-mon[29715]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:30.891 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:30 smithi066 ceph-mon[33264]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-04T03:03:30.891 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:30 smithi066 ceph-mon[33264]: osdmap e29: 5 total, 4 up, 5 in 2024-05-04T03:03:30.891 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:30.891 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:30 smithi066 ceph-mon[33264]: from='osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:03:30.891 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:30 smithi066 ceph-mon[33264]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:03:31.113 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:30 smithi167 ceph-mon[33614]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-04T03:03:31.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:30 smithi167 ceph-mon[33614]: osdmap e29: 5 total, 4 up, 5 in 2024-05-04T03:03:31.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:31.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:30 smithi167 ceph-mon[33614]: from='osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:03:31.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:30 smithi167 ceph-mon[33614]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:03:31.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:30 smithi039 ceph-mon[29715]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-04T03:03:31.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:30 smithi039 ceph-mon[29715]: osdmap e29: 5 total, 4 up, 5 in 2024-05-04T03:03:31.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:31.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:30 smithi039 ceph-mon[29715]: from='osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:03:31.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:30 smithi039 ceph-mon[29715]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:03:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-04T03:03:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-04T03:03:31.641 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-05-04T03:03:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-05-04T03:03:31.641 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113716 s, 922 MB/s 2024-05-04T03:03:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-05-04T03:03:31.806 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:03:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4[44802]: 2024-05-04T03:03:31.343+0000 7f2298e6b700 -1 osd.4 0 waiting for initial osdmap 2024-05-04T03:03:31.806 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:03:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4[44802]: 2024-05-04T03:03:31.351+0000 7f2290c18700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:03:32.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:31 smithi066 ceph-mon[33264]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:03:32.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:31 smithi066 ceph-mon[33264]: osdmap e30: 5 total, 4 up, 5 in 2024-05-04T03:03:32.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:31 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:32.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:31 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:32.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:31 smithi066 ceph-mon[33264]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:32.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:31 smithi066 ceph-mon[33264]: from='osd.4 ' entity='osd.4' 2024-05-04T03:03:32.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:31 smithi167 ceph-mon[33614]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:03:32.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:31 smithi167 ceph-mon[33614]: osdmap e30: 5 total, 4 up, 5 in 2024-05-04T03:03:32.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:31 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:32.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:31 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:32.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:31 smithi167 ceph-mon[33614]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:32.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:31 smithi167 ceph-mon[33614]: from='osd.4 ' entity='osd.4' 2024-05-04T03:03:32.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:31 smithi039 ceph-mon[29715]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:03:32.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:31 smithi039 ceph-mon[29715]: osdmap e30: 5 total, 4 up, 5 in 2024-05-04T03:03:32.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:31 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:32.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:31 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:32.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:31 smithi039 ceph-mon[29715]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:03:32.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:31 smithi039 ceph-mon[29715]: from='osd.4 ' entity='osd.4' 2024-05-04T03:03:32.225 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-05-04T03:03:32.461 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:03:32.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:32 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:03:32.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:32 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:03:32.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:32 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:32.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:32 smithi167 ceph-mon[33614]: osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] boot 2024-05-04T03:03:32.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:32 smithi167 ceph-mon[33614]: osdmap e31: 5 total, 5 up, 5 in 2024-05-04T03:03:32.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:32 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:33.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:32 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:03:33.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:32 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:03:33.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:32 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:33.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:32 smithi066 ceph-mon[33264]: osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] boot 2024-05-04T03:03:33.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:32 smithi066 ceph-mon[33264]: osdmap e31: 5 total, 5 up, 5 in 2024-05-04T03:03:33.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:32 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:33.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:32 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:03:33.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:32 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:03:33.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:32 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:33.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:32 smithi039 ceph-mon[29715]: osd.4 [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] boot 2024-05-04T03:03:33.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:32 smithi039 ceph-mon[29715]: osdmap e31: 5 total, 5 up, 5 in 2024-05-04T03:03:33.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:32 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.029+0000 7f5a0356f700 1 -- 172.21.15.167:0/3983498132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc1059b0 msgr2=0x7f59fc105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 --2- 172.21.15.167:0/3983498132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc1059b0 0x7f59fc105dc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009990 tx=0x7f59f8009ca0 comp rx=0 tx=0).stop 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 -- 172.21.15.167:0/3983498132 shutdown_connections 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 --2- 172.21.15.167:0/3983498132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59fc1049e0 0x7f59fc10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 --2- 172.21.15.167:0/3983498132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc1059b0 0x7f59fc105dc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 --2- 172.21.15.167:0/3983498132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59fc103f60 0x7f59fc104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 -- 172.21.15.167:0/3983498132 >> 172.21.15.167:0/3983498132 conn(0x7f59fc0ff9d0 msgr2=0x7f59fc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 -- 172.21.15.167:0/3983498132 shutdown_connections 2024-05-04T03:03:34.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 -- 172.21.15.167:0/3983498132 wait complete. 2024-05-04T03:03:34.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.030+0000 7f5a0356f700 1 Processor -- start 2024-05-04T03:03:34.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0356f700 1 -- start start 2024-05-04T03:03:34.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0356f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc103f60 0x7f59fc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0356f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59fc1049e0 0x7f59fc19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0356f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59fc1059b0 0x7f59fc1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0356f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10fe50 con 0x7f59fc1049e0 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0356f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10fcd0 con 0x7f59fc1059b0 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0356f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10ffd0 con 0x7f59fc103f60 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0130b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc103f60 0x7f59fc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0130b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc103f60 0x7f59fc19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:36548/0 (socket says 172.21.15.167:36548) 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0130b700 1 -- 172.21.15.167:0/4208082401 learned_addr learned my addr 172.21.15.167:0/4208082401 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a00b0a700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59fc1049e0 0x7f59fc19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a01b0c700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59fc1059b0 0x7f59fc1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0130b700 1 -- 172.21.15.167:0/4208082401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59fc1059b0 msgr2=0x7f59fc1a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:34.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0130b700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59fc1059b0 0x7f59fc1a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0130b700 1 -- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59fc1049e0 msgr2=0x7f59fc19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0130b700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59fc1049e0 0x7f59fc19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.031+0000 7f5a0130b700 1 -- 172.21.15.167:0/4208082401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc1a2610 con 0x7f59fc103f60 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.032+0000 7f5a0130b700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc103f60 0x7f59fc19aab0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59f8003820 tx=0x7f59f80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.032+0000 7f59f27fc700 1 -- 172.21.15.167:0/4208082401 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8026070 con 0x7f59fc103f60 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.032+0000 7f59f27fc700 1 -- 172.21.15.167:0/4208082401 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59f8003f60 con 0x7f59fc103f60 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.032+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc1a28a0 con 0x7f59fc103f60 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.032+0000 7f59f27fc700 1 -- 172.21.15.167:0/4208082401 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f800fca0 con 0x7f59fc103f60 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.032+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc1a2cf0 con 0x7f59fc103f60 2024-05-04T03:03:34.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.033+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc0512c0 con 0x7f59fc103f60 2024-05-04T03:03:34.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.034+0000 7f59f27fc700 1 -- 172.21.15.167:0/4208082401 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f59f8029070 con 0x7f59fc103f60 2024-05-04T03:03:34.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi066 2024-05-04T03:03:34.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:34.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:34.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi066 to 8544M 2024-05-04T03:03:34.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:34.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:34.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:33 smithi167 ceph-mon[33614]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:34.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.035+0000 7f59f27fc700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59e80b2880 0x7f59e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:34.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.035+0000 7f59f27fc700 1 -- 172.21.15.167:0/4208082401 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f59f80e1fa0 con 0x7f59fc103f60 2024-05-04T03:03:34.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.035+0000 7f5a00b0a700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59e80b2880 0x7f59e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:34.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.035+0000 7f5a00b0a700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59e80b2880 0x7f59e80b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59fc19bb60 tx=0x7f59ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:03:34.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.041+0000 7f59f27fc700 1 -- 172.21.15.167:0/4208082401 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59f8030080 con 0x7f59fc103f60 2024-05-04T03:03:34.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi066 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi066 to 8544M 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:33 smithi039 ceph-mon[29715]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:34.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:34.255+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f59fc02ea40 con 0x7f59e80b2880 2024-05-04T03:03:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi066 2024-05-04T03:03:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi066 to 8544M 2024-05-04T03:03:34.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:34.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:34.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:34.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:33 smithi066 ceph-mon[33264]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:35.144 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:34 smithi167 ceph-mon[33614]: osdmap e32: 5 total, 5 up, 5 in 2024-05-04T03:03:35.144 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:34 smithi167 ceph-mon[33614]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:03:35.144 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:34 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:35.144 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:34 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:35.144 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:34 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:34 smithi039 ceph-mon[29715]: osdmap e32: 5 total, 5 up, 5 in 2024-05-04T03:03:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:34 smithi039 ceph-mon[29715]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:03:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:34 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:35.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:34 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:35.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:34 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:34 smithi066 ceph-mon[33264]: osdmap e32: 5 total, 5 up, 5 in 2024-05-04T03:03:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:34 smithi066 ceph-mon[33264]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:03:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:34 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:34 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:34 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:36.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:35 smithi167 ceph-mon[33614]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:35 smithi039 ceph-mon[29715]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:35 smithi066 ceph-mon[33264]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:37.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:37 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/395541028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]: dispatch 2024-05-04T03:03:37.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:37 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]: dispatch 2024-05-04T03:03:37.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:37 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]': finished 2024-05-04T03:03:37.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:37 smithi066 ceph-mon[33264]: osdmap e33: 6 total, 5 up, 6 in 2024-05-04T03:03:37.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:37 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:37 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/395541028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]: dispatch 2024-05-04T03:03:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:37 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]: dispatch 2024-05-04T03:03:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:37 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]': finished 2024-05-04T03:03:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:37 smithi167 ceph-mon[33614]: osdmap e33: 6 total, 5 up, 6 in 2024-05-04T03:03:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:37 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:37.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:37 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/395541028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]: dispatch 2024-05-04T03:03:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:37 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]: dispatch 2024-05-04T03:03:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:37 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aa01070-d35f-4964-9ccd-f956e1c9e03c"}]': finished 2024-05-04T03:03:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:37 smithi039 ceph-mon[29715]: osdmap e33: 6 total, 5 up, 6 in 2024-05-04T03:03:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:37 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:38.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:38 smithi066 ceph-mon[33264]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:38.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:38 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/1666907880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:03:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:38 smithi167 ceph-mon[33614]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:38 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/1666907880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:03:38.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:38 smithi039 ceph-mon[29715]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:38.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:38 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/1666907880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:03:40.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:40 smithi066 ceph-mon[33264]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:40.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:40 smithi167 ceph-mon[33614]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:40.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:40 smithi039 ceph-mon[29715]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:42.261 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:42 smithi167 ceph-mon[33614]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:42 smithi066 ceph-mon[33264]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:42.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:42 smithi039 ceph-mon[29715]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:44.262 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:44 smithi167 ceph-mon[33614]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:44 smithi066 ceph-mon[33264]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:44.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:44 smithi039 ceph-mon[29715]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:45.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-04T03:03:45.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:45.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:45 smithi167 ceph-mon[33614]: Deploying daemon osd.5 on smithi167 2024-05-04T03:03:45.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-04T03:03:45.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:45.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:45 smithi066 ceph-mon[33264]: Deploying daemon osd.5 on smithi167 2024-05-04T03:03:45.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-04T03:03:45.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:45.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:45 smithi039 ceph-mon[29715]: Deploying daemon osd.5 on smithi167 2024-05-04T03:03:46.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:46 smithi066 ceph-mon[33264]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:46.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:46 smithi039 ceph-mon[29715]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:46.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:46 smithi167 ceph-mon[33614]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:48.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:48 smithi167 ceph-mon[33614]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:48.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:48 smithi066 ceph-mon[33264]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:48.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:48 smithi039 ceph-mon[29715]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:50.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:50 smithi167 ceph-mon[33614]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:50.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:50 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:50.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:50 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:50.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:50 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:50 smithi066 ceph-mon[33264]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:50 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:50 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:50 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:50.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:50 smithi039 ceph-mon[29715]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:50.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:50 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:50.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:50 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:50 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:52 smithi066 ceph-mon[33264]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:52 smithi167 ceph-mon[33614]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:52.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:52 smithi039 ceph-mon[29715]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:52.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.826+0000 7f59f27fc700 1 -- 172.21.15.167:0/4208082401 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f59fc02ea40 con 0x7f59e80b2880 2024-05-04T03:03:52.827 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59e80b2880 msgr2=0x7f59e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59e80b2880 0x7f59e80b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59fc19bb60 tx=0x7f59ec00b410 comp rx=0 tx=0).stop 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc103f60 msgr2=0x7f59fc19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc103f60 0x7f59fc19aab0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59f8003820 tx=0x7f59f80047b0 comp rx=0 tx=0).stop 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 shutdown_connections 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59e80b2880 0x7f59e80b4d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59fc1059b0 0x7f59fc1a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59fc103f60 0x7f59fc19aab0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 --2- 172.21.15.167:0/4208082401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59fc1049e0 0x7f59fc19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:52.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 >> 172.21.15.167:0/4208082401 conn(0x7f59fc0ff9d0 msgr2=0x7f59fc108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:03:52.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 shutdown_connections 2024-05-04T03:03:52.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:03:52.828+0000 7f5a0356f700 1 -- 172.21.15.167:0/4208082401 wait complete. 2024-05-04T03:03:53.450 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.5.service 2024-05-04T03:03:53.452 INFO:tasks.cephadm:Deploying osd.6 on smithi039 with /dev/vg_nvme/lv_2... 2024-05-04T03:03:53.452 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-04T03:03:53.711 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:03:53.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:53.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:53.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:53.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:53.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:53 smithi167 ceph-mon[33614]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:53.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:53 smithi167 ceph-mon[33614]: from='osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:03:53.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:53 smithi167 ceph-mon[33614]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:03:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:54.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:53 smithi066 ceph-mon[33264]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:54.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:53 smithi066 ceph-mon[33264]: from='osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:03:54.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:53 smithi066 ceph-mon[33264]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:03:54.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:54.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:54.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:54.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:54.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:53 smithi039 ceph-mon[29715]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:54.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:53 smithi039 ceph-mon[29715]: from='osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:03:54.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:53 smithi039 ceph-mon[29715]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:03:55.139 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:54 smithi039 ceph-mon[29715]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-04T03:03:55.139 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:54 smithi039 ceph-mon[29715]: osdmap e34: 6 total, 5 up, 6 in 2024-05-04T03:03:55.139 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:54 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:55.139 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:54 smithi039 ceph-mon[29715]: from='osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:03:55.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:54 smithi039 ceph-mon[29715]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:03:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:54 smithi167 ceph-mon[33614]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-04T03:03:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:54 smithi167 ceph-mon[33614]: osdmap e34: 6 total, 5 up, 6 in 2024-05-04T03:03:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:54 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:55.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:54 smithi167 ceph-mon[33614]: from='osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:03:55.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:54 smithi167 ceph-mon[33614]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:03:55.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:54 smithi066 ceph-mon[33264]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-04T03:03:55.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:54 smithi066 ceph-mon[33264]: osdmap e34: 6 total, 5 up, 6 in 2024-05-04T03:03:55.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:54 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:55.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:54 smithi066 ceph-mon[33264]: from='osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:03:55.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:54 smithi066 ceph-mon[33264]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:03:55.676 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:03:55 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:03:55.341+0000 7ff2b190d700 -1 osd.5 0 waiting for initial osdmap 2024-05-04T03:03:55.676 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:03:55 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:03:55.350+0000 7ff2a96ba700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:03:55.867 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-04T03:03:55.867 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-04T03:03:55.867 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-04T03:03:55.867 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-04T03:03:55.867 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00808315 s, 1.3 GB/s 2024-05-04T03:03:55.867 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-04T03:03:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:03:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: osdmap e35: 6 total, 5 up, 6 in 2024-05-04T03:03:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:56.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: from='osd.5 ' entity='osd.5' 2024-05-04T03:03:56.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] boot 2024-05-04T03:03:56.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: osdmap e36: 6 total, 6 up, 6 in 2024-05-04T03:03:56.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:03:56.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: osdmap e35: 6 total, 5 up, 6 in 2024-05-04T03:03:56.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:56.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: from='osd.5 ' entity='osd.5' 2024-05-04T03:03:56.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] boot 2024-05-04T03:03:56.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: osdmap e36: 6 total, 6 up, 6 in 2024-05-04T03:03:56.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.218 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:03:56.218 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: osdmap e35: 6 total, 5 up, 6 in 2024-05-04T03:03:56.219 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.219 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.219 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:03:56.219 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: from='osd.5 ' entity='osd.5' 2024-05-04T03:03:56.219 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: osd.5 [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] boot 2024-05-04T03:03:56.219 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: osdmap e36: 6 total, 6 up, 6 in 2024-05-04T03:03:56.219 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:03:56.570 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_2 2024-05-04T03:03:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:03:57.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: osdmap e37: 6 total, 6 up, 6 in 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:57.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:57.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:57.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:57.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: osdmap e37: 6 total, 6 up, 6 in 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:57.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:03:57.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:03:57.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: osdmap e37: 6 total, 6 up, 6 in 2024-05-04T03:03:57.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:57.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:03:57.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:57.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:57.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:57.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:03:57.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:57.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:03:57.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 -- 172.21.15.39:0/567105495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9001044b0 msgr2=0x7fb90010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 --2- 172.21.15.39:0/567105495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9001044b0 0x7fb90010ad60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b0a0 tx=0x7fb8f800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 -- 172.21.15.39:0/567105495 shutdown_connections 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 --2- 172.21.15.39:0/567105495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9001044b0 0x7fb90010ad60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 --2- 172.21.15.39:0/567105495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9001029e0 0x7fb900102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 --2- 172.21.15.39:0/567105495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb900103b10 0x7fb900103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 -- 172.21.15.39:0/567105495 >> 172.21.15.39:0/567105495 conn(0x7fb9000fe020 msgr2=0x7fb9001004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 -- 172.21.15.39:0/567105495 shutdown_connections 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.565+0000 7fb907771700 1 -- 172.21.15.39:0/567105495 wait complete. 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb907771700 1 Processor -- start 2024-05-04T03:03:58.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb907771700 1 -- start start 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb907771700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9001029e0 0x7fb9001067a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb907771700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900103b10 0x7fb900104e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb907771700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9001044b0 0x7fb900105390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb907771700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb90010eb70 con 0x7fb900103b10 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb907771700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb90010e9f0 con 0x7fb9001044b0 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb907771700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb90010ecf0 con 0x7fb9001029e0 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb904d0c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900103b10 0x7fb900104e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb904d0c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900103b10 0x7fb900104e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37574/0 (socket says 172.21.15.39:37574) 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb904d0c700 1 -- 172.21.15.39:0/1480849526 learned_addr learned my addr 172.21.15.39:0/1480849526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb905d0e700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9001044b0 0x7fb900105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.566+0000 7fb90550d700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9001029e0 0x7fb9001067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb904d0c700 1 -- 172.21.15.39:0/1480849526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9001029e0 msgr2=0x7fb9001067a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb904d0c700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9001029e0 0x7fb9001067a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb904d0c700 1 -- 172.21.15.39:0/1480849526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9001044b0 msgr2=0x7fb900105390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:03:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb904d0c700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9001044b0 0x7fb900105390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:03:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb904d0c700 1 -- 172.21.15.39:0/1480849526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb900105af0 con 0x7fb900103b10 2024-05-04T03:03:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb904d0c700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900103b10 0x7fb900104e50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000d950 tx=0x7fb8f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:03:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb8f67fc700 1 -- 172.21.15.39:0/1480849526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0016070 con 0x7fb900103b10 2024-05-04T03:03:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9001a48b0 con 0x7fb900103b10 2024-05-04T03:03:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb8f67fc700 1 -- 172.21.15.39:0/1480849526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f000fd20 con 0x7fb900103b10 2024-05-04T03:03:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb8f67fc700 1 -- 172.21.15.39:0/1480849526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f000e3b0 con 0x7fb900103b10 2024-05-04T03:03:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.567+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9001a4de0 con 0x7fb900103b10 2024-05-04T03:03:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.568+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9000512c0 con 0x7fb900103b10 2024-05-04T03:03:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.570+0000 7fb8f67fc700 1 -- 172.21.15.39:0/1480849526 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb8f0019070 con 0x7fb900103b10 2024-05-04T03:03:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.571+0000 7fb8f67fc700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb8ec0b2880 0x7fb8ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:03:58.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.571+0000 7fb90550d700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb8ec0b2880 0x7fb8ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:03:58.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:58 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi167 2024-05-04T03:03:58.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:58 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi167 to 8544M 2024-05-04T03:03:58.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:58 smithi039 ceph-mon[29715]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:03:58.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:58 smithi039 ceph-mon[29715]: osdmap e38: 6 total, 6 up, 6 in 2024-05-04T03:03:58.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.571+0000 7fb8f67fc700 1 -- 172.21.15.39:0/1480849526 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fb8f00e1260 con 0x7fb900103b10 2024-05-04T03:03:58.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.571+0000 7fb90550d700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb8ec0b2880 0x7fb8ec0b4d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb900103a10 tx=0x7fb8fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:03:58.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.575+0000 7fb8f67fc700 1 -- 172.21.15.39:0/1480849526 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb8f00a9da0 con 0x7fb900103b10 2024-05-04T03:03:58.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:58 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi167 2024-05-04T03:03:58.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:58 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi167 to 8544M 2024-05-04T03:03:58.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:58 smithi167 ceph-mon[33614]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:03:58.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:58 smithi167 ceph-mon[33614]: osdmap e38: 6 total, 6 up, 6 in 2024-05-04T03:03:58.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:03:58.800+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb9001a4fb0 con 0x7fb8ec0b2880 2024-05-04T03:03:58.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:58 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi167 2024-05-04T03:03:58.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:58 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi167 to 8544M 2024-05-04T03:03:58.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:58 smithi066 ceph-mon[33264]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:03:58.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:58 smithi066 ceph-mon[33264]: osdmap e38: 6 total, 6 up, 6 in 2024-05-04T03:03:59.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:59.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:59.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:03:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:59.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:59.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:59.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:03:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:03:59.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:03:59.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:03:59.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:03:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:00.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:00 smithi039 ceph-mon[29715]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:00.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:00 smithi039 ceph-mon[29715]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-05-04T03:04:00.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:00 smithi066 ceph-mon[33264]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:00.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:00 smithi066 ceph-mon[33264]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-05-04T03:04:00.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:00 smithi167 ceph-mon[33614]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:00.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:00 smithi167 ceph-mon[33614]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-05-04T03:04:01.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:01 smithi039 ceph-mon[29715]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:04:01.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:01 smithi066 ceph-mon[33264]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:04:01.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:01 smithi167 ceph-mon[33614]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:04:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:02 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2274201778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e73a0efc-8e1e-42dc-9c24-f45d08af466c"}]: dispatch 2024-05-04T03:04:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:02 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2274201778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e73a0efc-8e1e-42dc-9c24-f45d08af466c"}]': finished 2024-05-04T03:04:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:02 smithi039 ceph-mon[29715]: osdmap e39: 7 total, 6 up, 7 in 2024-05-04T03:04:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:02 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3772048937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:02 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2274201778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e73a0efc-8e1e-42dc-9c24-f45d08af466c"}]: dispatch 2024-05-04T03:04:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:02 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2274201778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e73a0efc-8e1e-42dc-9c24-f45d08af466c"}]': finished 2024-05-04T03:04:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:02 smithi066 ceph-mon[33264]: osdmap e39: 7 total, 6 up, 7 in 2024-05-04T03:04:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:02 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3772048937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:02 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2274201778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e73a0efc-8e1e-42dc-9c24-f45d08af466c"}]: dispatch 2024-05-04T03:04:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:02 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2274201778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e73a0efc-8e1e-42dc-9c24-f45d08af466c"}]': finished 2024-05-04T03:04:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:02 smithi167 ceph-mon[33614]: osdmap e39: 7 total, 6 up, 7 in 2024-05-04T03:04:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:02 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3772048937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:03.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:03 smithi066 ceph-mon[33264]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-04T03:04:03.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:03 smithi167 ceph-mon[33614]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-04T03:04:03.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:03 smithi039 ceph-mon[29715]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-04T03:04:06.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:05 smithi039 ceph-mon[29715]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:04:06.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:05 smithi066 ceph-mon[33264]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:04:06.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:05 smithi167 ceph-mon[33614]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:04:08.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:07 smithi066 ceph-mon[33264]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:04:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:07 smithi167 ceph-mon[33614]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:04:08.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:07 smithi039 ceph-mon[29715]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:04:10.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:09 smithi039 ceph-mon[29715]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:10.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-04T03:04:10.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:10.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:09 smithi039 ceph-mon[29715]: Deploying daemon osd.6 on smithi039 2024-05-04T03:04:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:09 smithi066 ceph-mon[33264]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-04T03:04:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:09 smithi066 ceph-mon[33264]: Deploying daemon osd.6 on smithi039 2024-05-04T03:04:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:09 smithi167 ceph-mon[33614]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-04T03:04:10.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:10.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:09 smithi167 ceph-mon[33614]: Deploying daemon osd.6 on smithi039 2024-05-04T03:04:12.183 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:11 smithi039 ceph-mon[29715]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:12.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:11 smithi066 ceph-mon[33264]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:12.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:11 smithi167 ceph-mon[33614]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:14.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:14 smithi066 ceph-mon[33264]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:14.386 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:14 smithi039 ceph-mon[29715]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:14 smithi167 ceph-mon[33614]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:16.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:15 smithi066 ceph-mon[33264]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:16.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:16.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:16.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:15 smithi167 ceph-mon[33614]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:15 smithi039 ceph-mon[29715]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:18.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:17 smithi066 ceph-mon[33264]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:18.406 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:17 smithi039 ceph-mon[29715]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:17 smithi167 ceph-mon[33614]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.258+0000 7fb8f67fc700 1 -- 172.21.15.39:0/1480849526 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb9001a4fb0 con 0x7fb8ec0b2880 2024-05-04T03:04:19.259 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 6 on host 'smithi039' 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb8ec0b2880 msgr2=0x7fb8ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb8ec0b2880 0x7fb8ec0b4d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb900103a10 tx=0x7fb8fc00b410 comp rx=0 tx=0).stop 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900103b10 msgr2=0x7fb900104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900103b10 0x7fb900104e50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000d950 tx=0x7fb8f000dc60 comp rx=0 tx=0).stop 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 shutdown_connections 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb8ec0b2880 0x7fb8ec0b4d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9001044b0 0x7fb900105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9001029e0 0x7fb9001067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 --2- 172.21.15.39:0/1480849526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900103b10 0x7fb900104e50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.260+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 >> 172.21.15.39:0/1480849526 conn(0x7fb9000fe020 msgr2=0x7fb9000ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:04:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.261+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 shutdown_connections 2024-05-04T03:04:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:04:19.261+0000 7fb907771700 1 -- 172.21.15.39:0/1480849526 wait complete. 2024-05-04T03:04:19.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:19.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:19.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:19 smithi167 ceph-mon[33614]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-04T03:04:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:19 smithi039 ceph-mon[29715]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-04T03:04:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:19 smithi066 ceph-mon[33264]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-04T03:04:19.954 DEBUG:teuthology.orchestra.run.smithi039:osd.6> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.6.service 2024-05-04T03:04:19.956 INFO:tasks.cephadm:Deploying osd.7 on smithi066 with /dev/vg_nvme/lv_2... 2024-05-04T03:04:19.956 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-04T03:04:20.228 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:04:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:20 smithi066 ceph-mon[33264]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:20 smithi066 ceph-mon[33264]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-04T03:04:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:20 smithi066 ceph-mon[33264]: osdmap e40: 7 total, 6 up, 7 in 2024-05-04T03:04:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:20 smithi066 ceph-mon[33264]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:04:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:20.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:20.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:20 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:20.430 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:04:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:04:20.182+0000 7f2ec3c49700 -1 osd.6 0 waiting for initial osdmap 2024-05-04T03:04:20.431 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:04:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:04:20.190+0000 7f2ebf9fe700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:04:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:20 smithi039 ceph-mon[29715]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:20 smithi039 ceph-mon[29715]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-04T03:04:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:20 smithi039 ceph-mon[29715]: osdmap e40: 7 total, 6 up, 7 in 2024-05-04T03:04:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:20 smithi039 ceph-mon[29715]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:04:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:20.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:20 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:20.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:20 smithi167 ceph-mon[33614]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:20.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:20 smithi167 ceph-mon[33614]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-04T03:04:20.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:20 smithi167 ceph-mon[33614]: osdmap e40: 7 total, 6 up, 7 in 2024-05-04T03:04:20.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:20 smithi167 ceph-mon[33614]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:04:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:20 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:21.278 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:21 smithi039 ceph-mon[29715]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:04:21.278 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:21 smithi039 ceph-mon[29715]: osdmap e41: 7 total, 6 up, 7 in 2024-05-04T03:04:21.278 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:21.278 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:21 smithi066 ceph-mon[33264]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:04:21.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:21 smithi066 ceph-mon[33264]: osdmap e41: 7 total, 6 up, 7 in 2024-05-04T03:04:21.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:21.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:21.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:21 smithi167 ceph-mon[33614]: from='osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:04:21.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:21 smithi167 ceph-mon[33614]: osdmap e41: 7 total, 6 up, 7 in 2024-05-04T03:04:21.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:21.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:22.304 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-04T03:04:22.304 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-04T03:04:22.304 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-05-04T03:04:22.304 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-05-04T03:04:22.304 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114032 s, 920 MB/s 2024-05-04T03:04:22.304 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-05-04T03:04:22.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:22 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:04:22.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:22 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:04:22.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:22 smithi039 ceph-mon[29715]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:22.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:22 smithi039 ceph-mon[29715]: osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] boot 2024-05-04T03:04:22.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:22 smithi039 ceph-mon[29715]: osdmap e42: 7 total, 7 up, 7 in 2024-05-04T03:04:22.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:22 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:22 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:04:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:22 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:04:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:22 smithi066 ceph-mon[33264]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:22 smithi066 ceph-mon[33264]: osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] boot 2024-05-04T03:04:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:22 smithi066 ceph-mon[33264]: osdmap e42: 7 total, 7 up, 7 in 2024-05-04T03:04:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:22 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:22 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:04:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:22 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:04:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:22 smithi167 ceph-mon[33614]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:04:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:22 smithi167 ceph-mon[33614]: osd.6 [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] boot 2024-05-04T03:04:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:22 smithi167 ceph-mon[33614]: osdmap e42: 7 total, 7 up, 7 in 2024-05-04T03:04:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:22 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:04:22.977 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_2 2024-05-04T03:04:23.226 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:04:23.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: osdmap e43: 7 total, 7 up, 7 in 2024-05-04T03:04:23.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:23.468 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:23.468 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:23 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: osdmap e43: 7 total, 7 up, 7 in 2024-05-04T03:04:23.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:23.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:23.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:23 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: osdmap e43: 7 total, 7 up, 7 in 2024-05-04T03:04:23.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:23.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:23.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:23.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:23.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:23 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:24 smithi066 ceph-mon[33264]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:24 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi039 2024-05-04T03:04:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:24 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi039 to 5696M 2024-05-04T03:04:24.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:24 smithi167 ceph-mon[33614]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:24.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:24 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi039 2024-05-04T03:04:24.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:24 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi039 to 5696M 2024-05-04T03:04:24.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:24 smithi039 ceph-mon[29715]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:24.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:24 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi039 2024-05-04T03:04:24.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:24 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi039 to 5696M 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 -- 172.21.15.66:0/799328744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8103240 msgr2=0x7f94f8103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 --2- 172.21.15.66:0/799328744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8103240 0x7f94f8103650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009990 tx=0x7f94ec009ca0 comp rx=0 tx=0).stop 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 -- 172.21.15.66:0/799328744 shutdown_connections 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 --2- 172.21.15.66:0/799328744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94f8104d10 0x7f94f810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 --2- 172.21.15.66:0/799328744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8103240 0x7f94f8103650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 --2- 172.21.15.66:0/799328744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94f8104370 0x7f94f8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 -- 172.21.15.66:0/799328744 >> 172.21.15.66:0/799328744 conn(0x7f94f80fe880 msgr2=0x7f94f8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 -- 172.21.15.66:0/799328744 shutdown_connections 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.834+0000 7f94fe024700 1 -- 172.21.15.66:0/799328744 wait complete. 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.835+0000 7f94fe024700 1 Processor -- start 2024-05-04T03:04:24.835 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.835+0000 7f94fe024700 1 -- start start 2024-05-04T03:04:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.835+0000 7f94fe024700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94f8103240 0x7f94f819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:04:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.835+0000 7f94fe024700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94f8104370 0x7f94f819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:04:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.835+0000 7f94fe024700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8104d10 0x7f94f81a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:04:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.835+0000 7f94fe024700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94f810ec00 con 0x7f94f8103240 2024-05-04T03:04:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.835+0000 7f94fe024700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f94f810ea80 con 0x7f94f8104d10 2024-05-04T03:04:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.835+0000 7f94fe024700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94f810ed80 con 0x7f94f8104370 2024-05-04T03:04:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8104d10 0x7f94f81a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:04:24.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8104d10 0x7f94f81a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47746/0 (socket says 172.21.15.66:47746) 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 -- 172.21.15.66:0/1227065126 learned_addr learned my addr 172.21.15.66:0/1227065126 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f6ffd700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94f8104370 0x7f94f819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f77fe700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94f8103240 0x7f94f819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 -- 172.21.15.66:0/1227065126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94f8104370 msgr2=0x7f94f819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94f8104370 0x7f94f819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 -- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94f8103240 msgr2=0x7f94f819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94f8103240 0x7f94f819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 -- 172.21.15.66:0/1227065126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f81a2450 con 0x7f94f8104d10 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f7fff700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8104d10 0x7f94f81a1dd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f94f000ef30 tx=0x7f94f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f4ff9700 1 -- 172.21.15.66:0/1227065126 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f00092e0 con 0x7f94f8104d10 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f4ff9700 1 -- 172.21.15.66:0/1227065126 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94f0004bb0 con 0x7f94f8104d10 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f8105590 con 0x7f94f8104d10 2024-05-04T03:04:24.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94f4ff9700 1 -- 172.21.15.66:0/1227065126 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0004020 con 0x7f94f8104d10 2024-05-04T03:04:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.836+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f8105b50 con 0x7f94f8104d10 2024-05-04T03:04:24.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.837+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f80512c0 con 0x7f94f8104d10 2024-05-04T03:04:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.839+0000 7f94f4ff9700 1 -- 172.21.15.66:0/1227065126 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f94f001b030 con 0x7f94f8104d10 2024-05-04T03:04:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.840+0000 7f94f4ff9700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94e00b2900 0x7f94e00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:04:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.840+0000 7f94f4ff9700 1 -- 172.21.15.66:0/1227065126 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f94f00e1990 con 0x7f94f8104d10 2024-05-04T03:04:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.840+0000 7f94f77fe700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94e00b2900 0x7f94e00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:04:24.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.841+0000 7f94f77fe700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94e00b2900 0x7f94e00b4e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b2e0 tx=0x7f94ec01e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:04:24.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:24.843+0000 7f94f4ff9700 1 -- 172.21.15.66:0/1227065126 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94f0022020 con 0x7f94f8104d10 2024-05-04T03:04:25.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:25.065+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f94f81009b0 con 0x7f94e00b2900 2024-05-04T03:04:25.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:04:25.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:04:25.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:25.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:04:25.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:04:25.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:25.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:04:25.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:04:25.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:26 smithi066 ceph-mon[33264]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:26 smithi066 ceph-mon[33264]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:26.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:26 smithi167 ceph-mon[33614]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:26.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:26 smithi167 ceph-mon[33614]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:26.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:26 smithi039 ceph-mon[29715]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:26.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:26 smithi039 ceph-mon[29715]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:28 smithi066 ceph-mon[33264]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:28 smithi066 ceph-mon[33264]: from='client.? 172.21.15.66:0/123570080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]: dispatch 2024-05-04T03:04:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:28 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]: dispatch 2024-05-04T03:04:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:28 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]': finished 2024-05-04T03:04:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:28 smithi066 ceph-mon[33264]: osdmap e44: 8 total, 7 up, 8 in 2024-05-04T03:04:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:28 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:28.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:28 smithi167 ceph-mon[33614]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:28.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:28 smithi167 ceph-mon[33614]: from='client.? 172.21.15.66:0/123570080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]: dispatch 2024-05-04T03:04:28.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:28 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]: dispatch 2024-05-04T03:04:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:28 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]': finished 2024-05-04T03:04:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:28 smithi167 ceph-mon[33614]: osdmap e44: 8 total, 7 up, 8 in 2024-05-04T03:04:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:28 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:28 smithi039 ceph-mon[29715]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:28 smithi039 ceph-mon[29715]: from='client.? 172.21.15.66:0/123570080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]: dispatch 2024-05-04T03:04:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:28 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]: dispatch 2024-05-04T03:04:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:28 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56289675-217d-4d0e-9875-fb5e48254898"}]': finished 2024-05-04T03:04:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:28 smithi039 ceph-mon[29715]: osdmap e44: 8 total, 7 up, 8 in 2024-05-04T03:04:28.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:28 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:29.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:29 smithi066 ceph-mon[33264]: from='client.? 172.21.15.66:0/3506435628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:29.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:29 smithi167 ceph-mon[33614]: from='client.? 172.21.15.66:0/3506435628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:29.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:29 smithi039 ceph-mon[29715]: from='client.? 172.21.15.66:0/3506435628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:30.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:30 smithi066 ceph-mon[33264]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:30.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:30 smithi167 ceph-mon[33614]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:30 smithi039 ceph-mon[29715]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:32.416 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:32 smithi066 ceph-mon[33264]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:32.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:32 smithi167 ceph-mon[33614]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:32.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:32 smithi039 ceph-mon[29715]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:34.294 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:34 smithi066 ceph-mon[33264]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:34.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:34 smithi167 ceph-mon[33614]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:34.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:34 smithi039 ceph-mon[29715]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:36 smithi066 ceph-mon[33264]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:36 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-04T03:04:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:36 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:36 smithi066 ceph-mon[33264]: Deploying daemon osd.7 on smithi066 2024-05-04T03:04:36.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:36 smithi167 ceph-mon[33614]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:36.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:36 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-04T03:04:36.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:36 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:36.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:36 smithi167 ceph-mon[33614]: Deploying daemon osd.7 on smithi066 2024-05-04T03:04:36.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:36 smithi039 ceph-mon[29715]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:36.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:36 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-04T03:04:36.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:36 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:36.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:36 smithi039 ceph-mon[29715]: Deploying daemon osd.7 on smithi066 2024-05-04T03:04:38.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:38 smithi066 ceph-mon[33264]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:38.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:38 smithi167 ceph-mon[33614]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:38.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:38 smithi039 ceph-mon[29715]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:40.437 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:40 smithi066 ceph-mon[33264]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:40.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:40 smithi167 ceph-mon[33614]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:40 smithi039 ceph-mon[29715]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:41 smithi066 ceph-mon[33264]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:41 smithi167 ceph-mon[33614]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:42.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:41 smithi039 ceph-mon[29715]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:42.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:42.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:42.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:44.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:43 smithi066 ceph-mon[33264]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:44.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.232+0000 7f94f4ff9700 1 -- 172.21.15.66:0/1227065126 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f94f81009b0 con 0x7f94e00b2900 2024-05-04T03:04:44.232 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 7 on host 'smithi066' 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94e00b2900 msgr2=0x7f94e00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94e00b2900 0x7f94e00b4e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b2e0 tx=0x7f94ec01e040 comp rx=0 tx=0).stop 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8104d10 msgr2=0x7f94f81a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8104d10 0x7f94f81a1dd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f94f000ef30 tx=0x7f94f000c3a0 comp rx=0 tx=0).stop 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 shutdown_connections 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94e00b2900 0x7f94e00b4e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94f8103240 0x7f94f819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94f8104d10 0x7f94f81a1dd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 --2- 172.21.15.66:0/1227065126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94f8104370 0x7f94f819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 >> 172.21.15.66:0/1227065126 conn(0x7f94f80fe880 msgr2=0x7f94f81002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:04:44.235 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 shutdown_connections 2024-05-04T03:04:44.236 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:04:44.235+0000 7f94fe024700 1 -- 172.21.15.66:0/1227065126 wait complete. 2024-05-04T03:04:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:43 smithi167 ceph-mon[33614]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:44.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:43 smithi039 ceph-mon[29715]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:45.039 DEBUG:teuthology.orchestra.run.smithi066:osd.7> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.7.service 2024-05-04T03:04:45.042 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-05-04T03:04:45.042 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-04T03:04:45.290 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:04:45.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:45.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:45.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:45 smithi066 ceph-mon[33264]: from='osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:04:45.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:45 smithi066 ceph-mon[33264]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:04:45.412 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:45.412 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:45.412 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:45 smithi167 ceph-mon[33614]: from='osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:04:45.412 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:45 smithi167 ceph-mon[33614]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:04:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:45 smithi039 ceph-mon[29715]: from='osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:04:45.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:45 smithi039 ceph-mon[29715]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:04:46.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:46 smithi167 ceph-mon[33614]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:46.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:46 smithi167 ceph-mon[33614]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-04T03:04:46.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:46 smithi167 ceph-mon[33614]: from='osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:04:46.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:46 smithi167 ceph-mon[33614]: osdmap e45: 8 total, 7 up, 8 in 2024-05-04T03:04:46.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:46 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:46.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:46 smithi167 ceph-mon[33614]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:04:46.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:46 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:46.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:46 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:46 smithi066 ceph-mon[33264]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:46 smithi066 ceph-mon[33264]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-04T03:04:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:46 smithi066 ceph-mon[33264]: from='osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:04:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:46 smithi066 ceph-mon[33264]: osdmap e45: 8 total, 7 up, 8 in 2024-05-04T03:04:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:46 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:46 smithi066 ceph-mon[33264]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:04:46.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:46 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:46.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:46 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:46.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:46 smithi039 ceph-mon[29715]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:46.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:46 smithi039 ceph-mon[29715]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-04T03:04:46.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:46 smithi039 ceph-mon[29715]: from='osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:04:46.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:46 smithi039 ceph-mon[29715]: osdmap e45: 8 total, 7 up, 8 in 2024-05-04T03:04:46.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:46.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:46 smithi039 ceph-mon[29715]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:04:46.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:46.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:46.978 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:04:46 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:04:46.693+0000 7f5df654b700 -1 osd.7 0 waiting for initial osdmap 2024-05-04T03:04:46.978 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:04:46 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:04:46.703+0000 7f5dee2f8700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:04:47.154 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-04T03:04:47.154 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-04T03:04:47.155 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-05-04T03:04:47.155 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-05-04T03:04:47.155 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013728 s, 764 MB/s 2024-05-04T03:04:47.155 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-05-04T03:04:47.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:47 smithi167 ceph-mon[33614]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:04:47.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:47 smithi167 ceph-mon[33614]: osdmap e46: 8 total, 7 up, 8 in 2024-05-04T03:04:47.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:47.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:47.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:47 smithi167 ceph-mon[33614]: from='osd.7 ' entity='osd.7' 2024-05-04T03:04:47.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:47 smithi066 ceph-mon[33264]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:04:47.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:47 smithi066 ceph-mon[33264]: osdmap e46: 8 total, 7 up, 8 in 2024-05-04T03:04:47.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:47.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:47.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:47 smithi066 ceph-mon[33264]: from='osd.7 ' entity='osd.7' 2024-05-04T03:04:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:47 smithi039 ceph-mon[29715]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:04:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:47 smithi039 ceph-mon[29715]: osdmap e46: 8 total, 7 up, 8 in 2024-05-04T03:04:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:47.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:47 smithi039 ceph-mon[29715]: from='osd.7 ' entity='osd.7' 2024-05-04T03:04:47.785 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-05-04T03:04:48.026 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:04:48.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:48 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:04:48.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:48 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:04:48.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:48 smithi167 ceph-mon[33614]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:48.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:48 smithi167 ceph-mon[33614]: osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] boot 2024-05-04T03:04:48.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:48 smithi167 ceph-mon[33614]: osdmap e47: 8 total, 8 up, 8 in 2024-05-04T03:04:48.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:48 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:48.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:48 smithi167 ceph-mon[33614]: osdmap e48: 8 total, 8 up, 8 in 2024-05-04T03:04:48.536 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:48 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:04:48.536 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:48 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:04:48.537 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:48 smithi066 ceph-mon[33264]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:48.537 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:48 smithi066 ceph-mon[33264]: osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] boot 2024-05-04T03:04:48.537 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:48 smithi066 ceph-mon[33264]: osdmap e47: 8 total, 8 up, 8 in 2024-05-04T03:04:48.537 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:48 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:48.537 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:48 smithi066 ceph-mon[33264]: osdmap e48: 8 total, 8 up, 8 in 2024-05-04T03:04:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:48 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:04:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:48 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:04:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:48 smithi039 ceph-mon[29715]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:04:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:48 smithi039 ceph-mon[29715]: osd.7 [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] boot 2024-05-04T03:04:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:48 smithi039 ceph-mon[29715]: osdmap e47: 8 total, 8 up, 8 in 2024-05-04T03:04:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:48 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:04:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:48 smithi039 ceph-mon[29715]: osdmap e48: 8 total, 8 up, 8 in 2024-05-04T03:04:49.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.540+0000 7f26c1420700 1 -- 172.21.15.167:0/3833269765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc103240 msgr2=0x7f26bc103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:49.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.540+0000 7f26c1420700 1 --2- 172.21.15.167:0/3833269765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc103240 0x7f26bc103650 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009990 tx=0x7f26b0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:04:49.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.540+0000 7f26c1420700 1 -- 172.21.15.167:0/3833269765 shutdown_connections 2024-05-04T03:04:49.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.540+0000 7f26c1420700 1 --2- 172.21.15.167:0/3833269765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26bc104d10 0x7f26bc10b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:49.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.540+0000 7f26c1420700 1 --2- 172.21.15.167:0/3833269765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc103240 0x7f26bc103650 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:49.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.540+0000 7f26c1420700 1 --2- 172.21.15.167:0/3833269765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26bc104370 0x7f26bc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:49.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.540+0000 7f26c1420700 1 -- 172.21.15.167:0/3833269765 >> 172.21.15.167:0/3833269765 conn(0x7f26bc0fe880 msgr2=0x7f26bc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:04:49.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.541+0000 7f26c1420700 1 -- 172.21.15.167:0/3833269765 shutdown_connections 2024-05-04T03:04:49.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.541+0000 7f26c1420700 1 -- 172.21.15.167:0/3833269765 wait complete. 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.541+0000 7f26c1420700 1 Processor -- start 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.541+0000 7f26c1420700 1 -- start start 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26c1420700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26bc103240 0x7f26bc19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26c1420700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc104370 0x7f26bc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26c1420700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26bc104d10 0x7f26bc1a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26c1420700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc10eb30 con 0x7f26bc104d10 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26c1420700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc10e9b0 con 0x7f26bc103240 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26c1420700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc10ecb0 con 0x7f26bc104370 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc104370 0x7f26bc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:04:49.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26baffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26bc103240 0x7f26bc19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc104370 0x7f26bc19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45142/0 (socket says 172.21.15.167:45142) 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 -- 172.21.15.167:0/1093551224 learned_addr learned my addr 172.21.15.167:0/1093551224 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26bb7fe700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26bc104d10 0x7f26bc1a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 -- 172.21.15.167:0/1093551224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26bc103240 msgr2=0x7f26bc19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26bc103240 0x7f26bc19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 -- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26bc104d10 msgr2=0x7f26bc1a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26bc104d10 0x7f26bc1a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 -- 172.21.15.167:0/1093551224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc1a2450 con 0x7f26bc104370 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.542+0000 7f26ba7fc700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc104370 0x7f26bc19ae30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00d950 tx=0x7f26ac00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.543+0000 7f26a3fff700 1 -- 172.21.15.167:0/1093551224 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac016070 con 0x7f26bc104370 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.543+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc1a2740 con 0x7f26bc104370 2024-05-04T03:04:49.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.543+0000 7f26a3fff700 1 -- 172.21.15.167:0/1093551224 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26ac00fd20 con 0x7f26bc104370 2024-05-04T03:04:49.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.543+0000 7f26a3fff700 1 -- 172.21.15.167:0/1093551224 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00e3b0 con 0x7f26bc104370 2024-05-04T03:04:49.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.543+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc1a2cd0 con 0x7f26bc104370 2024-05-04T03:04:49.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.544+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26bc0512c0 con 0x7f26bc104370 2024-05-04T03:04:49.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.545+0000 7f26a3fff700 1 -- 172.21.15.167:0/1093551224 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f26ac019070 con 0x7f26bc104370 2024-05-04T03:04:49.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.546+0000 7f26a3fff700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f26a40b2880 0x7f26a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:04:49.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.546+0000 7f26a3fff700 1 -- 172.21.15.167:0/1093551224 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f26ac0e1110 con 0x7f26bc104370 2024-05-04T03:04:49.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.546+0000 7f26baffd700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f26a40b2880 0x7f26a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:04:49.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.547+0000 7f26baffd700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f26a40b2880 0x7f26a40b4d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f26b0006010 tx=0x7f26b001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:04:49.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.550+0000 7f26a3fff700 1 -- 172.21.15.167:0/1093551224 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26ac0a9990 con 0x7f26bc104370 2024-05-04T03:04:49.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:04:49.766+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f26bc06bd20 con 0x7f26a40b2880 2024-05-04T03:04:50.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi066 2024-05-04T03:04:50.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi066 to 5696M 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:50.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: osdmap e49: 8 total, 8 up, 8 in 2024-05-04T03:04:50.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:04:50.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:04:50.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi066 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi066 to 5696M 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:04:50.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:50.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:50.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:50.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: osdmap e49: 8 total, 8 up, 8 in 2024-05-04T03:04:50.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:04:50.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:04:50.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi066 2024-05-04T03:04:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi066 to 5696M 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: osdmap e49: 8 total, 8 up, 8 in 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:04:50.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:04:51.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:50 smithi039 ceph-mon[29715]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:51.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:50 smithi066 ceph-mon[33264]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:50 smithi167 ceph-mon[33614]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:04:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:51 smithi066 ceph-mon[33264]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:04:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:51 smithi167 ceph-mon[33614]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:04:52.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:51 smithi039 ceph-mon[29715]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:04:53.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:52 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/3807897351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]: dispatch 2024-05-04T03:04:53.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:52 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]: dispatch 2024-05-04T03:04:53.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:52 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]': finished 2024-05-04T03:04:53.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:52 smithi066 ceph-mon[33264]: osdmap e50: 9 total, 8 up, 9 in 2024-05-04T03:04:53.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:04:53.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:52 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/4288126290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:52 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/3807897351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]: dispatch 2024-05-04T03:04:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:52 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]: dispatch 2024-05-04T03:04:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:52 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]': finished 2024-05-04T03:04:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:52 smithi167 ceph-mon[33614]: osdmap e50: 9 total, 8 up, 9 in 2024-05-04T03:04:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:04:53.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:52 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/4288126290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:53.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:52 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/3807897351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]: dispatch 2024-05-04T03:04:53.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:52 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]: dispatch 2024-05-04T03:04:53.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:52 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "755cbc15-ebe2-408f-8009-518e6f31b975"}]': finished 2024-05-04T03:04:53.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:52 smithi039 ceph-mon[29715]: osdmap e50: 9 total, 8 up, 9 in 2024-05-04T03:04:53.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:04:53.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:52 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/4288126290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:04:54.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:54 smithi167 ceph-mon[33614]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-04T03:04:54.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:54 smithi039 ceph-mon[29715]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-04T03:04:54.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:54 smithi066 ceph-mon[33264]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-04T03:04:56.620 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:56 smithi066 ceph-mon[33264]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-04T03:04:56.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:56 smithi167 ceph-mon[33614]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-04T03:04:56.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:56 smithi039 ceph-mon[29715]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-04T03:04:58.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:04:58 smithi167 ceph-mon[33614]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:04:58.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:04:58 smithi039 ceph-mon[29715]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:04:58.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:04:58 smithi066 ceph-mon[33264]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:05:00.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:00 smithi167 ceph-mon[33614]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-04T03:05:00.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-04T03:05:00.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:00.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:00 smithi039 ceph-mon[29715]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-04T03:05:00.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:00 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-04T03:05:00.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:00 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:00.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:00 smithi066 ceph-mon[33264]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-04T03:05:00.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:00 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-04T03:05:00.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:00 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:01.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:01 smithi167 ceph-mon[33614]: Deploying daemon osd.8 on smithi167 2024-05-04T03:05:01.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:01 smithi039 ceph-mon[29715]: Deploying daemon osd.8 on smithi167 2024-05-04T03:05:01.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:01 smithi066 ceph-mon[33264]: Deploying daemon osd.8 on smithi167 2024-05-04T03:05:02.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:02 smithi167 ceph-mon[33614]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:02.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:02 smithi039 ceph-mon[29715]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:02 smithi066 ceph-mon[33264]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:04 smithi167 ceph-mon[33614]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:04.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:04 smithi039 ceph-mon[29715]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:04 smithi066 ceph-mon[33264]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:06.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:06 smithi066 ceph-mon[33264]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:06.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:06 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:06.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:06 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:06.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:06 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:06.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:06 smithi167 ceph-mon[33614]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:06.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:06.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:06.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:06 smithi039 ceph-mon[29715]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:06.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:08 smithi066 ceph-mon[33264]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:08 smithi167 ceph-mon[33614]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:08 smithi039 ceph-mon[29715]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:09.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.035+0000 7f26a3fff700 1 -- 172.21.15.167:0/1093551224 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f26bc06bd20 con 0x7f26a40b2880 2024-05-04T03:05:09.036 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f26a40b2880 msgr2=0x7f26a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f26a40b2880 0x7f26a40b4d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f26b0006010 tx=0x7f26b001a040 comp rx=0 tx=0).stop 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc104370 msgr2=0x7f26bc19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc104370 0x7f26bc19ae30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00d950 tx=0x7f26ac00dc60 comp rx=0 tx=0).stop 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 shutdown_connections 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f26a40b2880 0x7f26a40b4d90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26bc104d10 0x7f26bc1a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f26bc103240 0x7f26bc19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 --2- 172.21.15.167:0/1093551224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc104370 0x7f26bc19ae30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 >> 172.21.15.167:0/1093551224 conn(0x7f26bc0fe880 msgr2=0x7f26bc06b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 shutdown_connections 2024-05-04T03:05:09.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:05:09.037+0000 7f26c1420700 1 -- 172.21.15.167:0/1093551224 wait complete. 2024-05-04T03:05:09.641 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.8.service 2024-05-04T03:05:09.644 INFO:tasks.cephadm:Deploying osd.9 on smithi039 with /dev/vg_nvme/lv_1... 2024-05-04T03:05:09.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-04T03:05:09.905 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:05:10.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:10 smithi167 ceph-mon[33614]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:10.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:10.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:10.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:10.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:10 smithi167 ceph-mon[33614]: from='osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:05:10.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:10 smithi167 ceph-mon[33614]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:05:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:10 smithi066 ceph-mon[33264]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:10 smithi066 ceph-mon[33264]: from='osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:05:10.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:10 smithi066 ceph-mon[33264]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:05:10.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:10 smithi039 ceph-mon[29715]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:10.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:10.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:10.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:10.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:10 smithi039 ceph-mon[29715]: from='osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:05:10.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:10 smithi039 ceph-mon[29715]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:05:11.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:11 smithi167 ceph-mon[33614]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-04T03:05:11.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:11 smithi167 ceph-mon[33614]: from='osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:05:11.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:11 smithi167 ceph-mon[33614]: osdmap e51: 9 total, 8 up, 9 in 2024-05-04T03:05:11.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:11 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:11.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:11 smithi167 ceph-mon[33614]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:05:11.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:11 smithi039 ceph-mon[29715]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-04T03:05:11.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:11 smithi039 ceph-mon[29715]: from='osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:05:11.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:11 smithi039 ceph-mon[29715]: osdmap e51: 9 total, 8 up, 9 in 2024-05-04T03:05:11.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:11 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:11.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:11 smithi039 ceph-mon[29715]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:05:11.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:11 smithi066 ceph-mon[33264]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-04T03:05:11.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:11 smithi066 ceph-mon[33264]: from='osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:05:11.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:11 smithi066 ceph-mon[33264]: osdmap e51: 9 total, 8 up, 9 in 2024-05-04T03:05:11.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:11 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:11.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:11 smithi066 ceph-mon[33264]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:05:11.926 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:05:11 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:05:11.559+0000 7fd9efa7b700 -1 osd.8 0 waiting for initial osdmap 2024-05-04T03:05:11.926 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:05:11 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:05:11.571+0000 7fd9e7828700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:05:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-04T03:05:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-04T03:05:12.025 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-05-04T03:05:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-05-04T03:05:12.026 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00719275 s, 1.5 GB/s 2024-05-04T03:05:12.026 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-05-04T03:05:12.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:12 smithi066 ceph-mon[33264]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:12.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:12 smithi066 ceph-mon[33264]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:05:12.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:12 smithi066 ceph-mon[33264]: osdmap e52: 9 total, 8 up, 9 in 2024-05-04T03:05:12.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:12 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:12.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:12 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:12.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:12 smithi066 ceph-mon[33264]: from='osd.8 ' entity='osd.8' 2024-05-04T03:05:12.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:12 smithi039 ceph-mon[29715]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:12.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:12 smithi039 ceph-mon[29715]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:05:12.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:12 smithi039 ceph-mon[29715]: osdmap e52: 9 total, 8 up, 9 in 2024-05-04T03:05:12.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:12 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:12.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:12 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:12.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:12 smithi039 ceph-mon[29715]: from='osd.8 ' entity='osd.8' 2024-05-04T03:05:12.410 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:12 smithi167 ceph-mon[33614]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:05:12.411 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:12 smithi167 ceph-mon[33614]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:05:12.411 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:12 smithi167 ceph-mon[33614]: osdmap e52: 9 total, 8 up, 9 in 2024-05-04T03:05:12.411 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:12 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:12.411 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:12 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:12.411 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:12 smithi167 ceph-mon[33614]: from='osd.8 ' entity='osd.8' 2024-05-04T03:05:12.638 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_1 2024-05-04T03:05:12.892 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:05:13.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:05:13.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:05:13.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] boot 2024-05-04T03:05:13.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: osdmap e53: 9 total, 9 up, 9 in 2024-05-04T03:05:13.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:13.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:13.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:13.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:05:13.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:05:13.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] boot 2024-05-04T03:05:13.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: osdmap e53: 9 total, 9 up, 9 in 2024-05-04T03:05:13.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:13.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:05:13.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: osd.8 [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] boot 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: osdmap e53: 9 total, 9 up, 9 in 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:14.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:14 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi167 2024-05-04T03:05:14.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:14 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi167 to 5696M 2024-05-04T03:05:14.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:14 smithi039 ceph-mon[29715]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:14.300 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:14 smithi039 ceph-mon[29715]: osdmap e54: 9 total, 9 up, 9 in 2024-05-04T03:05:14.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:14 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi167 2024-05-04T03:05:14.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:14 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi167 to 5696M 2024-05-04T03:05:14.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:14 smithi066 ceph-mon[33264]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:14.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:14 smithi066 ceph-mon[33264]: osdmap e54: 9 total, 9 up, 9 in 2024-05-04T03:05:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:14 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi167 2024-05-04T03:05:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:14 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi167 to 5696M 2024-05-04T03:05:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:14 smithi167 ceph-mon[33614]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:14 smithi167 ceph-mon[33614]: osdmap e54: 9 total, 9 up, 9 in 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 -- 172.21.15.39:0/4015311286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c10a0f0 msgr2=0x7f3c4c10c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 --2- 172.21.15.39:0/4015311286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c10a0f0 0x7f3c4c10c510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400b0a0 tx=0x7f3c4400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 -- 172.21.15.39:0/4015311286 shutdown_connections 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 --2- 172.21.15.39:0/4015311286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c10a0f0 0x7f3c4c10c510 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 --2- 172.21.15.39:0/4015311286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c4c104e30 0x7f3c4c107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 --2- 172.21.15.39:0/4015311286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c4c107790 0x7f3c4c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 -- 172.21.15.39:0/4015311286 >> 172.21.15.39:0/4015311286 conn(0x7f3c4c0fe870 msgr2=0x7f3c4c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 -- 172.21.15.39:0/4015311286 shutdown_connections 2024-05-04T03:05:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.541+0000 7f3c53546700 1 -- 172.21.15.39:0/4015311286 wait complete. 2024-05-04T03:05:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c53546700 1 Processor -- start 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c53546700 1 -- start start 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c53546700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c4c104e30 0x7f3c4c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c53546700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c107790 0x7f3c4c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c53546700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c4c10a0f0 0x7f3c4c1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c53546700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c10fc40 con 0x7f3c4c107790 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c53546700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c10fac0 con 0x7f3c4c104e30 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c53546700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c10fdc0 con 0x7f3c4c10a0f0 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c50ae1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c107790 0x7f3c4c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c50ae1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c107790 0x7f3c4c19acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53718/0 (socket says 172.21.15.39:53718) 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.542+0000 7f3c50ae1700 1 -- 172.21.15.39:0/1715662605 learned_addr learned my addr 172.21.15.39:0/1715662605 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c512e2700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c4c104e30 0x7f3c4c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c51ae3700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c4c10a0f0 0x7f3c4c1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:05:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c50ae1700 1 -- 172.21.15.39:0/1715662605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c4c10a0f0 msgr2=0x7f3c4c1a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c50ae1700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c4c10a0f0 0x7f3c4c1a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c50ae1700 1 -- 172.21.15.39:0/1715662605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c4c104e30 msgr2=0x7f3c4c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c50ae1700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c4c104e30 0x7f3c4c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c50ae1700 1 -- 172.21.15.39:0/1715662605 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c1a22e0 con 0x7f3c4c107790 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c50ae1700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c107790 0x7f3c4c19acc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00d950 tx=0x7f3c3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c427fc700 1 -- 172.21.15.39:0/1715662605 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c016070 con 0x7f3c4c107790 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c427fc700 1 -- 172.21.15.39:0/1715662605 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c3c00fd20 con 0x7f3c4c107790 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c102c80 con 0x7f3c4c107790 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c427fc700 1 -- 172.21.15.39:0/1715662605 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c00e3b0 con 0x7f3c4c107790 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.543+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c1031c0 con 0x7f3c4c107790 2024-05-04T03:05:14.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.544+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c0512c0 con 0x7f3c4c107790 2024-05-04T03:05:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.546+0000 7f3c427fc700 1 -- 172.21.15.39:0/1715662605 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3c3c019070 con 0x7f3c4c107790 2024-05-04T03:05:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.547+0000 7f3c427fc700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c380b2880 0x7f3c380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:05:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.547+0000 7f3c512e2700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c380b2880 0x7f3c380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:05:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.547+0000 7f3c427fc700 1 -- 172.21.15.39:0/1715662605 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f3c3c0e1270 con 0x7f3c4c107790 2024-05-04T03:05:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.547+0000 7f3c512e2700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c380b2880 0x7f3c380b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c06ca60 tx=0x7f3c480073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:05:14.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.550+0000 7f3c427fc700 1 -- 172.21.15.39:0/1715662605 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c3c0a9940 con 0x7f3c4c107790 2024-05-04T03:05:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:14.774+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3c4c063c00 con 0x7f3c380b2880 2024-05-04T03:05:15.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:15 smithi039 ceph-mon[29715]: osdmap e55: 9 total, 9 up, 9 in 2024-05-04T03:05:15.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:05:15.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:05:15.419 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:15.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:15 smithi167 ceph-mon[33614]: osdmap e55: 9 total, 9 up, 9 in 2024-05-04T03:05:15.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:05:15.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:05:15.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:15 smithi066 ceph-mon[33264]: osdmap e55: 9 total, 9 up, 9 in 2024-05-04T03:05:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:05:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:05:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:16 smithi167 ceph-mon[33614]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:05:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:16 smithi167 ceph-mon[33614]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:16 smithi039 ceph-mon[29715]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:05:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:16 smithi039 ceph-mon[29715]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:16 smithi066 ceph-mon[33264]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:05:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:16 smithi066 ceph-mon[33264]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:18.116 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:18 smithi039 ceph-mon[29715]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:18.116 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:18 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2114607099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956d274d-166b-4474-92d3-d31bda097f21"}]: dispatch 2024-05-04T03:05:18.116 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:18 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2114607099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "956d274d-166b-4474-92d3-d31bda097f21"}]': finished 2024-05-04T03:05:18.116 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:18 smithi039 ceph-mon[29715]: osdmap e56: 10 total, 9 up, 10 in 2024-05-04T03:05:18.117 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:18 smithi167 ceph-mon[33614]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:18 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2114607099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956d274d-166b-4474-92d3-d31bda097f21"}]: dispatch 2024-05-04T03:05:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:18 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2114607099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "956d274d-166b-4474-92d3-d31bda097f21"}]': finished 2024-05-04T03:05:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:18 smithi167 ceph-mon[33614]: osdmap e56: 10 total, 9 up, 10 in 2024-05-04T03:05:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:18 smithi066 ceph-mon[33264]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:18 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2114607099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "956d274d-166b-4474-92d3-d31bda097f21"}]: dispatch 2024-05-04T03:05:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:18 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2114607099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "956d274d-166b-4474-92d3-d31bda097f21"}]': finished 2024-05-04T03:05:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:18 smithi066 ceph-mon[33264]: osdmap e56: 10 total, 9 up, 10 in 2024-05-04T03:05:18.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:19.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:19 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2687867918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:05:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:19 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2687867918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:05:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:19 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2687867918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:05:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:20 smithi167 ceph-mon[33614]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:05:20.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:20 smithi039 ceph-mon[29715]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:05:20.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:20 smithi066 ceph-mon[33264]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:05:22.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:22 smithi039 ceph-mon[29715]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-04T03:05:22.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:22 smithi167 ceph-mon[33614]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-04T03:05:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:22 smithi066 ceph-mon[33264]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-04T03:05:24.385 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:24 smithi039 ceph-mon[29715]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-04T03:05:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:24 smithi167 ceph-mon[33614]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-04T03:05:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:24 smithi066 ceph-mon[33264]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-04T03:05:26.369 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:26 smithi066 ceph-mon[33264]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:26.369 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:26 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-04T03:05:26.369 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:26 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:26.399 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:26 smithi039 ceph-mon[29715]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:26.399 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-04T03:05:26.399 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:26 smithi167 ceph-mon[33614]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:26.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:26 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-04T03:05:26.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:26 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:27.383 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:27 smithi039 ceph-mon[29715]: Deploying daemon osd.9 on smithi039 2024-05-04T03:05:27.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:27 smithi167 ceph-mon[33614]: Deploying daemon osd.9 on smithi039 2024-05-04T03:05:27.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:27 smithi066 ceph-mon[33264]: Deploying daemon osd.9 on smithi039 2024-05-04T03:05:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:28 smithi167 ceph-mon[33614]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:28.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:28 smithi039 ceph-mon[29715]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:28.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:28 smithi066 ceph-mon[33264]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:05:30.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:30 smithi039 ceph-mon[29715]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:30.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:30 smithi167 ceph-mon[33614]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:30.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:30 smithi066 ceph-mon[33264]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:32.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:32 smithi167 ceph-mon[33614]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:32.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:32 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:32.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:32 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:32.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:32 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:32.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:32 smithi039 ceph-mon[29715]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:32.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:32 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:32.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:32 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:32.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:32 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:32.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:32 smithi066 ceph-mon[33264]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:32.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:32 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:32.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:32 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:32.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:32 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:34.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:34 smithi039 ceph-mon[29715]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:34.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:34 smithi167 ceph-mon[33614]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:34.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:34 smithi066 ceph-mon[33264]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:35.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.905+0000 7f3c427fc700 1 -- 172.21.15.39:0/1715662605 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3c4c063c00 con 0x7f3c380b2880 2024-05-04T03:05:35.906 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 9 on host 'smithi039' 2024-05-04T03:05:35.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c380b2880 msgr2=0x7f3c380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:35.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c380b2880 0x7f3c380b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c06ca60 tx=0x7f3c480073d0 comp rx=0 tx=0).stop 2024-05-04T03:05:35.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c107790 msgr2=0x7f3c4c19acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:35.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c107790 0x7f3c4c19acc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00d950 tx=0x7f3c3c00dc60 comp rx=0 tx=0).stop 2024-05-04T03:05:35.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 shutdown_connections 2024-05-04T03:05:35.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c380b2880 0x7f3c380b4d90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:35.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c4c10a0f0 0x7f3c4c1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:35.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c4c104e30 0x7f3c4c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:35.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 --2- 172.21.15.39:0/1715662605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c4c107790 0x7f3c4c19acc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:35.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 >> 172.21.15.39:0/1715662605 conn(0x7f3c4c0fe870 msgr2=0x7f3c4c105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:05:35.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 shutdown_connections 2024-05-04T03:05:35.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:05:35.908+0000 7f3c53546700 1 -- 172.21.15.39:0/1715662605 wait complete. 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:35 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:35 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:36.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-04T03:05:36.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:36.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:35 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:36.731 DEBUG:teuthology.orchestra.run.smithi039:osd.9> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.9.service 2024-05-04T03:05:36.734 INFO:tasks.cephadm:Deploying osd.10 on smithi066 with /dev/vg_nvme/lv_1... 2024-05-04T03:05:36.734 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-04T03:05:36.991 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:05:36.999 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:36 smithi039 ceph-mon[29715]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-04T03:05:36.999 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:36 smithi039 ceph-mon[29715]: osdmap e57: 10 total, 9 up, 10 in 2024-05-04T03:05:36.999 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:36 smithi039 ceph-mon[29715]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:05:37.000 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:36 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:37.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:36 smithi066 ceph-mon[33264]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-04T03:05:37.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:36 smithi066 ceph-mon[33264]: osdmap e57: 10 total, 9 up, 10 in 2024-05-04T03:05:37.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:36 smithi066 ceph-mon[33264]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:05:37.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:36 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:37.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:36 smithi167 ceph-mon[33614]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-04T03:05:37.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:36 smithi167 ceph-mon[33614]: osdmap e57: 10 total, 9 up, 10 in 2024-05-04T03:05:37.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:36 smithi167 ceph-mon[33614]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-05-04T03:05:37.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:36 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:37.690 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:05:37 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:05:37.369+0000 7fab5acf5700 -1 osd.9 0 waiting for initial osdmap 2024-05-04T03:05:37.691 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:05:37 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:05:37.384+0000 7fab52aa2700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:05:38.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:37 smithi039 ceph-mon[29715]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:05:38.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:37 smithi039 ceph-mon[29715]: osdmap e58: 10 total, 9 up, 10 in 2024-05-04T03:05:38.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:37 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:38.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:37 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:38.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:37 smithi039 ceph-mon[29715]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:38.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:37 smithi039 ceph-mon[29715]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' 2024-05-04T03:05:38.281 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:37 smithi066 ceph-mon[33264]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:05:38.281 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:37 smithi066 ceph-mon[33264]: osdmap e58: 10 total, 9 up, 10 in 2024-05-04T03:05:38.281 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:37 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:38.281 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:37 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:38.281 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:37 smithi066 ceph-mon[33264]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:38.281 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:37 smithi066 ceph-mon[33264]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' 2024-05-04T03:05:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:37 smithi167 ceph-mon[33614]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-05-04T03:05:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:37 smithi167 ceph-mon[33614]: osdmap e58: 10 total, 9 up, 10 in 2024-05-04T03:05:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:37 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:37 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:38.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:37 smithi167 ceph-mon[33614]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:05:38.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:37 smithi167 ceph-mon[33614]: from='osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283]' entity='osd.9' 2024-05-04T03:05:39.006 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-04T03:05:39.006 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-04T03:05:39.006 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-05-04T03:05:39.006 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-05-04T03:05:39.006 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113196 s, 926 MB/s 2024-05-04T03:05:39.007 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-05-04T03:05:39.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:38 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:05:39.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:38 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:05:39.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:39.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:38 smithi039 ceph-mon[29715]: osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] boot 2024-05-04T03:05:39.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:38 smithi039 ceph-mon[29715]: osdmap e59: 10 total, 10 up, 10 in 2024-05-04T03:05:39.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:38 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:39.314 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:38 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:05:39.315 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:38 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:05:39.315 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:39.315 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:38 smithi066 ceph-mon[33264]: osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] boot 2024-05-04T03:05:39.315 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:38 smithi066 ceph-mon[33264]: osdmap e59: 10 total, 10 up, 10 in 2024-05-04T03:05:39.315 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:38 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:39.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:38 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:05:39.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:38 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:05:39.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:39.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:38 smithi167 ceph-mon[33614]: osd.9 [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] boot 2024-05-04T03:05:39.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:38 smithi167 ceph-mon[33614]: osdmap e59: 10 total, 10 up, 10 in 2024-05-04T03:05:39.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:38 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:05:39.696 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_1 2024-05-04T03:05:39.949 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:05:40.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:40.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: osdmap e60: 10 total, 10 up, 10 in 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: osdmap e60: 10 total, 10 up, 10 in 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:40.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: osdmap e60: 10 total, 10 up, 10 in 2024-05-04T03:05:40.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:41.453 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:41 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi039 2024-05-04T03:05:41.453 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:41 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi039 to 4272M 2024-05-04T03:05:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 -- 172.21.15.66:0/3070706411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe778104190 msgr2=0x7fe778104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 --2- 172.21.15.66:0/3070706411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe778104190 0x7fe778104660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009990 tx=0x7fe76c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:05:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 -- 172.21.15.66:0/3070706411 shutdown_connections 2024-05-04T03:05:41.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 --2- 172.21.15.66:0/3070706411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7781037b0 0x7fe778103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 --2- 172.21.15.66:0/3070706411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe778104cf0 0x7fe77810b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 --2- 172.21.15.66:0/3070706411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe778104190 0x7fe778104660 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 -- 172.21.15.66:0/3070706411 >> 172.21.15.66:0/3070706411 conn(0x7fe7780fe020 msgr2=0x7fe7781004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:05:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 -- 172.21.15.66:0/3070706411 shutdown_connections 2024-05-04T03:05:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 -- 172.21.15.66:0/3070706411 wait complete. 2024-05-04T03:05:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 Processor -- start 2024-05-04T03:05:41.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.675+0000 7fe77e774700 1 -- start start 2024-05-04T03:05:41.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:41 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi039 2024-05-04T03:05:41.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:41 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi039 to 4272M 2024-05-04T03:05:41.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe77e774700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7781037b0 0x7fe778102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:05:41.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe77e774700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe778104190 0x7fe7781a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:05:41.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe77e774700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe778104cf0 0x7fe7781a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe77e774700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe77810ebf0 con 0x7fe778104cf0 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe77e774700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe77810ea70 con 0x7fe7781037b0 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe77e774700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe77810ed70 con 0x7fe778104190 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe777fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7781037b0 0x7fe778102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe777fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7781037b0 0x7fe778102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35816/0 (socket says 172.21.15.66:35816) 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe777fff700 1 -- 172.21.15.66:0/709935342 learned_addr learned my addr 172.21.15.66:0/709935342 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe7777fe700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe778104190 0x7fe7781a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe77cd11700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe778104cf0 0x7fe7781a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe777fff700 1 -- 172.21.15.66:0/709935342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe778104190 msgr2=0x7fe7781a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe777fff700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe778104190 0x7fe7781a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe777fff700 1 -- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe778104cf0 msgr2=0x7fe7781a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe777fff700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe778104cf0 0x7fe7781a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe777fff700 1 -- 172.21.15.66:0/709935342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7781a4640 con 0x7fe7781037b0 2024-05-04T03:05:41.680 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.676+0000 7fe7777fe700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe778104190 0x7fe7781a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:05:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.677+0000 7fe777fff700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7781037b0 0x7fe778102e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe76800d950 tx=0x7fe76800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:05:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.677+0000 7fe7757fa700 1 -- 172.21.15.66:0/709935342 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768017070 con 0x7fe7781037b0 2024-05-04T03:05:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.677+0000 7fe7757fa700 1 -- 172.21.15.66:0/709935342 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe76800de30 con 0x7fe7781037b0 2024-05-04T03:05:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.677+0000 7fe7757fa700 1 -- 172.21.15.66:0/709935342 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76800f3c0 con 0x7fe7781037b0 2024-05-04T03:05:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.677+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7781a4930 con 0x7fe7781037b0 2024-05-04T03:05:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.677+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7781a4e70 con 0x7fe7781037b0 2024-05-04T03:05:41.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.678+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7780512c0 con 0x7fe7781037b0 2024-05-04T03:05:41.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.680+0000 7fe7757fa700 1 -- 172.21.15.66:0/709935342 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe76801a070 con 0x7fe7781037b0 2024-05-04T03:05:41.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.680+0000 7fe7757fa700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe7600b2880 0x7fe7600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:05:41.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.680+0000 7fe7757fa700 1 -- 172.21.15.66:0/709935342 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fe7680e1320 con 0x7fe7781037b0 2024-05-04T03:05:41.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.680+0000 7fe7777fe700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe7600b2880 0x7fe7600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:05:41.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.681+0000 7fe7777fe700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe7600b2880 0x7fe7600b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe76c006010 tx=0x7fe76c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:05:41.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.684+0000 7fe7757fa700 1 -- 172.21.15.66:0/709935342 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7680a9840 con 0x7fe7781037b0 2024-05-04T03:05:41.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:41 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi039 2024-05-04T03:05:41.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:41 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi039 to 4272M 2024-05-04T03:05:41.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:05:41.909+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe778063c00 con 0x7fe7600b2880 2024-05-04T03:05:42.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:42 smithi167 ceph-mon[33614]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:42.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:05:42.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:05:42.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:42.690 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:42 smithi066 ceph-mon[33264]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:42.690 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:05:42.690 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:05:42.690 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:42.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:42 smithi039 ceph-mon[29715]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:42.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:05:42.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:05:42.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:43.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:43 smithi167 ceph-mon[33614]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:05:43.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:43 smithi039 ceph-mon[29715]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:05:43.809 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:43 smithi066 ceph-mon[33264]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:05:44.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:44 smithi167 ceph-mon[33614]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:44.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:44 smithi039 ceph-mon[29715]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:44.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:44 smithi066 ceph-mon[33264]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:45.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:45 smithi066 ceph-mon[33264]: from='client.? 172.21.15.66:0/1427626103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]: dispatch 2024-05-04T03:05:45.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:45 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]: dispatch 2024-05-04T03:05:45.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:45 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]': finished 2024-05-04T03:05:45.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:45 smithi066 ceph-mon[33264]: osdmap e61: 11 total, 10 up, 11 in 2024-05-04T03:05:45.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:05:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:45 smithi167 ceph-mon[33614]: from='client.? 172.21.15.66:0/1427626103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]: dispatch 2024-05-04T03:05:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:45 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]: dispatch 2024-05-04T03:05:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:45 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]': finished 2024-05-04T03:05:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:45 smithi167 ceph-mon[33614]: osdmap e61: 11 total, 10 up, 11 in 2024-05-04T03:05:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:05:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:45 smithi039 ceph-mon[29715]: from='client.? 172.21.15.66:0/1427626103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]: dispatch 2024-05-04T03:05:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:45 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]: dispatch 2024-05-04T03:05:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:45 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1ff6cd-7475-4b79-83bb-1f671fe132cf"}]': finished 2024-05-04T03:05:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:45 smithi039 ceph-mon[29715]: osdmap e61: 11 total, 10 up, 11 in 2024-05-04T03:05:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:05:46.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:46 smithi167 ceph-mon[33614]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:46.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:46 smithi167 ceph-mon[33614]: from='client.? 172.21.15.66:0/1935285918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:05:46.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:46 smithi039 ceph-mon[29715]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:46.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:46 smithi039 ceph-mon[29715]: from='client.? 172.21.15.66:0/1935285918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:05:46.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:46 smithi066 ceph-mon[33264]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:46.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:46 smithi066 ceph-mon[33264]: from='client.? 172.21.15.66:0/1935285918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:05:48.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:48 smithi167 ceph-mon[33614]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:48.678 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:48 smithi066 ceph-mon[33264]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:48.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:48 smithi039 ceph-mon[29715]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:50.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:50 smithi167 ceph-mon[33614]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:50.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:50 smithi039 ceph-mon[29715]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:50.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:50 smithi066 ceph-mon[33264]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:52.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:52 smithi167 ceph-mon[33614]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:52.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:52 smithi039 ceph-mon[29715]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:52.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:52 smithi066 ceph-mon[33264]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:53.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-04T03:05:53.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:53.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:53 smithi167 ceph-mon[33614]: Deploying daemon osd.10 on smithi066 2024-05-04T03:05:53.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-04T03:05:53.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:53.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:53 smithi039 ceph-mon[29715]: Deploying daemon osd.10 on smithi066 2024-05-04T03:05:53.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-04T03:05:53.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:05:53.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:53 smithi066 ceph-mon[33264]: Deploying daemon osd.10 on smithi066 2024-05-04T03:05:54.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:54 smithi167 ceph-mon[33614]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:54.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:54 smithi039 ceph-mon[29715]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:54.727 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:54 smithi066 ceph-mon[33264]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:56.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:56 smithi039 ceph-mon[29715]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:56.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:56 smithi066 ceph-mon[33264]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:56.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:56 smithi167 ceph-mon[33614]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:58.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:58 smithi066 ceph-mon[33264]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:58.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:58.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:58.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:05:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:58.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:58 smithi039 ceph-mon[29715]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:58.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:58.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:58.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:05:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:05:58.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:58 smithi167 ceph-mon[33614]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:05:58.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:58.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:05:58.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:05:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:00.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:00 smithi039 ceph-mon[29715]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:00.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:00 smithi066 ceph-mon[33264]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:00.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:00 smithi167 ceph-mon[33614]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:02.330 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 10 on host 'smithi066' 2024-05-04T03:06:02.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.329+0000 7fe7757fa700 1 -- 172.21.15.66:0/709935342 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe778063c00 con 0x7fe7600b2880 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe7600b2880 msgr2=0x7fe7600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe7600b2880 0x7fe7600b4d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe76c006010 tx=0x7fe76c01a040 comp rx=0 tx=0).stop 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7781037b0 msgr2=0x7fe778102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7781037b0 0x7fe778102e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe76800d950 tx=0x7fe76800dd10 comp rx=0 tx=0).stop 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 shutdown_connections 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe7600b2880 0x7fe7600b4d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe778104cf0 0x7fe7781a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7781037b0 0x7fe778102e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 --2- 172.21.15.66:0/709935342 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe778104190 0x7fe7781a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:02.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 >> 172.21.15.66:0/709935342 conn(0x7fe7780fe020 msgr2=0x7fe778107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:02.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 shutdown_connections 2024-05-04T03:06:02.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-05-04T03:06:02.332+0000 7fe77e774700 1 -- 172.21.15.66:0/709935342 wait complete. 2024-05-04T03:06:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:06:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:06:02.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:02.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:02.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:06:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:06:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:02.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:02.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:02.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:06:02.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:06:02.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:02.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:02.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:03.028 DEBUG:teuthology.orchestra.run.smithi066:osd.10> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.10.service 2024-05-04T03:06:03.030 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-05-04T03:06:03.031 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-04T03:06:03.279 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:06:03.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:03 smithi167 ceph-mon[33614]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-04T03:06:03.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:03 smithi167 ceph-mon[33614]: from='osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:06:03.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:03 smithi167 ceph-mon[33614]: osdmap e62: 11 total, 10 up, 11 in 2024-05-04T03:06:03.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:03.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:03 smithi167 ceph-mon[33614]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:06:03.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:03 smithi167 ceph-mon[33614]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:03.905 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:03 smithi066 ceph-mon[33264]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-04T03:06:03.905 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:03 smithi066 ceph-mon[33264]: from='osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:06:03.905 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:03 smithi066 ceph-mon[33264]: osdmap e62: 11 total, 10 up, 11 in 2024-05-04T03:06:03.905 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:03.905 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:03 smithi066 ceph-mon[33264]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:06:03.905 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:03 smithi066 ceph-mon[33264]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:03.906 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:06:03 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:06:03.603+0000 7f011df39700 -1 osd.10 0 waiting for initial osdmap 2024-05-04T03:06:03.906 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:06:03 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:06:03.615+0000 7f0119cee700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:06:03.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:03 smithi039 ceph-mon[29715]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-04T03:06:03.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:03 smithi039 ceph-mon[29715]: from='osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:06:03.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:03 smithi039 ceph-mon[29715]: osdmap e62: 11 total, 10 up, 11 in 2024-05-04T03:06:03.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:03.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:03 smithi039 ceph-mon[29715]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-05-04T03:06:03.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:03 smithi039 ceph-mon[29715]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:04 smithi167 ceph-mon[33614]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:06:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:04 smithi167 ceph-mon[33614]: osdmap e63: 11 total, 10 up, 11 in 2024-05-04T03:06:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:04 smithi066 ceph-mon[33264]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:06:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:04 smithi066 ceph-mon[33264]: osdmap e63: 11 total, 10 up, 11 in 2024-05-04T03:06:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:04.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:04 smithi039 ceph-mon[29715]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-05-04T03:06:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:04 smithi039 ceph-mon[29715]: osdmap e63: 11 total, 10 up, 11 in 2024-05-04T03:06:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:05.266 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-04T03:06:05.266 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-04T03:06:05.266 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-05-04T03:06:05.266 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-05-04T03:06:05.266 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112862 s, 929 MB/s 2024-05-04T03:06:05.266 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-05-04T03:06:05.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:05 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:06:05.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:05 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:06:05.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:05 smithi167 ceph-mon[33614]: osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] boot 2024-05-04T03:06:05.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:05 smithi167 ceph-mon[33614]: osdmap e64: 11 total, 11 up, 11 in 2024-05-04T03:06:05.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:05.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:05 smithi167 ceph-mon[33614]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:05.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:05 smithi167 ceph-mon[33614]: osdmap e65: 11 total, 11 up, 11 in 2024-05-04T03:06:05.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:05 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:06:05.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:05 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:06:05.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:05 smithi039 ceph-mon[29715]: osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] boot 2024-05-04T03:06:05.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:05 smithi039 ceph-mon[29715]: osdmap e64: 11 total, 11 up, 11 in 2024-05-04T03:06:05.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:05.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:05 smithi039 ceph-mon[29715]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:05.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:05 smithi039 ceph-mon[29715]: osdmap e65: 11 total, 11 up, 11 in 2024-05-04T03:06:05.963 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-05-04T03:06:06.022 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:05 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:06:06.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:05 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:06:06.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:05 smithi066 ceph-mon[33264]: osd.10 [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] boot 2024-05-04T03:06:06.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:05 smithi066 ceph-mon[33264]: osdmap e64: 11 total, 11 up, 11 in 2024-05-04T03:06:06.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:06:06.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:05 smithi066 ceph-mon[33264]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:06:06.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:05 smithi066 ceph-mon[33264]: osdmap e65: 11 total, 11 up, 11 in 2024-05-04T03:06:06.204 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:06:07.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi066 2024-05-04T03:06:07.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi066 to 4272M 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:07 smithi167 ceph-mon[33614]: osdmap e66: 11 total, 11 up, 11 in 2024-05-04T03:06:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi066 2024-05-04T03:06:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi066 to 4272M 2024-05-04T03:06:07.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:07.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:07.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:07 smithi039 ceph-mon[29715]: osdmap e66: 11 total, 11 up, 11 in 2024-05-04T03:06:07.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi066 2024-05-04T03:06:07.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:07.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi066 to 4272M 2024-05-04T03:06:07.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:07.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:07.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:07.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:07 smithi066 ceph-mon[33264]: osdmap e66: 11 total, 11 up, 11 in 2024-05-04T03:06:07.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 -- 172.21.15.167:0/228830468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84104370 msgr2=0x7efd84104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:07.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 --2- 172.21.15.167:0/228830468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84104370 0x7efd84104780 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009990 tx=0x7efd6c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:06:07.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 -- 172.21.15.167:0/228830468 shutdown_connections 2024-05-04T03:06:07.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 --2- 172.21.15.167:0/228830468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd84104d10 0x7efd8410b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:07.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 --2- 172.21.15.167:0/228830468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd84103240 0x7efd84103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:07.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 --2- 172.21.15.167:0/228830468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84104370 0x7efd84104780 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:07.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 -- 172.21.15.167:0/228830468 >> 172.21.15.167:0/228830468 conn(0x7efd840fe880 msgr2=0x7efd84100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:07.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 -- 172.21.15.167:0/228830468 shutdown_connections 2024-05-04T03:06:07.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.943+0000 7efd89716700 1 -- 172.21.15.167:0/228830468 wait complete. 2024-05-04T03:06:07.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd89716700 1 Processor -- start 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd89716700 1 -- start start 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd89716700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84103240 0x7efd84199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd89716700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd84104370 0x7efd84199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd89716700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd84104d10 0x7efd841a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd89716700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd8410ec20 con 0x7efd84104370 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd89716700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd8410eaa0 con 0x7efd84104d10 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd89716700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd8410eda0 con 0x7efd84103240 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.944+0000 7efd82ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84103240 0x7efd84199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd82ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84103240 0x7efd84199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37222/0 (socket says 172.21.15.167:37222) 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd82ffd700 1 -- 172.21.15.167:0/4177128472 learned_addr learned my addr 172.21.15.167:0/4177128472 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd7a7fc700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd84104370 0x7efd84199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:07.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd837fe700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd84104d10 0x7efd841a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd82ffd700 1 -- 172.21.15.167:0/4177128472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd84104d10 msgr2=0x7efd841a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd82ffd700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd84104d10 0x7efd841a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd82ffd700 1 -- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd84104370 msgr2=0x7efd84199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd82ffd700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd84104370 0x7efd84199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd82ffd700 1 -- 172.21.15.167:0/4177128472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd841a1360 con 0x7efd84103240 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd82ffd700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84103240 0x7efd84199850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd7400d950 tx=0x7efd7400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd80ff9700 1 -- 172.21.15.167:0/4177128472 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74017070 con 0x7efd84103240 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd80ff9700 1 -- 172.21.15.167:0/4177128472 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd7400f690 con 0x7efd84103240 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd841a1650 con 0x7efd84103240 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd80ff9700 1 -- 172.21.15.167:0/4177128472 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd740039e0 con 0x7efd84103240 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.945+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd841a1c10 con 0x7efd84103240 2024-05-04T03:06:07.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.946+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd840512c0 con 0x7efd84103240 2024-05-04T03:06:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.948+0000 7efd80ff9700 1 -- 172.21.15.167:0/4177128472 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efd74003b80 con 0x7efd84103240 2024-05-04T03:06:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.948+0000 7efd80ff9700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd700b2880 0x7efd700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.949+0000 7efd80ff9700 1 -- 172.21.15.167:0/4177128472 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7efd740e1de0 con 0x7efd84103240 2024-05-04T03:06:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.949+0000 7efd7a7fc700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd700b2880 0x7efd700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:07.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.949+0000 7efd7a7fc700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd700b2880 0x7efd700b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00b5c0 tx=0x7efd6c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:07.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:07.952+0000 7efd80ff9700 1 -- 172.21.15.167:0/4177128472 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd74020030 con 0x7efd84103240 2024-05-04T03:06:08.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:08.163+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efd84063c00 con 0x7efd700b2880 2024-05-04T03:06:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:08 smithi039 ceph-mon[29715]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:06:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:06:08.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:08.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:08 smithi167 ceph-mon[33614]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:08.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:06:08.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:06:08.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:08.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:08 smithi066 ceph-mon[33264]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:08.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:06:08.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:06:08.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:09.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:09 smithi167 ceph-mon[33614]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:06:09.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:09 smithi066 ceph-mon[33264]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:06:09.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:09 smithi039 ceph-mon[29715]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:06:10.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:10 smithi066 ceph-mon[33264]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-04T03:06:10.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:10 smithi167 ceph-mon[33614]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-04T03:06:10.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:10 smithi039 ceph-mon[29715]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-05-04T03:06:11.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:11 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/1273944623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]: dispatch 2024-05-04T03:06:11.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:11 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]: dispatch 2024-05-04T03:06:11.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:11 smithi167 ceph-mon[33614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]': finished 2024-05-04T03:06:11.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:11 smithi167 ceph-mon[33614]: osdmap e67: 12 total, 11 up, 12 in 2024-05-04T03:06:11.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:11 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:11.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:11 smithi167 ceph-mon[33614]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:06:11.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:11 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/1273944623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]: dispatch 2024-05-04T03:06:11.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:11 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]: dispatch 2024-05-04T03:06:11.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:11 smithi066 ceph-mon[33264]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]': finished 2024-05-04T03:06:11.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:11 smithi066 ceph-mon[33264]: osdmap e67: 12 total, 11 up, 12 in 2024-05-04T03:06:11.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:11 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:11.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:11 smithi066 ceph-mon[33264]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:06:11.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:11 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/1273944623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]: dispatch 2024-05-04T03:06:11.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:11 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]: dispatch 2024-05-04T03:06:11.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:11 smithi039 ceph-mon[29715]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8940073-e9c7-4d49-af48-0a4c9a4b6f6e"}]': finished 2024-05-04T03:06:11.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:11 smithi039 ceph-mon[29715]: osdmap e67: 12 total, 11 up, 12 in 2024-05-04T03:06:11.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:11 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:11.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:11 smithi039 ceph-mon[29715]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:06:12.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:12 smithi066 ceph-mon[33264]: from='client.? 172.21.15.167:0/367062287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:06:12.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:12 smithi167 ceph-mon[33614]: from='client.? 172.21.15.167:0/367062287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:06:12.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:12 smithi039 ceph-mon[29715]: from='client.? 172.21.15.167:0/367062287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:06:13.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:13 smithi066 ceph-mon[33264]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-04T03:06:13.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:13 smithi167 ceph-mon[33614]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-04T03:06:13.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:13 smithi039 ceph-mon[29715]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-04T03:06:16.277 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:16 smithi167 ceph-mon[33614]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-04T03:06:16.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:16 smithi066 ceph-mon[33264]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-04T03:06:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:16 smithi039 ceph-mon[29715]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-04T03:06:18.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:18 smithi066 ceph-mon[33264]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:06:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:18 smithi167 ceph-mon[33614]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:06:18.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:18 smithi039 ceph-mon[29715]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:06:19.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-04T03:06:19.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:19.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:19 smithi167 ceph-mon[33614]: Deploying daemon osd.11 on smithi167 2024-05-04T03:06:19.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-04T03:06:19.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:19.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:19 smithi066 ceph-mon[33264]: Deploying daemon osd.11 on smithi167 2024-05-04T03:06:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-04T03:06:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:19.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:19 smithi039 ceph-mon[29715]: Deploying daemon osd.11 on smithi167 2024-05-04T03:06:20.160 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:20 smithi167 ceph-mon[33614]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:20.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:20 smithi066 ceph-mon[33264]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:20 smithi039 ceph-mon[29715]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:22.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:22 smithi167 ceph-mon[33614]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:22.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:22 smithi066 ceph-mon[33264]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:22.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:22 smithi039 ceph-mon[29715]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:24.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:24 smithi167 ceph-mon[33614]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:24.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:24 smithi066 ceph-mon[33264]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:24.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:24 smithi039 ceph-mon[29715]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:25.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:25.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:25.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:25.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:25.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:25.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:25.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:25.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:25.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:26.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:26 smithi167 ceph-mon[33614]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:26.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:26 smithi039 ceph-mon[29715]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:26.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:26 smithi066 ceph-mon[33264]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:28.439 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-05-04T03:06:28.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.438+0000 7efd80ff9700 1 -- 172.21.15.167:0/4177128472 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7efd84063c00 con 0x7efd700b2880 2024-05-04T03:06:28.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.440+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd700b2880 msgr2=0x7efd700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:28.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.440+0000 7efd89716700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd700b2880 0x7efd700b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00b5c0 tx=0x7efd6c01a040 comp rx=0 tx=0).stop 2024-05-04T03:06:28.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.440+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84103240 msgr2=0x7efd84199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:28.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.440+0000 7efd89716700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84103240 0x7efd84199850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd7400d950 tx=0x7efd7400dc60 comp rx=0 tx=0).stop 2024-05-04T03:06:28.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.441+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 shutdown_connections 2024-05-04T03:06:28.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.441+0000 7efd89716700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd700b2880 0x7efd700b4d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:28.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.441+0000 7efd89716700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd84104d10 0x7efd841a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:28.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.441+0000 7efd89716700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd84103240 0x7efd84199850 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:28.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.441+0000 7efd89716700 1 --2- 172.21.15.167:0/4177128472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd84104370 0x7efd84199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:28.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.441+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 >> 172.21.15.167:0/4177128472 conn(0x7efd840fe880 msgr2=0x7efd841075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:28.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.441+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 shutdown_connections 2024-05-04T03:06:28.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:28.441+0000 7efd89716700 1 -- 172.21.15.167:0/4177128472 wait complete. 2024-05-04T03:06:28.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:28 smithi167 ceph-mon[33614]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:28.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:28 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:28 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:28 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:28.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:28 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:28 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:28 smithi167 ceph-mon[33614]: from='osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:06:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:28 smithi167 ceph-mon[33614]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:06:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:28 smithi039 ceph-mon[29715]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:28 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:28 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:28 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:28 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:28 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:28 smithi039 ceph-mon[29715]: from='osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:06:28.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:28 smithi039 ceph-mon[29715]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:06:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:28 smithi066 ceph-mon[33264]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:28 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:28 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:28 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:28 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:28 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:28 smithi066 ceph-mon[33264]: from='osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:06:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:28 smithi066 ceph-mon[33264]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:06:29.115 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.11.service 2024-05-04T03:06:29.118 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-04T03:06:29.118 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-04T03:06:29.379 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:06:29.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:29.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:29.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:29.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:29 smithi039 ceph-mon[29715]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-04T03:06:29.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:29 smithi039 ceph-mon[29715]: osdmap e68: 12 total, 11 up, 12 in 2024-05-04T03:06:29.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:29 smithi039 ceph-mon[29715]: from='osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:06:29.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:29 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:29.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:29 smithi039 ceph-mon[29715]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:06:29.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:29.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:29.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:29.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:29 smithi167 ceph-mon[33614]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-04T03:06:29.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:29 smithi167 ceph-mon[33614]: osdmap e68: 12 total, 11 up, 12 in 2024-05-04T03:06:29.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:29 smithi167 ceph-mon[33614]: from='osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:06:29.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:29 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:29.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:29 smithi167 ceph-mon[33614]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:06:29.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:29.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:29.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:29.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:29 smithi066 ceph-mon[33264]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-04T03:06:29.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:29 smithi066 ceph-mon[33264]: osdmap e68: 12 total, 11 up, 12 in 2024-05-04T03:06:29.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:29 smithi066 ceph-mon[33264]: from='osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:06:29.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:29 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:29.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:29 smithi066 ceph-mon[33264]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-05-04T03:06:30.371 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:06:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:06:29.976+0000 7f77d78dd700 -1 osd.11 0 waiting for initial osdmap 2024-05-04T03:06:30.371 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:06:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:06:29.992+0000 7f77cf68a700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:06:30.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:30 smithi167 ceph-mon[33614]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:30.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:30 smithi167 ceph-mon[33614]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:06:30.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:30 smithi167 ceph-mon[33614]: osdmap e69: 12 total, 11 up, 12 in 2024-05-04T03:06:30.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:30.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:30.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:30 smithi167 ceph-mon[33614]: from='osd.11 ' entity='osd.11' 2024-05-04T03:06:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:30 smithi039 ceph-mon[29715]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:30 smithi039 ceph-mon[29715]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:06:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:30 smithi039 ceph-mon[29715]: osdmap e69: 12 total, 11 up, 12 in 2024-05-04T03:06:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:30 smithi039 ceph-mon[29715]: from='osd.11 ' entity='osd.11' 2024-05-04T03:06:30.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:30 smithi066 ceph-mon[33264]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:30.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:30 smithi066 ceph-mon[33264]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-05-04T03:06:30.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:30 smithi066 ceph-mon[33264]: osdmap e69: 12 total, 11 up, 12 in 2024-05-04T03:06:30.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:30.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:30.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:30 smithi066 ceph-mon[33264]: from='osd.11 ' entity='osd.11' 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.318+0000 7f860fb33700 1 -- 172.21.15.39:0/1591357064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 msgr2=0x7f8608104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.318+0000 7f860fb33700 1 --2- 172.21.15.39:0/1591357064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 0x7f8608104780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8604009990 tx=0x7f8604009ca0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.318+0000 7f860fb33700 1 -- 172.21.15.39:0/1591357064 shutdown_connections 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.318+0000 7f860fb33700 1 --2- 172.21.15.39:0/1591357064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8608104d10 0x7f860810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.318+0000 7f860fb33700 1 --2- 172.21.15.39:0/1591357064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8608103240 0x7f8608103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.318+0000 7f860fb33700 1 --2- 172.21.15.39:0/1591357064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 0x7f8608104780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.318+0000 7f860fb33700 1 -- 172.21.15.39:0/1591357064 >> 172.21.15.39:0/1591357064 conn(0x7f86080fe880 msgr2=0x7f8608100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 -- 172.21.15.39:0/1591357064 shutdown_connections 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 -- 172.21.15.39:0/1591357064 wait complete. 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 Processor -- start 2024-05-04T03:06:31.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 -- start start 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8608103240 0x7f860819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 0x7f860819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8608104d10 0x7f86081a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f860810eb50 con 0x7f8608104370 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f860810e9d0 con 0x7f8608104d10 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.319+0000 7f860fb33700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f860810ecd0 con 0x7f8608103240 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 0x7f860819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 0x7f860819ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56122/0 (socket says 172.21.15.39:56122) 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d8cf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8608103240 0x7f860819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 -- 172.21.15.39:0/4001150648 learned_addr learned my addr 172.21.15.39:0/4001150648 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860e0d0700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8608104d10 0x7f86081a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 -- 172.21.15.39:0/4001150648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8608103240 msgr2=0x7f860819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8608103240 0x7f860819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 -- 172.21.15.39:0/4001150648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8608104d10 msgr2=0x7f86081a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8608104d10 0x7f86081a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 -- 172.21.15.39:0/4001150648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86081a2390 con 0x7f8608104370 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d8cf700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8608103240 0x7f860819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860d0ce700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 0x7f860819ad70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f86040098e0 tx=0x7f860400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f85feffd700 1 -- 172.21.15.39:0/4001150648 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604026070 con 0x7f8608104370 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f85feffd700 1 -- 172.21.15.39:0/4001150648 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8604003d20 con 0x7f8608104370 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f85feffd700 1 -- 172.21.15.39:0/4001150648 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860400fe10 con 0x7f8608104370 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86081a2620 con 0x7f8608104370 2024-05-04T03:06:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.320+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a2a20 con 0x7f8608104370 2024-05-04T03:06:31.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.322+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86080512c0 con 0x7f8608104370 2024-05-04T03:06:31.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.323+0000 7f85feffd700 1 -- 172.21.15.39:0/4001150648 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8604029070 con 0x7f8608104370 2024-05-04T03:06:31.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.324+0000 7f85feffd700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85f40b2880 0x7f85f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:31.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.324+0000 7f860d8cf700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85f40b2880 0x7f85f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:31.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.324+0000 7f85feffd700 1 -- 172.21.15.39:0/4001150648 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f86040e30e0 con 0x7f8608104370 2024-05-04T03:06:31.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.324+0000 7f860d8cf700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85f40b2880 0x7f85f40b4d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8608104270 tx=0x7f85f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:31.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.327+0000 7f85feffd700 1 -- 172.21.15.39:0/4001150648 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8604030080 con 0x7f8608104370 2024-05-04T03:06:31.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.546+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f86081a2e40 con 0x7f8608104370 2024-05-04T03:06:31.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.546+0000 7f85feffd700 1 -- 172.21.15.39:0/4001150648 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f8604013070 con 0x7f8608104370 2024-05-04T03:06:31.547 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.548+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85f40b2880 msgr2=0x7f85f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.548+0000 7f860fb33700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85f40b2880 0x7f85f40b4d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8608104270 tx=0x7f85f8009260 comp rx=0 tx=0).stop 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.548+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 msgr2=0x7f860819ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.548+0000 7f860fb33700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 0x7f860819ad70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f86040098e0 tx=0x7f860400b9e0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.549+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 shutdown_connections 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.549+0000 7f860fb33700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85f40b2880 0x7f85f40b4d90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.549+0000 7f860fb33700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8608104d10 0x7f86081a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.549+0000 7f860fb33700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8608103240 0x7f860819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.549+0000 7f860fb33700 1 --2- 172.21.15.39:0/4001150648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8608104370 0x7f860819ad70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:31.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.549+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 >> 172.21.15.39:0/4001150648 conn(0x7f86080fe880 msgr2=0x7f8608100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:31.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.549+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 shutdown_connections 2024-05-04T03:06:31.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:31.549+0000 7f860fb33700 1 -- 172.21.15.39:0/4001150648 wait complete. 2024-05-04T03:06:31.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:31 smithi167 ceph-mon[33614]: purged_snaps scrub starts 2024-05-04T03:06:31.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:31 smithi167 ceph-mon[33614]: purged_snaps scrub ok 2024-05-04T03:06:31.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:31 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:31.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:31 smithi167 ceph-mon[33614]: osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] boot 2024-05-04T03:06:31.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:31 smithi167 ceph-mon[33614]: osdmap e70: 12 total, 12 up, 12 in 2024-05-04T03:06:31.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:31 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:31.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:31 smithi039 ceph-mon[29715]: purged_snaps scrub starts 2024-05-04T03:06:31.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:31 smithi039 ceph-mon[29715]: purged_snaps scrub ok 2024-05-04T03:06:31.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:31 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:31.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:31 smithi039 ceph-mon[29715]: osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] boot 2024-05-04T03:06:31.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:31 smithi039 ceph-mon[29715]: osdmap e70: 12 total, 12 up, 12 in 2024-05-04T03:06:31.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:31 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:31.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:31 smithi066 ceph-mon[33264]: purged_snaps scrub starts 2024-05-04T03:06:31.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:31 smithi066 ceph-mon[33264]: purged_snaps scrub ok 2024-05-04T03:06:31.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:31 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:31.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:31 smithi066 ceph-mon[33264]: osd.11 [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] boot 2024-05-04T03:06:31.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:31 smithi066 ceph-mon[33264]: osdmap e70: 12 total, 12 up, 12 in 2024-05-04T03:06:31.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:31 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:06:32.356 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1714791990,"num_in_osds":12,"osd_in_since":1714791970,"num_remapped_pgs":0} 2024-05-04T03:06:32.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:06:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:06:32.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:32 smithi167 ceph-mon[33614]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:32.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:32 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4001150648' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:06:32.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:32 smithi167 ceph-mon[33614]: osdmap e71: 12 total, 12 up, 12 in 2024-05-04T03:06:32.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:32 smithi039 ceph-mon[29715]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:32.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:32 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4001150648' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:06:32.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:32 smithi039 ceph-mon[29715]: osdmap e71: 12 total, 12 up, 12 in 2024-05-04T03:06:32.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:32 smithi066 ceph-mon[33264]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:06:32.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:32 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4001150648' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:06:32.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:32 smithi066 ceph-mon[33264]: osdmap e71: 12 total, 12 up, 12 in 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: Detected new or changed devices on smithi167 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi167 to 4272M 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:33.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: Detected new or changed devices on smithi167 2024-05-04T03:06:33.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi167 to 4272M 2024-05-04T03:06:33.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:33.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:33.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: Detected new or changed devices on smithi167 2024-05-04T03:06:33.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi167 to 4272M 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:34.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 -- 172.21.15.39:0/1744303021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 msgr2=0x7f08001038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:34.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 --2- 172.21.15.39:0/1744303021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 0x7f08001038f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0099c0 tx=0x7f07fc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 -- 172.21.15.39:0/1744303021 shutdown_connections 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 --2- 172.21.15.39:0/1744303021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 0x7f08001038f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 --2- 172.21.15.39:0/1744303021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0800108760 0x7f08001100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 --2- 172.21.15.39:0/1744303021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0800107e10 0x7f0800108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 -- 172.21.15.39:0/1744303021 >> 172.21.15.39:0/1744303021 conn(0x7f08000fef30 msgr2=0x7f0800101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 -- 172.21.15.39:0/1744303021 shutdown_connections 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.348+0000 7f080864c700 1 -- 172.21.15.39:0/1744303021 wait complete. 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.349+0000 7f080864c700 1 Processor -- start 2024-05-04T03:06:34.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.349+0000 7f080864c700 1 -- start start 2024-05-04T03:06:34.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f080864c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 0x7f080019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:34.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f080864c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0800107e10 0x7f080019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:34.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f080864c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0800108760 0x7f0800198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:34.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f080864c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f08001121a0 con 0x7f08001034e0 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f080864c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112020 con 0x7f0800108760 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f080864c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112320 con 0x7f0800107e10 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 0x7f080019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 0x7f080019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56138/0 (socket says 172.21.15.39:56138) 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 -- 172.21.15.39:0/1097938746 learned_addr learned my addr 172.21.15.39:0/1097938746 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f0805be7700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0800107e10 0x7f080019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f0806be9700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0800108760 0x7f0800198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 -- 172.21.15.39:0/1097938746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0800107e10 msgr2=0x7f080019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0800107e10 0x7f080019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 -- 172.21.15.39:0/1097938746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0800108760 msgr2=0x7f0800198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0800108760 0x7f0800198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 -- 172.21.15.39:0/1097938746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0800198ff0 con 0x7f08001034e0 2024-05-04T03:06:34.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f0806be9700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0800108760 0x7f0800198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:06:34.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.350+0000 7f08063e8700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 0x7f080019e720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07fc000c00 tx=0x7f07fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:34.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.351+0000 7f07f77fe700 1 -- 172.21.15.39:0/1097938746 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc027070 con 0x7f08001034e0 2024-05-04T03:06:34.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.351+0000 7f07f77fe700 1 -- 172.21.15.39:0/1097938746 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07fc00fd30 con 0x7f08001034e0 2024-05-04T03:06:34.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.351+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0800199280 con 0x7f08001034e0 2024-05-04T03:06:34.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.351+0000 7f07f77fe700 1 -- 172.21.15.39:0/1097938746 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0173f0 con 0x7f08001034e0 2024-05-04T03:06:34.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.351+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001a99c0 con 0x7f08001034e0 2024-05-04T03:06:34.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.352+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0800104990 con 0x7f08001034e0 2024-05-04T03:06:34.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.354+0000 7f07f77fe700 1 -- 172.21.15.39:0/1097938746 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f07fc02a070 con 0x7f08001034e0 2024-05-04T03:06:34.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.354+0000 7f07f77fe700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f07ec0b2900 0x7f07ec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:34.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.354+0000 7f0805be7700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f07ec0b2900 0x7f07ec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:34.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.354+0000 7f07f77fe700 1 -- 172.21.15.39:0/1097938746 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f07fc0e3180 con 0x7f08001034e0 2024-05-04T03:06:34.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.355+0000 7f0805be7700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f07ec0b2900 0x7f07ec0b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f08001029e0 tx=0x7f07f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:34.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.358+0000 7f07f77fe700 1 -- 172.21.15.39:0/1097938746 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07fc031020 con 0x7f08001034e0 2024-05-04T03:06:34.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.580+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08000512c0 con 0x7f08001034e0 2024-05-04T03:06:34.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.581+0000 7f07f77fe700 1 -- 172.21.15.39:0/1097938746 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18383 (secure 0 0 0) 0x7f07fc013070 con 0x7f08001034e0 2024-05-04T03:06:34.581 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:06:34.582 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":71,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:06:31.982044+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.583+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f07ec0b2900 msgr2=0x7f07ec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.583+0000 7f080864c700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f07ec0b2900 0x7f07ec0b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f08001029e0 tx=0x7f07f000b410 comp rx=0 tx=0).stop 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.583+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 msgr2=0x7f080019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.583+0000 7f080864c700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 0x7f080019e720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f07fc000c00 tx=0x7f07fc0047b0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.584+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 shutdown_connections 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.584+0000 7f080864c700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f07ec0b2900 0x7f07ec0b4e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.584+0000 7f080864c700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08001034e0 0x7f080019e720 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.584+0000 7f080864c700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0800108760 0x7f0800198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.584+0000 7f080864c700 1 --2- 172.21.15.39:0/1097938746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0800107e10 0x7f080019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.584+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 >> 172.21.15.39:0/1097938746 conn(0x7f08000fef30 msgr2=0x7f08001096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.584+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 shutdown_connections 2024-05-04T03:06:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:34.584+0000 7f080864c700 1 -- 172.21.15.39:0/1097938746 wait complete. 2024-05-04T03:06:34.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:34 smithi039 ceph-mon[29715]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:34.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:34 smithi167 ceph-mon[33614]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:34.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:34 smithi066 ceph-mon[33264]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:35.287 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T03:02:42.983535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T03:06:35.287 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-04T03:06:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:06:35.662 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:35 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1097938746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:06:35.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:35 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1097938746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:06:35.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:35 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1097938746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:06:36.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:36 smithi039 ceph-mon[29715]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:36.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:36 smithi066 ceph-mon[33264]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:36.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:36 smithi167 ceph-mon[33614]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 -- 172.21.15.39:0/1804258369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364108f10 msgr2=0x7f9364109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 --2- 172.21.15.39:0/1804258369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364108f10 0x7f9364109320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f93540099c0 tx=0x7f9354009cd0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 -- 172.21.15.39:0/1804258369 shutdown_connections 2024-05-04T03:06:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 --2- 172.21.15.39:0/1804258369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9364109860 0x7f93641111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 --2- 172.21.15.39:0/1804258369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93641045e0 0x7f93641049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 --2- 172.21.15.39:0/1804258369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364108f10 0x7f9364109320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 -- 172.21.15.39:0/1804258369 >> 172.21.15.39:0/1804258369 conn(0x7f93640ffed0 msgr2=0x7f9364102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 -- 172.21.15.39:0/1804258369 shutdown_connections 2024-05-04T03:06:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.252+0000 7f936bc17700 1 -- 172.21.15.39:0/1804258369 wait complete. 2024-05-04T03:06:37.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936bc17700 1 Processor -- start 2024-05-04T03:06:37.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936bc17700 1 -- start start 2024-05-04T03:06:37.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936bc17700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93641045e0 0x7f9364076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:37.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936bc17700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9364108f10 0x7f9364077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:37.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936bc17700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364109860 0x7f9364077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936bc17700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9364114390 con 0x7f9364109860 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936bc17700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9364114210 con 0x7f93641045e0 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936bc17700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9364114510 con 0x7f9364108f10 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936a1b4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364109860 0x7f9364077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f93699b3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93641045e0 0x7f9364076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936a1b4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364109860 0x7f9364077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47758/0 (socket says 172.21.15.39:47758) 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.253+0000 7f936a1b4700 1 -- 172.21.15.39:0/1546444869 learned_addr learned my addr 172.21.15.39:0/1546444869 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f93691b2700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9364108f10 0x7f9364077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f936a1b4700 1 -- 172.21.15.39:0/1546444869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9364108f10 msgr2=0x7f9364077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f936a1b4700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9364108f10 0x7f9364077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f936a1b4700 1 -- 172.21.15.39:0/1546444869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93641045e0 msgr2=0x7f9364076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f936a1b4700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93641045e0 0x7f9364076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f936a1b4700 1 -- 172.21.15.39:0/1546444869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93641a91e0 con 0x7f9364109860 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f93691b2700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9364108f10 0x7f9364077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f936a1b4700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364109860 0x7f9364077650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b970 tx=0x7f935c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f935affd700 1 -- 172.21.15.39:0/1546444869 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c019070 con 0x7f9364109860 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.254+0000 7f935affd700 1 -- 172.21.15.39:0/1546444869 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f935c00cdc0 con 0x7f9364109860 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.255+0000 7f935affd700 1 -- 172.21.15.39:0/1546444869 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c014e30 con 0x7f9364109860 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.255+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93641a94d0 con 0x7f9364109860 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.255+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93641a9990 con 0x7f9364109860 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.256+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93640512c0 con 0x7f9364109860 2024-05-04T03:06:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.257+0000 7f935affd700 1 -- 172.21.15.39:0/1546444869 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f935c01c070 con 0x7f9364109860 2024-05-04T03:06:37.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.258+0000 7f935affd700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f93500b2880 0x7f93500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:37.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.258+0000 7f93699b3700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f93500b2880 0x7f93500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:37.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.259+0000 7f935affd700 1 -- 172.21.15.39:0/1546444869 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f935c0e1630 con 0x7f9364109860 2024-05-04T03:06:37.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.259+0000 7f93699b3700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f93500b2880 0x7f93500b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9360005950 tx=0x7f936000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.262+0000 7f935affd700 1 -- 172.21.15.39:0/1546444869 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f935c0a9750 con 0x7f9364109860 2024-05-04T03:06:37.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.474+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9364078100 con 0x7f9364109860 2024-05-04T03:06:37.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.474+0000 7f935affd700 1 -- 172.21.15.39:0/1546444869 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f935c0a9050 con 0x7f9364109860 2024-05-04T03:06:37.475 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-04T03:06:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f93500b2880 msgr2=0x7f93500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f93500b2880 0x7f93500b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9360005950 tx=0x7f936000a300 comp rx=0 tx=0).stop 2024-05-04T03:06:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364109860 msgr2=0x7f9364077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364109860 0x7f9364077650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b970 tx=0x7f935c00bc80 comp rx=0 tx=0).stop 2024-05-04T03:06:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 shutdown_connections 2024-05-04T03:06:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f93500b2880 0x7f93500b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9364109860 0x7f9364077650 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93641045e0 0x7f9364076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 --2- 172.21.15.39:0/1546444869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9364108f10 0x7f9364077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 >> 172.21.15.39:0/1546444869 conn(0x7f93640ffed0 msgr2=0x7f936410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 shutdown_connections 2024-05-04T03:06:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:37.477+0000 7f936bc17700 1 -- 172.21.15.39:0/1546444869 wait complete. 2024-05-04T03:06:38.224 INFO:tasks.cephadm:Adding mds.a on smithi039 2024-05-04T03:06:38.224 INFO:tasks.cephadm:Adding mds.d on smithi039 2024-05-04T03:06:38.224 INFO:tasks.cephadm:Adding mds.g on smithi039 2024-05-04T03:06:38.224 INFO:tasks.cephadm:Adding mds.j on smithi039 2024-05-04T03:06:38.225 INFO:tasks.cephadm:Adding mds.b on smithi066 2024-05-04T03:06:38.225 INFO:tasks.cephadm:Adding mds.e on smithi066 2024-05-04T03:06:38.225 INFO:tasks.cephadm:Adding mds.h on smithi066 2024-05-04T03:06:38.225 INFO:tasks.cephadm:Adding mds.k on smithi066 2024-05-04T03:06:38.225 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-05-04T03:06:38.225 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-05-04T03:06:38.225 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-05-04T03:06:38.225 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-05-04T03:06:38.225 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-05-04T03:06:38.474 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:06:38.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:38 smithi066 ceph-mon[33264]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:38.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:38 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1546444869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:06:38.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:38 smithi167 ceph-mon[33614]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:38.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:38 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1546444869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:06:38.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:38 smithi039 ceph-mon[29715]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:38.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:38 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1546444869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1664965102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38103260 msgr2=0x7f6f38103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1664965102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38103260 0x7f6f38103670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800b0a0 tx=0x7f6f2800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1664965102 shutdown_connections 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1664965102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38104d30 0x7f6f3810b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1664965102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38103260 0x7f6f38103670 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1664965102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f38104390 0x7f6f381047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1664965102 >> 172.21.15.167:0/1664965102 conn(0x7f6f380fe8c0 msgr2=0x7f6f38100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1664965102 shutdown_connections 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.980+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1664965102 wait complete. 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f3dd07700 1 Processor -- start 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f3dd07700 1 -- start start 2024-05-04T03:06:39.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f3dd07700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38103260 0x7f6f38199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f3dd07700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f38104390 0x7f6f38199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f3dd07700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38104d30 0x7f6f381a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f3dd07700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3810eb50 con 0x7f6f38103260 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f3dd07700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3810e9d0 con 0x7f6f38104390 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f3dd07700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3810ecd0 con 0x7f6f38104d30 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f37fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38104d30 0x7f6f381a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f37fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38104d30 0x7f6f381a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58794/0 (socket says 172.21.15.167:58794) 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f37fff700 1 -- 172.21.15.167:0/1780233285 learned_addr learned my addr 172.21.15.167:0/1780233285 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.981+0000 7f6f377fe700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38103260 0x7f6f38199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f36ffd700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f38104390 0x7f6f38199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f37fff700 1 -- 172.21.15.167:0/1780233285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f38104390 msgr2=0x7f6f38199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f37fff700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f38104390 0x7f6f38199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f37fff700 1 -- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38103260 msgr2=0x7f6f38199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:39.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f37fff700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38103260 0x7f6f38199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:39.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f37fff700 1 -- 172.21.15.167:0/1780233285 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f381a11f0 con 0x7f6f38104d30 2024-05-04T03:06:39.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f37fff700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38104d30 0x7f6f381a0c00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6f300098d0 tx=0x7f6f30009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:39.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f34ff9700 1 -- 172.21.15.167:0/1780233285 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000d020 con 0x7f6f38104d30 2024-05-04T03:06:39.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f38075a50 con 0x7f6f38104d30 2024-05-04T03:06:39.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f34ff9700 1 -- 172.21.15.167:0/1780233285 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f3000d1c0 con 0x7f6f38104d30 2024-05-04T03:06:39.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f34ff9700 1 -- 172.21.15.167:0/1780233285 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000c460 con 0x7f6f38104d30 2024-05-04T03:06:39.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.982+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f38075f90 con 0x7f6f38104d30 2024-05-04T03:06:39.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.984+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f380512c0 con 0x7f6f38104d30 2024-05-04T03:06:39.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.985+0000 7f6f34ff9700 1 -- 172.21.15.167:0/1780233285 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6f3001f020 con 0x7f6f38104d30 2024-05-04T03:06:39.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.986+0000 7f6f34ff9700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6f200b2880 0x7f6f200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:39.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.986+0000 7f6f34ff9700 1 -- 172.21.15.167:0/1780233285 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f6f300e2d50 con 0x7f6f38104d30 2024-05-04T03:06:39.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.986+0000 7f6f377fe700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6f200b2880 0x7f6f200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:39.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.987+0000 7f6f377fe700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6f200b2880 0x7f6f200b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800b830 tx=0x7f6f28007ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:39.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:39.991+0000 7f6f34ff9700 1 -- 172.21.15.167:0/1780233285 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f30054df0 con 0x7f6f38104d30 2024-05-04T03:06:40.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.201+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) -- 0x7f6f380762b0 con 0x7f6f200b2880 2024-05-04T03:06:40.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.210+0000 7f6f34ff9700 1 -- 172.21.15.167:0/1780233285 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6f380762b0 con 0x7f6f200b2880 2024-05-04T03:06:40.210 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6f200b2880 msgr2=0x7f6f200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6f200b2880 0x7f6f200b4d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800b830 tx=0x7f6f28007ed0 comp rx=0 tx=0).stop 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38104d30 msgr2=0x7f6f381a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38104d30 0x7f6f381a0c00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6f300098d0 tx=0x7f6f30009be0 comp rx=0 tx=0).stop 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 shutdown_connections 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6f200b2880 0x7f6f200b4d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38103260 0x7f6f38199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f38104d30 0x7f6f381a0c00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:40.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 --2- 172.21.15.167:0/1780233285 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6f38104390 0x7f6f38199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:40.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.212+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 >> 172.21.15.167:0/1780233285 conn(0x7f6f380fe8c0 msgr2=0x7f6f381075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:40.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.213+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 shutdown_connections 2024-05-04T03:06:40.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-04T03:06:40.213+0000 7f6f3dd07700 1 -- 172.21.15.167:0/1780233285 wait complete. 2024-05-04T03:06:40.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:40 smithi167 ceph-mon[33614]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:40.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:40.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:40 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:40 smithi066 ceph-mon[33264]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:40 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:40.920 DEBUG:teuthology.orchestra.run.smithi039:mds.a> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.a.service 2024-05-04T03:06:40.922 DEBUG:teuthology.orchestra.run.smithi039:mds.d> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.d.service 2024-05-04T03:06:40.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:40 smithi039 ceph-mon[29715]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:40.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:40.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:40 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:06:40.941 DEBUG:teuthology.orchestra.run.smithi039:mds.g> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.g.service 2024-05-04T03:06:40.984 DEBUG:teuthology.orchestra.run.smithi039:mds.j> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.j.service 2024-05-04T03:06:41.028 DEBUG:teuthology.orchestra.run.smithi066:mds.b> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.b.service 2024-05-04T03:06:41.030 DEBUG:teuthology.orchestra.run.smithi066:mds.e> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.e.service 2024-05-04T03:06:41.074 DEBUG:teuthology.orchestra.run.smithi066:mds.h> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.h.service 2024-05-04T03:06:41.116 DEBUG:teuthology.orchestra.run.smithi066:mds.k> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.k.service 2024-05-04T03:06:41.160 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.c.service 2024-05-04T03:06:41.163 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.f.service 2024-05-04T03:06:41.207 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.i.service 2024-05-04T03:06:41.250 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.l.service 2024-05-04T03:06:41.293 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-04T03:06:41.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:06:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:06:41.660 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:06:41.660 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-05-04T03:06:41.660 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:41.661 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:41.661 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:41.661 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:41.661 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:41.661 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:41.661 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:41.661 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:41 smithi039 ceph-mon[29715]: Deploying daemon mds.l on smithi167 2024-05-04T03:06:41.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:06:41.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-05-04T03:06:41.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:41.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:41.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:41.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:41.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:41.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:41.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:41.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:41 smithi167 ceph-mon[33614]: Deploying daemon mds.l on smithi167 2024-05-04T03:06:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:06:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-05-04T03:06:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:41.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:06:41.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:41.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:41.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:41.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:41.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:41.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:41 smithi066 ceph-mon[33264]: Deploying daemon mds.l on smithi167 2024-05-04T03:06:43.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 -- 172.21.15.39:0/905858627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c108740 msgr2=0x7fa79c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 --2- 172.21.15.39:0/905858627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c108740 0x7fa79c1100a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa79400b0d0 tx=0x7fa79400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 -- 172.21.15.39:0/905858627 shutdown_connections 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 --2- 172.21.15.39:0/905858627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c108740 0x7fa79c1100a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 --2- 172.21.15.39:0/905858627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa79c1034c0 0x7fa79c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 --2- 172.21.15.39:0/905858627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c107df0 0x7fa79c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 -- 172.21.15.39:0/905858627 >> 172.21.15.39:0/905858627 conn(0x7fa79c0fedf0 msgr2=0x7fa79c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 -- 172.21.15.39:0/905858627 shutdown_connections 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.421+0000 7fa7a24da700 1 -- 172.21.15.39:0/905858627 wait complete. 2024-05-04T03:06:43.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a24da700 1 Processor -- start 2024-05-04T03:06:43.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a24da700 1 -- start start 2024-05-04T03:06:43.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a24da700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c1034c0 0x7fa79c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:43.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a24da700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c107df0 0x7fa79c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:43.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a24da700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa79c108740 0x7fa79c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a24da700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c113220 con 0x7fa79c107df0 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a24da700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1130a0 con 0x7fa79c108740 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a24da700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1133a0 con 0x7fa79c1034c0 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa79b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c107df0 0x7fa79c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa79b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c107df0 0x7fa79c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47768/0 (socket says 172.21.15.39:47768) 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa79b7fe700 1 -- 172.21.15.39:0/2511543648 learned_addr learned my addr 172.21.15.39:0/2511543648 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa79bfff700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c1034c0 0x7fa79c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.422+0000 7fa7a0a77700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa79c108740 0x7fa79c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa79b7fe700 1 -- 172.21.15.39:0/2511543648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c1034c0 msgr2=0x7fa79c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa79b7fe700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c1034c0 0x7fa79c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa79b7fe700 1 -- 172.21.15.39:0/2511543648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa79c108740 msgr2=0x7fa79c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:43.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa79b7fe700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa79c108740 0x7fa79c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa79b7fe700 1 -- 172.21.15.39:0/2511543648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c19a0d0 con 0x7fa79c107df0 2024-05-04T03:06:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa79b7fe700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c107df0 0x7fa79c19fd40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa79000d980 tx=0x7fa79000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa7997fa700 1 -- 172.21.15.39:0/2511543648 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790017070 con 0x7fa79c107df0 2024-05-04T03:06:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa7997fa700 1 -- 172.21.15.39:0/2511543648 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa79000fd20 con 0x7fa79c107df0 2024-05-04T03:06:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c19a3c0 con 0x7fa79c107df0 2024-05-04T03:06:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa7997fa700 1 -- 172.21.15.39:0/2511543648 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79000e3b0 con 0x7fa79c107df0 2024-05-04T03:06:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.423+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c1a6690 con 0x7fa79c107df0 2024-05-04T03:06:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.424+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79c0512c0 con 0x7fa79c107df0 2024-05-04T03:06:43.427 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:06:43 smithi167 podman[64128]: 2024-05-04 03:06:43.368181897 +0000 UTC m=+0.143843604 container create 5db8322d980a8103ee51b2ea2d250989854fe42b3bb0549f556e0ad24cbe63eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-04T03:06:43.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.426+0000 7fa7997fa700 1 -- 172.21.15.39:0/2511543648 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa79000f890 con 0x7fa79c107df0 2024-05-04T03:06:43.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.427+0000 7fa7997fa700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7840b2900 0x7fa7840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:43.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.427+0000 7fa79bfff700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7840b2900 0x7fa7840b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:43.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.427+0000 7fa7997fa700 1 -- 172.21.15.39:0/2511543648 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fa7900e16e0 con 0x7fa79c107df0 2024-05-04T03:06:43.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.427+0000 7fa79bfff700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7840b2900 0x7fa7840b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa79c19ab60 tx=0x7fa78c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:43.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.430+0000 7fa7997fa700 1 -- 172.21.15.39:0/2511543648 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7900a9920 con 0x7fa79c107df0 2024-05-04T03:06:43.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.695+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa79c19af50 con 0x7fa79c107df0 2024-05-04T03:06:43.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.696+0000 7fa7997fa700 1 -- 172.21.15.39:0/2511543648 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fa7900a9220 con 0x7fa79c107df0 2024-05-04T03:06:43.696 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:06:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-04T03:06:43.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7840b2900 msgr2=0x7fa7840b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7840b2900 0x7fa7840b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa79c19ab60 tx=0x7fa78c009260 comp rx=0 tx=0).stop 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c107df0 msgr2=0x7fa79c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c107df0 0x7fa79c19fd40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa79000d980 tx=0x7fa79000dc90 comp rx=0 tx=0).stop 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 shutdown_connections 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7840b2900 0x7fa7840b4e10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa79c108740 0x7fa79c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c1034c0 0x7fa79c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 --2- 172.21.15.39:0/2511543648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa79c107df0 0x7fa79c19fd40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 >> 172.21.15.39:0/2511543648 conn(0x7fa79c0fedf0 msgr2=0x7fa79c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 shutdown_connections 2024-05-04T03:06:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:43.698+0000 7fa7a24da700 1 -- 172.21.15.39:0/2511543648 wait complete. 2024-05-04T03:06:43.700 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2024-05-04T03:06:43.926 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:06:43 smithi167 podman[64128]: 2024-05-04 03:06:43.621378193 +0000 UTC m=+0.397039838 container init 5db8322d980a8103ee51b2ea2d250989854fe42b3bb0549f556e0ad24cbe63eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:06:43.926 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:06:43 smithi167 podman[64128]: 2024-05-04 03:06:43.623957481 +0000 UTC m=+0.399619129 container start 5db8322d980a8103ee51b2ea2d250989854fe42b3bb0549f556e0ad24cbe63eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:06:43.926 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:06:43 smithi167 ceph-mds[64147]: -- 172.21.15.167:0/1292542607 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6191b3a40 con 0x55d6191fb180 2024-05-04T03:06:43.926 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:06:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l[64143]: starting mds.l at 2024-05-04T03:06:43.926 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:06:43 smithi167 bash[64128]: 5db8322d980a8103ee51b2ea2d250989854fe42b3bb0549f556e0ad24cbe63eb 2024-05-04T03:06:43.927 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:06:43 smithi167 systemd[1]: Started Ceph mds.l for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6832/1077677237,v1:172.21.15.167:6833/1077677237] up:boot 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: fsmap 1 up:standby 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2511543648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:44.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6832/1077677237,v1:172.21.15.167:6833/1077677237] up:boot 2024-05-04T03:06:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: fsmap 1 up:standby 2024-05-04T03:06:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:06:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2511543648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:44.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:44.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6832/1077677237,v1:172.21.15.167:6833/1077677237] up:boot 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: fsmap 1 up:standby 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2511543648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:44.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:45.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:45 smithi066 ceph-mon[33264]: Deploying daemon mds.k on smithi066 2024-05-04T03:06:45.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:45 smithi167 ceph-mon[33614]: Deploying daemon mds.k on smithi066 2024-05-04T03:06:45.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:45 smithi039 ceph-mon[29715]: Deploying daemon mds.k on smithi066 2024-05-04T03:06:46.072 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:46 smithi066 ceph-mon[33264]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:46.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:46 smithi039 ceph-mon[29715]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:46.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:46 smithi167 ceph-mon[33614]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:46.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:06:46.490 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 systemd[1]: Starting Ceph mds.k for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:06:46.739 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:06:46.752 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 podman[62895]: 2024-05-04 03:06:46.393510561 +0000 UTC m=+0.018350998 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:06:46.752 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 podman[62895]: 2024-05-04 03:06:46.507557971 +0000 UTC m=+0.132398465 container create ba2bcf5d2559911a365479cdd5769bd0384521601f323188b18317cfaaf0485c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-04T03:06:47.067 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 podman[62895]: 2024-05-04 03:06:46.752292843 +0000 UTC m=+0.377133274 container init ba2bcf5d2559911a365479cdd5769bd0384521601f323188b18317cfaaf0485c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-04T03:06:47.067 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 podman[62895]: 2024-05-04 03:06:46.75563717 +0000 UTC m=+0.380477596 container start ba2bcf5d2559911a365479cdd5769bd0384521601f323188b18317cfaaf0485c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-04T03:06:47.067 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 ceph-mds[62914]: -- 172.21.15.66:0/2728603281 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e26effba40 con 0x55e26f043a80 2024-05-04T03:06:47.067 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k[62910]: starting mds.k at 2024-05-04T03:06:47.067 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 bash[62895]: ba2bcf5d2559911a365479cdd5769bd0384521601f323188b18317cfaaf0485c 2024-05-04T03:06:47.067 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:06:46 smithi066 systemd[1]: Started Ceph mds.k for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:06:48.026 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.026 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: Deploying daemon mds.j on smithi039 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6832/462065299,v1:172.21.15.66:6833/462065299] up:boot 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: fsmap 2 up:standby 2024-05-04T03:06:48.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:06:48.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: Deploying daemon mds.j on smithi039 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6832/462065299,v1:172.21.15.66:6833/462065299] up:boot 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: fsmap 2 up:standby 2024-05-04T03:06:48.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:06:48.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: Deploying daemon mds.j on smithi039 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6832/462065299,v1:172.21.15.66:6833/462065299] up:boot 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: fsmap 2 up:standby 2024-05-04T03:06:48.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:06:48.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 -- 172.21.15.39:0/1467501607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 msgr2=0x7fd178109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:48.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 --2- 172.21.15.39:0/1467501607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 0x7fd178109320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0099c0 tx=0x7fd16c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:06:48.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 -- 172.21.15.39:0/1467501607 shutdown_connections 2024-05-04T03:06:48.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 --2- 172.21.15.39:0/1467501607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd178109860 0x7fd1781111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 --2- 172.21.15.39:0/1467501607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1781045e0 0x7fd1781049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 --2- 172.21.15.39:0/1467501607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 0x7fd178109320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 -- 172.21.15.39:0/1467501607 >> 172.21.15.39:0/1467501607 conn(0x7fd1780ffef0 msgr2=0x7fd178102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 -- 172.21.15.39:0/1467501607 shutdown_connections 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.804+0000 7fd17de97700 1 -- 172.21.15.39:0/1467501607 wait complete. 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd17de97700 1 Processor -- start 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd17de97700 1 -- start start 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd17de97700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1781045e0 0x7fd17819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd17de97700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 0x7fd17819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd17de97700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd178109860 0x7fd17819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd17de97700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd178114320 con 0x7fd178108f10 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd17de97700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781141a0 con 0x7fd1781045e0 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd17de97700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781144a0 con 0x7fd178109860 2024-05-04T03:06:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd176ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 0x7fd17819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd176ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 0x7fd17819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48936/0 (socket says 172.21.15.39:48936) 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd176ffd700 1 -- 172.21.15.39:0/2244833583 learned_addr learned my addr 172.21.15.39:0/2244833583 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd1777fe700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1781045e0 0x7fd17819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.805+0000 7fd177fff700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd178109860 0x7fd17819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd176ffd700 1 -- 172.21.15.39:0/2244833583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd178109860 msgr2=0x7fd17819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd176ffd700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd178109860 0x7fd17819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd176ffd700 1 -- 172.21.15.39:0/2244833583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1781045e0 msgr2=0x7fd17819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd176ffd700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1781045e0 0x7fd17819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd176ffd700 1 -- 172.21.15.39:0/2244833583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17819fb50 con 0x7fd178108f10 2024-05-04T03:06:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd177fff700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd178109860 0x7fd17819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:06:48.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd176ffd700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 0x7fd17819b2d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b580 tx=0x7fd16c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:48.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd174ff9700 1 -- 172.21.15.39:0/2244833583 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00f8a0 con 0x7fd178108f10 2024-05-04T03:06:48.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd174ff9700 1 -- 172.21.15.39:0/2244833583 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd16c004830 con 0x7fd178108f10 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17819fde0 con 0x7fd178108f10 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd174ff9700 1 -- 172.21.15.39:0/2244833583 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c005350 con 0x7fd178108f10 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.806+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a0260 con 0x7fd178108f10 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.807+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1780512c0 con 0x7fd178108f10 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.809+0000 7fd174ff9700 1 -- 172.21.15.39:0/2244833583 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd16c029060 con 0x7fd178108f10 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.809+0000 7fd174ff9700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1600b2880 0x7fd1600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.809+0000 7fd1777fe700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1600b2880 0x7fd1600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.809+0000 7fd174ff9700 1 -- 172.21.15.39:0/2244833583 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd16c0e2c30 con 0x7fd178108f10 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.810+0000 7fd1777fe700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1600b2880 0x7fd1600b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd16800ac50 tx=0x7fd16800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:48.813+0000 7fd174ff9700 1 -- 172.21.15.39:0/2244833583 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd16c013070 con 0x7fd178108f10 2024-05-04T03:06:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.085+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd17802e5f0 con 0x7fd178108f10 2024-05-04T03:06:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.086+0000 7fd174ff9700 1 -- 172.21.15.39:0/2244833583 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fd16c0b1e20 con 0x7fd178108f10 2024-05-04T03:06:49.087 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:06:49.087 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1600b2880 msgr2=0x7fd1600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1600b2880 0x7fd1600b4d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd16800ac50 tx=0x7fd16800a380 comp rx=0 tx=0).stop 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 msgr2=0x7fd17819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 0x7fd17819b2d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00b580 tx=0x7fd16c004790 comp rx=0 tx=0).stop 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 shutdown_connections 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1600b2880 0x7fd1600b4d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd178109860 0x7fd17819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1781045e0 0x7fd17819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:49.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 --2- 172.21.15.39:0/2244833583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd178108f10 0x7fd17819b2d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:49.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 >> 172.21.15.39:0/2244833583 conn(0x7fd1780ffef0 msgr2=0x7fd178102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:49.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 shutdown_connections 2024-05-04T03:06:49.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:49.088+0000 7fd17de97700 1 -- 172.21.15.39:0/2244833583 wait complete. 2024-05-04T03:06:49.092 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 3 2024-05-04T03:06:50.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:50 smithi039 ceph-mon[29715]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:50.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:50 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2244833583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:50 smithi066 ceph-mon[33264]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:50.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:50 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2244833583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:50.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:50 smithi167 ceph-mon[33614]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:50.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:50 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2244833583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:51.149 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:50 smithi039 systemd[1]: Starting Ceph mds.j for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:06:51.413 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:51 smithi039 podman[68315]: 2024-05-04 03:06:51.05201636 +0000 UTC m=+0.018570843 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:06:51.414 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:51 smithi039 podman[68315]: 2024-05-04 03:06:51.169556873 +0000 UTC m=+0.136111395 container create a7f4071649bb588c62874ab4745c1d691283391fc06b68a2e79335c8e19176ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-04T03:06:51.414 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:51 smithi039 podman[68315]: 2024-05-04 03:06:51.414277859 +0000 UTC m=+0.380832340 container init a7f4071649bb588c62874ab4745c1d691283391fc06b68a2e79335c8e19176ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0) 2024-05-04T03:06:51.690 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:51 smithi039 podman[68315]: 2024-05-04 03:06:51.417104 +0000 UTC m=+0.383658481 container start a7f4071649bb588c62874ab4745c1d691283391fc06b68a2e79335c8e19176ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-04T03:06:51.690 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:51 smithi039 ceph-mds[68334]: -- 172.21.15.39:0/749737942 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a77bf1a40 con 0x560a77c39600 2024-05-04T03:06:51.690 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:51 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j[68330]: starting mds.j at 2024-05-04T03:06:51.690 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:51 smithi039 bash[68315]: a7f4071649bb588c62874ab4745c1d691283391fc06b68a2e79335c8e19176ca 2024-05-04T03:06:51.691 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:06:51 smithi039 systemd[1]: Started Ceph mds.j for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:06:51.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:06:52.117 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:06:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:52 smithi039 ceph-mon[29715]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:52.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:52.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:52 smithi066 ceph-mon[33264]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:52.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:52 smithi167 ceph-mon[33614]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:53.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:53 smithi066 ceph-mon[33264]: Deploying daemon mds.d on smithi039 2024-05-04T03:06:53.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:53 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.39:6834/3587796449,v1:172.21.15.39:6835/3587796449] up:boot 2024-05-04T03:06:53.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:53 smithi066 ceph-mon[33264]: fsmap 3 up:standby 2024-05-04T03:06:53.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-04T03:06:53.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:53 smithi039 ceph-mon[29715]: Deploying daemon mds.d on smithi039 2024-05-04T03:06:53.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:53 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.39:6834/3587796449,v1:172.21.15.39:6835/3587796449] up:boot 2024-05-04T03:06:53.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:53 smithi039 ceph-mon[29715]: fsmap 3 up:standby 2024-05-04T03:06:53.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-04T03:06:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:53 smithi167 ceph-mon[33614]: Deploying daemon mds.d on smithi039 2024-05-04T03:06:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:53 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.39:6834/3587796449,v1:172.21.15.39:6835/3587796449] up:boot 2024-05-04T03:06:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:53 smithi167 ceph-mon[33614]: fsmap 3 up:standby 2024-05-04T03:06:53.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-04T03:06:54.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:54 smithi066 ceph-mon[33264]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:54.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:54 smithi167 ceph-mon[33614]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:54.370 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:54 smithi039 ceph-mon[29715]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:54.690 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:54 smithi039 systemd[1]: Starting Ceph mds.d for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:06:55.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 -- 172.21.15.39:0/2669640752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 msgr2=0x7f3270109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:55.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 --2- 172.21.15.39:0/2669640752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 0x7f3270109320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f32580099c0 tx=0x7f3258009cd0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 -- 172.21.15.39:0/2669640752 shutdown_connections 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 --2- 172.21.15.39:0/2669640752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3270109860 0x7f32701111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 --2- 172.21.15.39:0/2669640752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32701045e0 0x7f32701049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 --2- 172.21.15.39:0/2669640752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 0x7f3270109320 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 -- 172.21.15.39:0/2669640752 >> 172.21.15.39:0/2669640752 conn(0x7f32700ffe70 msgr2=0x7f32701022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 -- 172.21.15.39:0/2669640752 shutdown_connections 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 -- 172.21.15.39:0/2669640752 wait complete. 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 Processor -- start 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.171+0000 7f32764fb700 1 -- start start 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f32764fb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32701045e0 0x7f327019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:55.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f32764fb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 0x7f327019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f32764fb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3270109860 0x7f327019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f32764fb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3270114320 con 0x7f3270108f10 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f32764fb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f32701141a0 con 0x7f32701045e0 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f32764fb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32701144a0 con 0x7f3270109860 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 0x7f327019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 0x7f327019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48976/0 (socket says 172.21.15.39:48976) 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326f7fe700 1 -- 172.21.15.39:0/999753202 learned_addr learned my addr 172.21.15.39:0/999753202 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f3274a98700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3270109860 0x7f327019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326ffff700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32701045e0 0x7f327019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326f7fe700 1 -- 172.21.15.39:0/999753202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3270109860 msgr2=0x7f327019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326f7fe700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3270109860 0x7f327019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326f7fe700 1 -- 172.21.15.39:0/999753202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32701045e0 msgr2=0x7f327019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326f7fe700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32701045e0 0x7f327019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.172+0000 7f326f7fe700 1 -- 172.21.15.39:0/999753202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327019fc20 con 0x7f3270108f10 2024-05-04T03:06:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.173+0000 7f326f7fe700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 0x7f327019b2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32580045e0 tx=0x7f3258004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.173+0000 7f326d7fa700 1 -- 172.21.15.39:0/999753202 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258026070 con 0x7f3270108f10 2024-05-04T03:06:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.173+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327019feb0 con 0x7f3270108f10 2024-05-04T03:06:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.173+0000 7f326d7fa700 1 -- 172.21.15.39:0/999753202 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f325800f460 con 0x7f3270108f10 2024-05-04T03:06:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.173+0000 7f326d7fa700 1 -- 172.21.15.39:0/999753202 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258009e60 con 0x7f3270108f10 2024-05-04T03:06:55.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.173+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32701a0390 con 0x7f3270108f10 2024-05-04T03:06:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.174+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32700512c0 con 0x7f3270108f10 2024-05-04T03:06:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.175+0000 7f326d7fa700 1 -- 172.21.15.39:0/999753202 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3258029070 con 0x7f3270108f10 2024-05-04T03:06:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.176+0000 7f326d7fa700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f325c0b2880 0x7f325c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:06:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.176+0000 7f326ffff700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f325c0b2880 0x7f325c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:06:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.176+0000 7f326d7fa700 1 -- 172.21.15.39:0/999753202 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f32580e24b0 con 0x7f3270108f10 2024-05-04T03:06:55.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.177+0000 7f326ffff700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f325c0b2880 0x7f325c0b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3260005950 tx=0x7f326000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:06:55.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.181+0000 7f326d7fa700 1 -- 172.21.15.39:0/999753202 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3258013070 con 0x7f3270108f10 2024-05-04T03:06:55.331 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:55 smithi039 podman[68743]: 2024-05-04 03:06:55.205755002 +0000 UTC m=+0.250734528 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:06:55.331 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:55 smithi039 podman[68743]: 2024-05-04 03:06:55.331974754 +0000 UTC m=+0.376954286 container create 712b077342d1d79fb3b16319996f78e88e9f49fbf4861fdc675d3353509c0bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:06:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.458+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f327002e5f0 con 0x7f3270108f10 2024-05-04T03:06:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.459+0000 7f326d7fa700 1 -- 172.21.15.39:0/999753202 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f32580aa700 con 0x7f3270108f10 2024-05-04T03:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:06:55.459 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-04T03:06:55.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f325c0b2880 msgr2=0x7f325c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f325c0b2880 0x7f325c0b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3260005950 tx=0x7f326000b410 comp rx=0 tx=0).stop 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 msgr2=0x7f327019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 0x7f327019b2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32580045e0 tx=0x7f3258004610 comp rx=0 tx=0).stop 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 shutdown_connections 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f325c0b2880 0x7f325c0b4d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3270109860 0x7f327019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32701045e0 0x7f327019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 --2- 172.21.15.39:0/999753202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3270108f10 0x7f327019b2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 >> 172.21.15.39:0/999753202 conn(0x7f32700ffe70 msgr2=0x7f3270102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 shutdown_connections 2024-05-04T03:06:55.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:06:55.461+0000 7f32764fb700 1 -- 172.21.15.39:0/999753202 wait complete. 2024-05-04T03:06:55.463 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 4 2024-05-04T03:06:55.615 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:55 smithi039 podman[68743]: 2024-05-04 03:06:55.577087031 +0000 UTC m=+0.622066566 container init 712b077342d1d79fb3b16319996f78e88e9f49fbf4861fdc675d3353509c0bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:06:55.615 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:55 smithi039 podman[68743]: 2024-05-04 03:06:55.579903474 +0000 UTC m=+0.624883002 container start 712b077342d1d79fb3b16319996f78e88e9f49fbf4861fdc675d3353509c0bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:06:55.940 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:55 smithi039 ceph-mds[68797]: -- 172.21.15.39:0/1445285365 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d03529da40 con 0x55d0352e5180 2024-05-04T03:06:55.940 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:55 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d[68780]: starting mds.d at 2024-05-04T03:06:55.940 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:55 smithi039 bash[68743]: 712b077342d1d79fb3b16319996f78e88e9f49fbf4861fdc675d3353509c0bc8 2024-05-04T03:06:55.941 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:06:55 smithi039 systemd[1]: Started Ceph mds.d for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:06:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:56 smithi066 ceph-mon[33264]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:56 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/999753202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:56.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:56.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:56 smithi039 ceph-mon[29715]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:56.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:56 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/999753202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:56.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:56.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:56.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:56 smithi167 ceph-mon[33614]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:56.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:56 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/999753202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:06:56.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:56.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:56.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:56.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:57.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:57 smithi066 ceph-mon[33264]: Deploying daemon mds.e on smithi066 2024-05-04T03:06:57.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:57 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.39:6836/2949312289,v1:172.21.15.39:6837/2949312289] up:boot 2024-05-04T03:06:57.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:57 smithi066 ceph-mon[33264]: fsmap 4 up:standby 2024-05-04T03:06:57.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:57 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-04T03:06:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:57 smithi167 ceph-mon[33614]: Deploying daemon mds.e on smithi066 2024-05-04T03:06:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:57 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.39:6836/2949312289,v1:172.21.15.39:6837/2949312289] up:boot 2024-05-04T03:06:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:57 smithi167 ceph-mon[33614]: fsmap 4 up:standby 2024-05-04T03:06:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:57 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-04T03:06:57.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:57 smithi039 ceph-mon[29715]: Deploying daemon mds.e on smithi066 2024-05-04T03:06:57.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:57 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.39:6836/2949312289,v1:172.21.15.39:6837/2949312289] up:boot 2024-05-04T03:06:57.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:57 smithi039 ceph-mon[29715]: fsmap 4 up:standby 2024-05-04T03:06:57.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:57 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-04T03:06:58.067 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:57 smithi066 systemd[1]: Starting Ceph mds.e for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:06:58.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:58 smithi039 ceph-mon[29715]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:58.330 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:58 smithi066 ceph-mon[33264]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:58.330 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:58 smithi066 podman[63216]: 2024-05-04 03:06:58.179697214 +0000 UTC m=+0.109220865 container create 2ac28a1a403f299934fafdc974d4f80e9ceec66b990a8798b74bd4fa1d0e4159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-04T03:06:58.330 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:58 smithi066 podman[63216]: 2024-05-04 03:06:58.089150461 +0000 UTC m=+0.018674113 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:06:58.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:06:58.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:58 smithi167 ceph-mon[33614]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:06:58.817 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:58 smithi066 podman[63216]: 2024-05-04 03:06:58.432627536 +0000 UTC m=+0.362151187 container init 2ac28a1a403f299934fafdc974d4f80e9ceec66b990a8798b74bd4fa1d0e4159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:06:58.817 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:58 smithi066 podman[63216]: 2024-05-04 03:06:58.435314107 +0000 UTC m=+0.364837757 container start 2ac28a1a403f299934fafdc974d4f80e9ceec66b990a8798b74bd4fa1d0e4159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:06:58.817 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:58 smithi066 ceph-mds[63235]: -- 172.21.15.66:0/1836871998 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597b9fa1a40 con 0x5597b9fe9180 2024-05-04T03:06:58.817 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e[63231]: starting mds.e at 2024-05-04T03:06:58.817 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:58 smithi066 bash[63216]: 2ac28a1a403f299934fafdc974d4f80e9ceec66b990a8798b74bd4fa1d0e4159 2024-05-04T03:06:58.817 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:06:58 smithi066 systemd[1]: Started Ceph mds.e for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:06:59.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: Deploying daemon mds.f on smithi167 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] up:boot 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: fsmap 5 up:standby 2024-05-04T03:06:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:06:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:06:59.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:59.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:59.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:06:59.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:06:59.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:06:59.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:06:59.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: Deploying daemon mds.f on smithi167 2024-05-04T03:06:59.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:06:59.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] up:boot 2024-05-04T03:06:59.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: fsmap 5 up:standby 2024-05-04T03:06:59.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:06:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:07:00.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:00.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:00.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:00.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:00.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:00.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:00.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: Deploying daemon mds.f on smithi167 2024-05-04T03:07:00.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:00.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] up:boot 2024-05-04T03:07:00.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: fsmap 5 up:standby 2024-05-04T03:07:00.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:06:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:07:00.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 -- 172.21.15.39:0/531828186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a81034c0 msgr2=0x7ff6a81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:00.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/531828186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a81034c0 0x7ff6a81038d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff69800b0d0 tx=0x7ff69800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 -- 172.21.15.39:0/531828186 shutdown_connections 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/531828186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a81034c0 0x7ff6a81038d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/531828186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6a8108740 0x7ff6a81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/531828186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a8107df0 0x7ff6a8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 -- 172.21.15.39:0/531828186 >> 172.21.15.39:0/531828186 conn(0x7ff6a80fedf0 msgr2=0x7ff6a8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 -- 172.21.15.39:0/531828186 shutdown_connections 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.369+0000 7ff6b04e6700 1 -- 172.21.15.39:0/531828186 wait complete. 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.370+0000 7ff6b04e6700 1 Processor -- start 2024-05-04T03:07:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.370+0000 7ff6b04e6700 1 -- start start 2024-05-04T03:07:00.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6b04e6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6a81034c0 0x7ff6a819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:00.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6b04e6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a8107df0 0x7ff6a819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:00.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6b04e6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a8108740 0x7ff6a8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:00.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6b04e6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8113240 con 0x7ff6a8108740 2024-05-04T03:07:00.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6b04e6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81130c0 con 0x7ff6a81034c0 2024-05-04T03:07:00.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6b04e6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a81133c0 con 0x7ff6a8107df0 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a8108740 0x7ff6a8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6ae282700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6a81034c0 0x7ff6a819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6ada81700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a8107df0 0x7ff6a819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a8108740 0x7ff6a8199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51394/0 (socket says 172.21.15.39:51394) 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 -- 172.21.15.39:0/2269682875 learned_addr learned my addr 172.21.15.39:0/2269682875 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6ae282700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6a81034c0 0x7ff6a819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:45452/0 (socket says 172.21.15.39:45452) 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 -- 172.21.15.39:0/2269682875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a8107df0 msgr2=0x7ff6a819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a8107df0 0x7ff6a819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 -- 172.21.15.39:0/2269682875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6a81034c0 msgr2=0x7ff6a819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6a81034c0 0x7ff6a819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 -- 172.21.15.39:0/2269682875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a819a130 con 0x7ff6a8108740 2024-05-04T03:07:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.371+0000 7ff6aea83700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a8108740 0x7ff6a8199a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000d980 tx=0x7ff6a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.372+0000 7ff6a77fe700 1 -- 172.21.15.39:0/2269682875 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0017070 con 0x7ff6a8108740 2024-05-04T03:07:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.372+0000 7ff6a77fe700 1 -- 172.21.15.39:0/2269682875 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6a0005020 con 0x7ff6a8108740 2024-05-04T03:07:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.372+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a819a420 con 0x7ff6a8108740 2024-05-04T03:07:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.372+0000 7ff6a77fe700 1 -- 172.21.15.39:0/2269682875 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000f980 con 0x7ff6a8108740 2024-05-04T03:07:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.372+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a81a66f0 con 0x7ff6a8108740 2024-05-04T03:07:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.373+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a80512c0 con 0x7ff6a8108740 2024-05-04T03:07:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.375+0000 7ff6a77fe700 1 -- 172.21.15.39:0/2269682875 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff6a0004a30 con 0x7ff6a8108740 2024-05-04T03:07:00.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.376+0000 7ff6a77fe700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff6900b2880 0x7ff6900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:00.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.376+0000 7ff6ae282700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff6900b2880 0x7ff6900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:00.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.376+0000 7ff6a77fe700 1 -- 172.21.15.39:0/2269682875 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff6a00e1ef0 con 0x7ff6a8108740 2024-05-04T03:07:00.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.376+0000 7ff6ae282700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff6900b2880 0x7ff6900b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff698009fd0 tx=0x7ff69800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.380+0000 7ff6a77fe700 1 -- 172.21.15.39:0/2269682875 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff6a0020020 con 0x7ff6a8108740 2024-05-04T03:07:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.645+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6a802e5f0 con 0x7ff6a8108740 2024-05-04T03:07:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.645+0000 7ff6a77fe700 1 -- 172.21.15.39:0/2269682875 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7ff6a00aa0b0 con 0x7ff6a8108740 2024-05-04T03:07:00.646 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:00.646 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-04T03:07:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff6900b2880 msgr2=0x7ff6900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff6900b2880 0x7ff6900b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff698009fd0 tx=0x7ff69800b040 comp rx=0 tx=0).stop 2024-05-04T03:07:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a8108740 msgr2=0x7ff6a8199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a8108740 0x7ff6a8199a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000d980 tx=0x7ff6a000dc90 comp rx=0 tx=0).stop 2024-05-04T03:07:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 shutdown_connections 2024-05-04T03:07:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff6900b2880 0x7ff6900b4d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6a8108740 0x7ff6a8199a80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff6a81034c0 0x7ff6a819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 --2- 172.21.15.39:0/2269682875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a8107df0 0x7ff6a819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 >> 172.21.15.39:0/2269682875 conn(0x7ff6a80fedf0 msgr2=0x7ff6a8101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 shutdown_connections 2024-05-04T03:07:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:00.648+0000 7ff6b04e6700 1 -- 172.21.15.39:0/2269682875 wait complete. 2024-05-04T03:07:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 6 2024-05-04T03:07:00.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:00 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2269682875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:00.927 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:00 smithi167 systemd[1]: Starting Ceph mds.f for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:07:00.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:00 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2269682875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:01.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:00 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2269682875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:01.220 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:00 smithi167 podman[64448]: 2024-05-04 03:07:00.866278482 +0000 UTC m=+0.018203366 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:07:01.220 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:00 smithi167 podman[64448]: 2024-05-04 03:07:00.99322743 +0000 UTC m=+0.145152313 container create 4c4e8da1ce11ae0af13d91cc17ebd5a906ade5ce31c9ff4d09db574e8d4e2ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-05-04T03:07:01.676 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:01 smithi167 podman[64448]: 2024-05-04 03:07:01.220775065 +0000 UTC m=+0.372699953 container init 4c4e8da1ce11ae0af13d91cc17ebd5a906ade5ce31c9ff4d09db574e8d4e2ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-04T03:07:01.676 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:01 smithi167 podman[64448]: 2024-05-04 03:07:01.223748395 +0000 UTC m=+0.375673287 container start 4c4e8da1ce11ae0af13d91cc17ebd5a906ade5ce31c9ff4d09db574e8d4e2ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True) 2024-05-04T03:07:01.676 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:01 smithi167 ceph-mds[64467]: -- 172.21.15.167:0/738959799 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc4dc7ba40 con 0x55cc4dcc3a80 2024-05-04T03:07:01.676 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:01 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f[64463]: starting mds.f at 2024-05-04T03:07:01.676 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:01 smithi167 bash[64448]: 4c4e8da1ce11ae0af13d91cc17ebd5a906ade5ce31c9ff4d09db574e8d4e2ba8 2024-05-04T03:07:01.676 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:07:01 smithi167 systemd[1]: Started Ceph mds.f for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:07:02.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: Deploying daemon mds.c on smithi167 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] up:boot 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: fsmap 6 up:standby 2024-05-04T03:07:02.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:07:02.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:02.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: Deploying daemon mds.c on smithi167 2024-05-04T03:07:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] up:boot 2024-05-04T03:07:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: fsmap 6 up:standby 2024-05-04T03:07:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:07:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: Deploying daemon mds.c on smithi167 2024-05-04T03:07:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] up:boot 2024-05-04T03:07:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: fsmap 6 up:standby 2024-05-04T03:07:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:07:03.354 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:07:03.617 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:03.676 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:03 smithi167 systemd[1]: Starting Ceph mds.c for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:07:03.992 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:03 smithi167 podman[64763]: 2024-05-04 03:07:03.637422139 +0000 UTC m=+0.030130162 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:07:03.992 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:03 smithi167 podman[64763]: 2024-05-04 03:07:03.739948312 +0000 UTC m=+0.132656337 container create c63b4a6b05de4274b4e876e8a7a81b30497dea6cec76f35963f541892e3e8de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:07:03.993 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:03 smithi167 podman[64763]: 2024-05-04 03:07:03.993026129 +0000 UTC m=+0.385734153 container init c63b4a6b05de4274b4e876e8a7a81b30497dea6cec76f35963f541892e3e8de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0) 2024-05-04T03:07:04.369 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:04 smithi039 ceph-mon[29715]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:04.369 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.369 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.369 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.369 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:04.369 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:04.369 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:04.370 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:03 smithi167 podman[64763]: 2024-05-04 03:07:03.995691186 +0000 UTC m=+0.388399219 container start c63b4a6b05de4274b4e876e8a7a81b30497dea6cec76f35963f541892e3e8de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-05-04T03:07:04.370 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-mds[64783]: -- 172.21.15.167:0/3530107583 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a767b37a40 con 0x55a767b7fa80 2024-05-04T03:07:04.370 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c[64779]: starting mds.c at 2024-05-04T03:07:04.370 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:04 smithi167 bash[64763]: c63b4a6b05de4274b4e876e8a7a81b30497dea6cec76f35963f541892e3e8de2 2024-05-04T03:07:04.370 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:07:04 smithi167 systemd[1]: Started Ceph mds.c for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:07:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-mon[33614]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:04.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:04 smithi066 ceph-mon[33264]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:04.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:04.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:04.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:04.814 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:05.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 -- 172.21.15.39:0/3876774888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc274108740 msgr2=0x7fc2741100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 --2- 172.21.15.39:0/3876774888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc274108740 0x7fc2741100a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00b0d0 tx=0x7fc26c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 -- 172.21.15.39:0/3876774888 shutdown_connections 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 --2- 172.21.15.39:0/3876774888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc274108740 0x7fc2741100a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 --2- 172.21.15.39:0/3876774888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2741034c0 0x7fc2741038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 --2- 172.21.15.39:0/3876774888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc274107df0 0x7fc274108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 -- 172.21.15.39:0/3876774888 >> 172.21.15.39:0/3876774888 conn(0x7fc2740fedc0 msgr2=0x7fc2741011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 -- 172.21.15.39:0/3876774888 shutdown_connections 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.363+0000 7fc27925f700 1 -- 172.21.15.39:0/3876774888 wait complete. 2024-05-04T03:07:05.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.364+0000 7fc27925f700 1 Processor -- start 2024-05-04T03:07:05.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27925f700 1 -- start start 2024-05-04T03:07:05.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27925f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2741034c0 0x7fc27419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27925f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc274107df0 0x7fc27419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27925f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc274108740 0x7fc2741999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27925f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc274113220 con 0x7fc2741034c0 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27925f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741130a0 con 0x7fc274107df0 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27925f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741133a0 con 0x7fc274108740 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc272d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2741034c0 0x7fc27419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc272d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2741034c0 0x7fc27419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51408/0 (socket says 172.21.15.39:51408) 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc272d9d700 1 -- 172.21.15.39:0/1762996823 learned_addr learned my addr 172.21.15.39:0/1762996823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27259c700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc274107df0 0x7fc27419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:05.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.365+0000 7fc27359e700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc274108740 0x7fc2741999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:05.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc272d9d700 1 -- 172.21.15.39:0/1762996823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc274108740 msgr2=0x7fc2741999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:05.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc272d9d700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc274108740 0x7fc2741999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc272d9d700 1 -- 172.21.15.39:0/1762996823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc274107df0 msgr2=0x7fc27419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:05.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc272d9d700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc274107df0 0x7fc27419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc272d9d700 1 -- 172.21.15.39:0/1762996823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27419a090 con 0x7fc2741034c0 2024-05-04T03:07:05.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc27259c700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc274107df0 0x7fc27419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:07:05.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc272d9d700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2741034c0 0x7fc27419f7c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc264007b50 tx=0x7fc264007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:05.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc25bfff700 1 -- 172.21.15.39:0/1762996823 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264019020 con 0x7fc2741034c0 2024-05-04T03:07:05.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc25bfff700 1 -- 172.21.15.39:0/1762996823 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc26400ce50 con 0x7fc2741034c0 2024-05-04T03:07:05.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27419a380 con 0x7fc2741034c0 2024-05-04T03:07:05.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc25bfff700 1 -- 172.21.15.39:0/1762996823 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26400e580 con 0x7fc2741034c0 2024-05-04T03:07:05.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.366+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27406b610 con 0x7fc2741034c0 2024-05-04T03:07:05.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.367+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc274104970 con 0x7fc2741034c0 2024-05-04T03:07:05.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.369+0000 7fc25bfff700 1 -- 172.21.15.39:0/1762996823 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc264004750 con 0x7fc2741034c0 2024-05-04T03:07:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.370+0000 7fc25bfff700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc25c0b2880 0x7fc25c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.370+0000 7fc27259c700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc25c0b2880 0x7fc25c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.370+0000 7fc25bfff700 1 -- 172.21.15.39:0/1762996823 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fc2640e3750 con 0x7fc2741034c0 2024-05-04T03:07:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.370+0000 7fc27259c700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc25c0b2880 0x7fc25c0b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc274102360 tx=0x7fc26800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:05.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.373+0000 7fc25bfff700 1 -- 172.21.15.39:0/1762996823 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc264023020 con 0x7fc2741034c0 2024-05-04T03:07:05.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.636+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2740512c0 con 0x7fc2741034c0 2024-05-04T03:07:05.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:05 smithi039 ceph-mon[29715]: Deploying daemon mds.h on smithi066 2024-05-04T03:07:05.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:05 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] up:boot 2024-05-04T03:07:05.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:05 smithi039 ceph-mon[29715]: fsmap 7 up:standby 2024-05-04T03:07:05.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:07:05.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.637+0000 7fc25bfff700 1 -- 172.21.15.39:0/1762996823 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fc2640ab9a0 con 0x7fc2741034c0 2024-05-04T03:07:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:05.638 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-04T03:07:05.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc25c0b2880 msgr2=0x7fc25c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc25c0b2880 0x7fc25c0b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc274102360 tx=0x7fc26800b410 comp rx=0 tx=0).stop 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2741034c0 msgr2=0x7fc27419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2741034c0 0x7fc27419f7c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc264007b50 tx=0x7fc264007e60 comp rx=0 tx=0).stop 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 shutdown_connections 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc25c0b2880 0x7fc25c0b4d90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2741034c0 0x7fc27419f7c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc274108740 0x7fc2741999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 --2- 172.21.15.39:0/1762996823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc274107df0 0x7fc27419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 >> 172.21.15.39:0/1762996823 conn(0x7fc2740fedc0 msgr2=0x7fc274107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 shutdown_connections 2024-05-04T03:07:05.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:05.639+0000 7fc27925f700 1 -- 172.21.15.39:0/1762996823 wait complete. 2024-05-04T03:07:05.641 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-05-04T03:07:05.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:05 smithi167 ceph-mon[33614]: Deploying daemon mds.h on smithi066 2024-05-04T03:07:05.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:05 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] up:boot 2024-05-04T03:07:05.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:05 smithi167 ceph-mon[33614]: fsmap 7 up:standby 2024-05-04T03:07:05.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:07:05.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:05 smithi066 ceph-mon[33264]: Deploying daemon mds.h on smithi066 2024-05-04T03:07:05.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:05 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] up:boot 2024-05-04T03:07:05.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:05 smithi066 ceph-mon[33264]: fsmap 7 up:standby 2024-05-04T03:07:05.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:07:06.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:06 smithi066 ceph-mon[33264]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:06.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:06 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1762996823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:06.567 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:06 smithi066 systemd[1]: Starting Ceph mds.h for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:07:06.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:06 smithi039 ceph-mon[29715]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:06.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:06 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1762996823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:06.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:06 smithi167 ceph-mon[33614]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:06.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:06 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1762996823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:06.937 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:06 smithi066 podman[63526]: 2024-05-04 03:07:06.570847368 +0000 UTC m=+0.018290818 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:07:06.938 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:06 smithi066 podman[63526]: 2024-05-04 03:07:06.685161948 +0000 UTC m=+0.132605403 container create 3405fdf73df94a03f86b826c86ec0d44858bcbbaf5dc7c020f14f51146282b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-04T03:07:07.317 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:06 smithi066 podman[63526]: 2024-05-04 03:07:06.938152092 +0000 UTC m=+0.385595544 container init 3405fdf73df94a03f86b826c86ec0d44858bcbbaf5dc7c020f14f51146282b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-04T03:07:07.317 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:06 smithi066 podman[63526]: 2024-05-04 03:07:06.940989667 +0000 UTC m=+0.388433119 container start 3405fdf73df94a03f86b826c86ec0d44858bcbbaf5dc7c020f14f51146282b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux ) 2024-05-04T03:07:07.317 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:06 smithi066 ceph-mds[63545]: -- 172.21.15.66:0/3798329361 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfcf9dba40 con 0x55cfcfa23180 2024-05-04T03:07:07.317 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:06 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h[63541]: starting mds.h at 2024-05-04T03:07:07.317 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:07 smithi066 bash[63526]: 3405fdf73df94a03f86b826c86ec0d44858bcbbaf5dc7c020f14f51146282b49 2024-05-04T03:07:07.317 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:07:07 smithi066 systemd[1]: Started Ceph mds.h for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:07:08.347 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:07:08.376 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: Deploying daemon mds.i on smithi167 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6836/2860496376,v1:172.21.15.66:6837/2860496376] up:boot 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: fsmap 8 up:standby 2024-05-04T03:07:08.377 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: Deploying daemon mds.i on smithi167 2024-05-04T03:07:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6836/2860496376,v1:172.21.15.66:6837/2860496376] up:boot 2024-05-04T03:07:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: fsmap 8 up:standby 2024-05-04T03:07:08.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: Deploying daemon mds.i on smithi167 2024-05-04T03:07:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6836/2860496376,v1:172.21.15.66:6837/2860496376] up:boot 2024-05-04T03:07:08.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: fsmap 8 up:standby 2024-05-04T03:07:08.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:07:08.611 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:09.399 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 systemd[1]: Starting Ceph mds.i for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:07:09.658 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 podman[65074]: 2024-05-04 03:07:09.3024779 +0000 UTC m=+0.018157712 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:07:09.658 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 podman[65074]: 2024-05-04 03:07:09.41182938 +0000 UTC m=+0.127509193 container create bee2384c7f1af00cb8e3bfc3f8a4bdfd574fdf18cbcda55c953321be34e4ddbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-04T03:07:09.658 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 podman[65074]: 2024-05-04 03:07:09.622712585 +0000 UTC m=+0.338392396 container init bee2384c7f1af00cb8e3bfc3f8a4bdfd574fdf18cbcda55c953321be34e4ddbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:07:09.658 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 podman[65074]: 2024-05-04 03:07:09.625212233 +0000 UTC m=+0.340892046 container start bee2384c7f1af00cb8e3bfc3f8a4bdfd574fdf18cbcda55c953321be34e4ddbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:07:09.926 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 ceph-mds[65093]: -- 172.21.15.167:0/1185336374 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5568bb7a9a40 con 0x5568bb7f1600 2024-05-04T03:07:09.926 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i[65089]: starting mds.i at 2024-05-04T03:07:09.926 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 bash[65074]: bee2384c7f1af00cb8e3bfc3f8a4bdfd574fdf18cbcda55c953321be34e4ddbd 2024-05-04T03:07:09.926 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:07:09 smithi167 systemd[1]: Started Ceph mds.i for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:07:10.298 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:10 smithi039 ceph-mon[29715]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:10.298 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.298 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.298 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.298 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:10.299 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:10.299 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 -- 172.21.15.39:0/2702451867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60109880 msgr2=0x7f3b601111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 --2- 172.21.15.39:0/2702451867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60109880 0x7f3b601111e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b500099c0 tx=0x7f3b50009cd0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 -- 172.21.15.39:0/2702451867 shutdown_connections 2024-05-04T03:07:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 --2- 172.21.15.39:0/2702451867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60109880 0x7f3b601111e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 --2- 172.21.15.39:0/2702451867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b60104650 0x7f3b60104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 --2- 172.21.15.39:0/2702451867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b60108f30 0x7f3b60109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 -- 172.21.15.39:0/2702451867 >> 172.21.15.39:0/2702451867 conn(0x7f3b60079f00 msgr2=0x7f3b6007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 -- 172.21.15.39:0/2702451867 shutdown_connections 2024-05-04T03:07:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.416+0000 7f3b682f1700 1 -- 172.21.15.39:0/2702451867 wait complete. 2024-05-04T03:07:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.417+0000 7f3b682f1700 1 Processor -- start 2024-05-04T03:07:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.417+0000 7f3b682f1700 1 -- start start 2024-05-04T03:07:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.417+0000 7f3b682f1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60104650 0x7f3b601a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.417+0000 7f3b682f1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b60108f30 0x7f3b601a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.417+0000 7f3b682f1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b60109880 0x7f3b6019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.417+0000 7f3b682f1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b60114360 con 0x7f3b60104650 2024-05-04T03:07:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.417+0000 7f3b682f1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3b601141e0 con 0x7f3b60108f30 2024-05-04T03:07:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.417+0000 7f3b682f1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b601144e0 con 0x7f3b60109880 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60104650 0x7f3b601a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60104650 0x7f3b601a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48920/0 (socket says 172.21.15.39:48920) 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 -- 172.21.15.39:0/3417546138 learned_addr learned my addr 172.21.15.39:0/3417546138 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6588c700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b60108f30 0x7f3b601a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6688e700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b60109880 0x7f3b6019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 -- 172.21.15.39:0/3417546138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b60109880 msgr2=0x7f3b6019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b60109880 0x7f3b6019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 -- 172.21.15.39:0/3417546138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b60108f30 msgr2=0x7f3b601a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b60108f30 0x7f3b601a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 -- 172.21.15.39:0/3417546138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6019b170 con 0x7f3b60104650 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6588c700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b60108f30 0x7f3b601a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b6608d700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60104650 0x7f3b601a08c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b58007e20 tx=0x7f3b5800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b5f7fe700 1 -- 172.21.15.39:0/3417546138 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58016020 con 0x7f3b60104650 2024-05-04T03:07:10.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b5f7fe700 1 -- 172.21.15.39:0/3417546138 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b5800ae00 con 0x7f3b60104650 2024-05-04T03:07:10.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6019b460 con 0x7f3b60104650 2024-05-04T03:07:10.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b5f7fe700 1 -- 172.21.15.39:0/3417546138 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58011670 con 0x7f3b60104650 2024-05-04T03:07:10.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.418+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b601a7770 con 0x7f3b60104650 2024-05-04T03:07:10.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.419+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b600512c0 con 0x7f3b60104650 2024-05-04T03:07:10.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.421+0000 7f3b5f7fe700 1 -- 172.21.15.39:0/3417546138 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3b58019020 con 0x7f3b60104650 2024-05-04T03:07:10.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.422+0000 7f3b5f7fe700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b480b2900 0x7f3b480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:10.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.422+0000 7f3b6588c700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b480b2900 0x7f3b480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:10.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.422+0000 7f3b5f7fe700 1 -- 172.21.15.39:0/3417546138 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3b580e22b0 con 0x7f3b60104650 2024-05-04T03:07:10.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.422+0000 7f3b6588c700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b480b2900 0x7f3b480b4e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b54005fd0 tx=0x7f3b54005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:10 smithi167 ceph-mon[33614]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.426+0000 7f3b5f7fe700 1 -- 172.21.15.39:0/3417546138 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b580aa470 con 0x7f3b60104650 2024-05-04T03:07:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:10 smithi066 ceph-mon[33264]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.690+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b6019bff0 con 0x7f3b60104650 2024-05-04T03:07:10.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.691+0000 7f3b5f7fe700 1 -- 172.21.15.39:0/3417546138 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f3b580a9d70 con 0x7f3b60104650 2024-05-04T03:07:10.691 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:10.691 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-04T03:07:10.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b480b2900 msgr2=0x7f3b480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b480b2900 0x7f3b480b4e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b54005fd0 tx=0x7f3b54005db0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60104650 msgr2=0x7f3b601a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60104650 0x7f3b601a08c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b58007e20 tx=0x7f3b5800a340 comp rx=0 tx=0).stop 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 shutdown_connections 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b480b2900 0x7f3b480b4e10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b60104650 0x7f3b601a08c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b60109880 0x7f3b6019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 --2- 172.21.15.39:0/3417546138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b60108f30 0x7f3b601a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 >> 172.21.15.39:0/3417546138 conn(0x7f3b60079f00 msgr2=0x7f3b60102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.693+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 shutdown_connections 2024-05-04T03:07:10.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:10.694+0000 7f3b682f1700 1 -- 172.21.15.39:0/3417546138 wait complete. 2024-05-04T03:07:10.695 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 10 2024-05-04T03:07:11.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:11 smithi039 ceph-mon[29715]: Deploying daemon mds.g on smithi039 2024-05-04T03:07:11.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:11 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] up:boot 2024-05-04T03:07:11.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:11 smithi039 ceph-mon[29715]: fsmap 9 up:standby 2024-05-04T03:07:11.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:11 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:07:11.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:11 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3417546138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:11.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:11 smithi167 ceph-mon[33614]: Deploying daemon mds.g on smithi039 2024-05-04T03:07:11.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:11 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] up:boot 2024-05-04T03:07:11.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:11 smithi167 ceph-mon[33614]: fsmap 9 up:standby 2024-05-04T03:07:11.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:11 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:07:11.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:11 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3417546138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:11.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:11 smithi066 ceph-mon[33264]: Deploying daemon mds.g on smithi039 2024-05-04T03:07:11.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:11 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] up:boot 2024-05-04T03:07:11.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:11 smithi066 ceph-mon[33264]: fsmap 9 up:standby 2024-05-04T03:07:11.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:11 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:07:11.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:11 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3417546138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:12.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:12 smithi039 ceph-mon[29715]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:12.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:12 smithi167 ceph-mon[33614]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:12 smithi066 ceph-mon[33264]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:12.940 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:12 smithi039 systemd[1]: Starting Ceph mds.g for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:07:13.337 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:13 smithi039 podman[69584]: 2024-05-04 03:07:12.987793626 +0000 UTC m=+0.018579433 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:07:13.337 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:13 smithi039 podman[69584]: 2024-05-04 03:07:13.0900967 +0000 UTC m=+0.120882540 container create cfacbc7484eee37738833ee0d543b3a37542b460fe83947a00c37e174bb08201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-04T03:07:13.337 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:13 smithi039 podman[69584]: 2024-05-04 03:07:13.334748775 +0000 UTC m=+0.365534570 container init cfacbc7484eee37738833ee0d543b3a37542b460fe83947a00c37e174bb08201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:07:13.337 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:13 smithi039 podman[69584]: 2024-05-04 03:07:13.337499056 +0000 UTC m=+0.368284852 container start cfacbc7484eee37738833ee0d543b3a37542b460fe83947a00c37e174bb08201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-04T03:07:13.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:07:13.676 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:13.690 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:13 smithi039 ceph-mds[69603]: -- 172.21.15.39:0/2727793201 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558648339a40 con 0x558648381600 2024-05-04T03:07:13.690 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g[69599]: starting mds.g at 2024-05-04T03:07:13.690 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:13 smithi039 bash[69584]: cfacbc7484eee37738833ee0d543b3a37542b460fe83947a00c37e174bb08201 2024-05-04T03:07:13.690 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:07:13 smithi039 systemd[1]: Started Ceph mds.g for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:07:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:14 smithi167 ceph-mon[33614]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:14.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:14.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:14 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:14.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:14 smithi039 ceph-mon[29715]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:14.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:14.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:14.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:14 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:14.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:14 smithi066 ceph-mon[33264]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:14.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:14.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:14.571 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:14.571 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:14 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:15.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:15 smithi167 ceph-mon[33614]: Deploying daemon mds.a on smithi039 2024-05-04T03:07:15.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:15 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.39:6838/4175892611,v1:172.21.15.39:6839/4175892611] up:boot 2024-05-04T03:07:15.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:15 smithi167 ceph-mon[33614]: fsmap 10 up:standby 2024-05-04T03:07:15.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:07:15.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:15 smithi039 ceph-mon[29715]: Deploying daemon mds.a on smithi039 2024-05-04T03:07:15.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:15 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.39:6838/4175892611,v1:172.21.15.39:6839/4175892611] up:boot 2024-05-04T03:07:15.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:15 smithi039 ceph-mon[29715]: fsmap 10 up:standby 2024-05-04T03:07:15.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.470+0000 7f90d3b87700 1 -- 172.21.15.39:0/397558047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 msgr2=0x7f90cc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.470+0000 7f90d3b87700 1 --2- 172.21.15.39:0/397558047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 0x7f90cc105890 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90c400b0a0 tx=0x7f90c400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.470+0000 7f90d3b87700 1 -- 172.21.15.39:0/397558047 shutdown_connections 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.470+0000 7f90d3b87700 1 --2- 172.21.15.39:0/397558047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 0x7f90cc105890 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.470+0000 7f90d3b87700 1 --2- 172.21.15.39:0/397558047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90cc10e8f0 0x7f90cc110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.470+0000 7f90d3b87700 1 --2- 172.21.15.39:0/397558047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90cc105dd0 0x7f90cc10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.470+0000 7f90d3b87700 1 -- 172.21.15.39:0/397558047 >> 172.21.15.39:0/397558047 conn(0x7f90cc0fef30 msgr2=0x7f90cc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.471+0000 7f90d3b87700 1 -- 172.21.15.39:0/397558047 shutdown_connections 2024-05-04T03:07:15.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.471+0000 7f90d3b87700 1 -- 172.21.15.39:0/397558047 wait complete. 2024-05-04T03:07:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.471+0000 7f90d3b87700 1 Processor -- start 2024-05-04T03:07:15.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.471+0000 7f90d3b87700 1 -- start start 2024-05-04T03:07:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d3b87700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 0x7f90cc1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d3b87700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90cc105dd0 0x7f90cc1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d3b87700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90cc10e8f0 0x7f90cc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d3b87700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1142f0 con 0x7f90cc105480 2024-05-04T03:07:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d3b87700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc114170 con 0x7f90cc105dd0 2024-05-04T03:07:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d3b87700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc114470 con 0x7f90cc10e8f0 2024-05-04T03:07:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 0x7f90cc1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:15.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 0x7f90cc1a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48968/0 (socket says 172.21.15.39:48968) 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 -- 172.21.15.39:0/3090353195 learned_addr learned my addr 172.21.15.39:0/3090353195 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d2124700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90cc10e8f0 0x7f90cc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1122700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90cc105dd0 0x7f90cc1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 -- 172.21.15.39:0/3090353195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90cc10e8f0 msgr2=0x7f90cc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90cc10e8f0 0x7f90cc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 -- 172.21.15.39:0/3090353195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90cc105dd0 msgr2=0x7f90cc1a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90cc105dd0 0x7f90cc1a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 -- 172.21.15.39:0/3090353195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc19b280 con 0x7f90cc105480 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.472+0000 7f90d1923700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 0x7f90cc1a09c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90c40062a0 tx=0x7f90c4004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.473+0000 7f90caffd700 1 -- 172.21.15.39:0/3090353195 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4021070 con 0x7f90cc105480 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.473+0000 7f90caffd700 1 -- 172.21.15.39:0/3090353195 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90c4007b80 con 0x7f90cc105480 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.473+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc19b510 con 0x7f90cc105480 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.473+0000 7f90caffd700 1 -- 172.21.15.39:0/3090353195 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40052a0 con 0x7f90cc105480 2024-05-04T03:07:15.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.473+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc1a7770 con 0x7f90cc105480 2024-05-04T03:07:15.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.474+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90cc06c920 con 0x7f90cc105480 2024-05-04T03:07:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.475+0000 7f90caffd700 1 -- 172.21.15.39:0/3090353195 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f90c4024070 con 0x7f90cc105480 2024-05-04T03:07:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.476+0000 7f90caffd700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f90b40b2880 0x7f90b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.476+0000 7f90d1122700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f90b40b2880 0x7f90b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.476+0000 7f90caffd700 1 -- 172.21.15.39:0/3090353195 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f90c400f020 con 0x7f90cc105480 2024-05-04T03:07:15.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.476+0000 7f90d1122700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f90b40b2880 0x7f90b40b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f90c0005950 tx=0x7f90c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:15.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.481+0000 7f90caffd700 1 -- 172.21.15.39:0/3090353195 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f90c402b020 con 0x7f90cc105480 2024-05-04T03:07:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:15 smithi066 ceph-mon[33264]: Deploying daemon mds.a on smithi039 2024-05-04T03:07:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:15 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.39:6838/4175892611,v1:172.21.15.39:6839/4175892611] up:boot 2024-05-04T03:07:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:15 smithi066 ceph-mon[33264]: fsmap 10 up:standby 2024-05-04T03:07:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:07:15.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.750+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90cc19bff0 con 0x7f90cc105480 2024-05-04T03:07:15.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.751+0000 7f90caffd700 1 -- 172.21.15.39:0/3090353195 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f90c40ab680 con 0x7f90cc105480 2024-05-04T03:07:15.751 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:15.751 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-04T03:07:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f90b40b2880 msgr2=0x7f90b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f90b40b2880 0x7f90b40b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f90c0005950 tx=0x7f90c00058e0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 msgr2=0x7f90cc1a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 0x7f90cc1a09c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90c40062a0 tx=0x7f90c4004dd0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 shutdown_connections 2024-05-04T03:07:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f90b40b2880 0x7f90b40b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90cc105480 0x7f90cc1a09c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90cc10e8f0 0x7f90cc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 --2- 172.21.15.39:0/3090353195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90cc105dd0 0x7f90cc1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:15.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 >> 172.21.15.39:0/3090353195 conn(0x7f90cc0fef30 msgr2=0x7f90cc0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:15.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 shutdown_connections 2024-05-04T03:07:15.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:15.753+0000 7f90d3b87700 1 -- 172.21.15.39:0/3090353195 wait complete. 2024-05-04T03:07:15.754 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 11 2024-05-04T03:07:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:16 smithi167 ceph-mon[33614]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:16 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3090353195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:16 smithi039 ceph-mon[29715]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:16 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3090353195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:16 smithi066 ceph-mon[33264]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:16 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3090353195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:17.938 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:17 smithi039 systemd[1]: Starting Ceph mds.a for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:07:18.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:18 smithi039 ceph-mon[29715]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:18.190 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:17 smithi039 podman[70052]: 2024-05-04 03:07:17.860976246 +0000 UTC m=+0.018703233 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:07:18.190 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:17 smithi039 podman[70052]: 2024-05-04 03:07:17.989832047 +0000 UTC m=+0.147559022 container create 2d319958b6f3924d18b066ad8542eec1945502ca820fed1c4bef92263bba7bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:07:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:18 smithi167 ceph-mon[33614]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:18 smithi066 ceph-mon[33264]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:18.690 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:18 smithi039 podman[70052]: 2024-05-04 03:07:18.251064838 +0000 UTC m=+0.408791813 container init 2d319958b6f3924d18b066ad8542eec1945502ca820fed1c4bef92263bba7bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-04T03:07:18.690 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:18 smithi039 podman[70052]: 2024-05-04 03:07:18.254120121 +0000 UTC m=+0.411847096 container start 2d319958b6f3924d18b066ad8542eec1945502ca820fed1c4bef92263bba7bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:07:18.690 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:18 smithi039 ceph-mds[70073]: -- 172.21.15.39:0/489385469 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3c0767a40 con 0x55c3c07af600 2024-05-04T03:07:18.690 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:18 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a[70069]: starting mds.a at 2024-05-04T03:07:18.690 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:18 smithi039 bash[70052]: 2d319958b6f3924d18b066ad8542eec1945502ca820fed1c4bef92263bba7bc8 2024-05-04T03:07:18.690 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:07:18 smithi039 systemd[1]: Started Ceph mds.a for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:07:18.837 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:07:19.091 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:19.488 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.488 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.488 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.489 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:19.489 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:19.489 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:19.489 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: Deploying daemon mds.b on smithi066 2024-05-04T03:07:19.489 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.39:6840/1534369514,v1:172.21.15.39:6841/1534369514] up:boot 2024-05-04T03:07:19.489 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: fsmap 11 up:standby 2024-05-04T03:07:19.489 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:19 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:07:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:19.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: Deploying daemon mds.b on smithi066 2024-05-04T03:07:19.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.39:6840/1534369514,v1:172.21.15.39:6841/1534369514] up:boot 2024-05-04T03:07:19.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: fsmap 11 up:standby 2024-05-04T03:07:19.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:19 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:07:19.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:19.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:07:19.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:07:19.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:19.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: Deploying daemon mds.b on smithi066 2024-05-04T03:07:19.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.39:6840/1534369514,v1:172.21.15.39:6841/1534369514] up:boot 2024-05-04T03:07:19.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: fsmap 11 up:standby 2024-05-04T03:07:19.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:19 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:07:20.595 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:20 smithi039 ceph-mon[29715]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:20.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:20 smithi167 ceph-mon[33614]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:20.685 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:20 smithi066 ceph-mon[33264]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 -- 172.21.15.39:0/3427847747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 msgr2=0x7f23681100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 --2- 172.21.15.39:0/3427847747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 0x7f23681100a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f23580099c0 tx=0x7f2358009cd0 comp rx=0 tx=0).stop 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 -- 172.21.15.39:0/3427847747 shutdown_connections 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 --2- 172.21.15.39:0/3427847747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 0x7f23681100a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 --2- 172.21.15.39:0/3427847747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23681034c0 0x7f23681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 --2- 172.21.15.39:0/3427847747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2368107df0 0x7f2368108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 -- 172.21.15.39:0/3427847747 >> 172.21.15.39:0/3427847747 conn(0x7f23680fedc0 msgr2=0x7f23681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 -- 172.21.15.39:0/3427847747 shutdown_connections 2024-05-04T03:07:20.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.879+0000 7f236f01d700 1 -- 172.21.15.39:0/3427847747 wait complete. 2024-05-04T03:07:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236f01d700 1 Processor -- start 2024-05-04T03:07:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236f01d700 1 -- start start 2024-05-04T03:07:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236f01d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23681034c0 0x7f23680ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236f01d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2368107df0 0x7f23680ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236f01d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 0x7f23680ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236f01d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2368113240 con 0x7f2368108740 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236f01d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23681130c0 con 0x7f23681034c0 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236f01d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23681133c0 con 0x7f2368107df0 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236d5ba700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 0x7f23680ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236d5ba700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 0x7f23680ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44694/0 (socket says 172.21.15.39:44694) 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.880+0000 7f236d5ba700 1 -- 172.21.15.39:0/3809253335 learned_addr learned my addr 172.21.15.39:0/3809253335 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236cdb9700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23681034c0 0x7f23680ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f2367fff700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2368107df0 0x7f23680ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236d5ba700 1 -- 172.21.15.39:0/3809253335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2368107df0 msgr2=0x7f23680ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236d5ba700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2368107df0 0x7f23680ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236d5ba700 1 -- 172.21.15.39:0/3809253335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23681034c0 msgr2=0x7f23680ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236d5ba700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23681034c0 0x7f23680ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236d5ba700 1 -- 172.21.15.39:0/3809253335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23681a9120 con 0x7f2368108740 2024-05-04T03:07:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236d5ba700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 0x7f23680ffd20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2358009990 tx=0x7f23580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:20.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f2365ffb700 1 -- 172.21.15.39:0/3809253335 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800f8a0 con 0x7f2368108740 2024-05-04T03:07:20.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f2365ffb700 1 -- 172.21.15.39:0/3809253335 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2358004830 con 0x7f2368108740 2024-05-04T03:07:20.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23681a93b0 con 0x7f2368108740 2024-05-04T03:07:20.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f2365ffb700 1 -- 172.21.15.39:0/3809253335 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358005350 con 0x7f2368108740 2024-05-04T03:07:20.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.881+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681a9800 con 0x7f2368108740 2024-05-04T03:07:20.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.883+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23680512c0 con 0x7f2368108740 2024-05-04T03:07:20.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.884+0000 7f2365ffb700 1 -- 172.21.15.39:0/3809253335 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2358005530 con 0x7f2368108740 2024-05-04T03:07:20.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.885+0000 7f2365ffb700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f23500b2900 0x7f23500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:20.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.885+0000 7f236cdb9700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f23500b2900 0x7f23500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:20.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.885+0000 7f2365ffb700 1 -- 172.21.15.39:0/3809253335 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f23580e2e20 con 0x7f2368108740 2024-05-04T03:07:20.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.885+0000 7f236cdb9700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f23500b2900 0x7f23500b4e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2360003f90 tx=0x7f2360009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:20.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:20.890+0000 7f2365ffb700 1 -- 172.21.15.39:0/3809253335 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2358013070 con 0x7f2368108740 2024-05-04T03:07:20.940 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:20 smithi066 podman[63835]: 2024-05-04 03:07:20.807099536 +0000 UTC m=+0.018390760 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:07:20.940 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:20 smithi066 podman[63835]: 2024-05-04 03:07:20.940740004 +0000 UTC m=+0.152031242 container create e870791c88c2cac3647aba47ddea47e9d44073b652e1646e99c80bb384b965cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:07:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.157+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f236802e7b0 con 0x7f2368108740 2024-05-04T03:07:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.158+0000 7f2365ffb700 1 -- 172.21.15.39:0/3809253335 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f23580ab070 con 0x7f2368108740 2024-05-04T03:07:21.160 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:21.160 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-04T03:07:21.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.160+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f23500b2900 msgr2=0x7f23500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:21.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.160+0000 7f236f01d700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f23500b2900 0x7f23500b4e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2360003f90 tx=0x7f2360009180 comp rx=0 tx=0).stop 2024-05-04T03:07:21.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.160+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 msgr2=0x7f23680ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:21.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.160+0000 7f236f01d700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 0x7f23680ffd20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2358009990 tx=0x7f23580047b0 comp rx=0 tx=0).stop 2024-05-04T03:07:21.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.161+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 shutdown_connections 2024-05-04T03:07:21.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.161+0000 7f236f01d700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f23500b2900 0x7f23500b4e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:21.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.161+0000 7f236f01d700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2368108740 0x7f23680ffd20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:21.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.161+0000 7f236f01d700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23681034c0 0x7f23680ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:21.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.161+0000 7f236f01d700 1 --2- 172.21.15.39:0/3809253335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2368107df0 0x7f23680ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:21.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.161+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 >> 172.21.15.39:0/3809253335 conn(0x7f23680fedc0 msgr2=0x7f236806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:21.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.161+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 shutdown_connections 2024-05-04T03:07:21.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:21.161+0000 7f236f01d700 1 -- 172.21.15.39:0/3809253335 wait complete. 2024-05-04T03:07:21.164 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 12 2024-05-04T03:07:21.212 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:21 smithi066 podman[63835]: 2024-05-04 03:07:21.176981962 +0000 UTC m=+0.388273160 container init e870791c88c2cac3647aba47ddea47e9d44073b652e1646e99c80bb384b965cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-04T03:07:21.212 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:21 smithi066 podman[63835]: 2024-05-04 03:07:21.179891943 +0000 UTC m=+0.391183147 container start e870791c88c2cac3647aba47ddea47e9d44073b652e1646e99c80bb384b965cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-04T03:07:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:21 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3809253335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:21 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.568 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:21 smithi066 ceph-mds[63854]: -- 172.21.15.66:0/2807067358 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c0fb31a40 con 0x564c0fb79180 2024-05-04T03:07:21.568 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:21 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b[63850]: starting mds.b at 2024-05-04T03:07:21.568 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:21 smithi066 bash[63835]: e870791c88c2cac3647aba47ddea47e9d44073b652e1646e99c80bb384b965cf 2024-05-04T03:07:21.568 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:21 smithi066 systemd[1]: Started Ceph mds.b for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:07:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:21 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3809253335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:21 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:21 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3809253335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:21.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:21.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:21 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:22.661 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:22 smithi066 ceph-mon[33264]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:22.661 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:22 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] up:boot 2024-05-04T03:07:22.661 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:22 smithi066 ceph-mon[33264]: fsmap 12 up:standby 2024-05-04T03:07:22.661 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:22 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:07:22.661 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:22 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:07:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:22 smithi167 ceph-mon[33614]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:22 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] up:boot 2024-05-04T03:07:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:22 smithi167 ceph-mon[33614]: fsmap 12 up:standby 2024-05-04T03:07:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:22 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:07:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:22 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:07:22.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:22 smithi039 ceph-mon[29715]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:22.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:22 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] up:boot 2024-05-04T03:07:22.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:22 smithi039 ceph-mon[29715]: fsmap 12 up:standby 2024-05-04T03:07:22.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:22 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:07:22.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:22 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:07:23.805 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:07:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:24.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:24.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:24.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:07:24.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:24 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:07:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:24 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:07:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:07:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:24 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:07:26.315 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:25 smithi039 ceph-mon[29715]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:26.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:25 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:25 smithi066 ceph-mon[33264]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:25 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:25 smithi167 ceph-mon[33614]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:25 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:26.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 -- 172.21.15.39:0/2103219918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 msgr2=0x7fd3181100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:26.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/2103219918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 0x7fd3181100a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd31000b0d0 tx=0x7fd31000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 -- 172.21.15.39:0/2103219918 shutdown_connections 2024-05-04T03:07:26.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/2103219918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 0x7fd3181100a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/2103219918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3181034c0 0x7fd3181038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/2103219918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318107df0 0x7fd318108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 -- 172.21.15.39:0/2103219918 >> 172.21.15.39:0/2103219918 conn(0x7fd3180fedc0 msgr2=0x7fd3181011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 -- 172.21.15.39:0/2103219918 shutdown_connections 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.430+0000 7fd31d4cd700 1 -- 172.21.15.39:0/2103219918 wait complete. 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.431+0000 7fd31d4cd700 1 Processor -- start 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd31d4cd700 1 -- start start 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd31d4cd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3181034c0 0x7fd31819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd31d4cd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318107df0 0x7fd31819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd31d4cd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 0x7fd31819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd31d4cd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd318113270 con 0x7fd318108740 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd31d4cd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181130f0 con 0x7fd3181034c0 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd31d4cd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181133f0 con 0x7fd318107df0 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 0x7fd31819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:26.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 0x7fd31819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33600/0 (socket says 172.21.15.39:33600) 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 -- 172.21.15.39:0/544965710 learned_addr learned my addr 172.21.15.39:0/544965710 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd316ffd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3181034c0 0x7fd31819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3167fc700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318107df0 0x7fd31819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 -- 172.21.15.39:0/544965710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318107df0 msgr2=0x7fd31819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318107df0 0x7fd31819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 -- 172.21.15.39:0/544965710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3181034c0 msgr2=0x7fd31819bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3181034c0 0x7fd31819bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 -- 172.21.15.39:0/544965710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3181a6db0 con 0x7fd318108740 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd316ffd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3181034c0 0x7fd31819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.432+0000 7fd3177fe700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 0x7fd31819c590 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd310007d10 tx=0x7fd310007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.433+0000 7fd2fffff700 1 -- 172.21.15.39:0/544965710 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31001f020 con 0x7fd318108740 2024-05-04T03:07:26.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.433+0000 7fd2fffff700 1 -- 172.21.15.39:0/544965710 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd310003e10 con 0x7fd318108740 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.433+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3181a7040 con 0x7fd318108740 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.433+0000 7fd2fffff700 1 -- 172.21.15.39:0/544965710 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31001bab0 con 0x7fd318108740 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.433+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3181a74f0 con 0x7fd318108740 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.434+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3180512c0 con 0x7fd318108740 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.435+0000 7fd2fffff700 1 -- 172.21.15.39:0/544965710 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd310022020 con 0x7fd318108740 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.436+0000 7fd2fffff700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd3000b2880 0x7fd3000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.436+0000 7fd316ffd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd3000b2880 0x7fd3000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.436+0000 7fd2fffff700 1 -- 172.21.15.39:0/544965710 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd31000f020 con 0x7fd318108740 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.436+0000 7fd316ffd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd3000b2880 0x7fd3000b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd308005fd0 tx=0x7fd308005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:26.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.439+0000 7fd2fffff700 1 -- 172.21.15.39:0/544965710 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3100abd40 con 0x7fd318108740 2024-05-04T03:07:26.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.722+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd31802e7b0 con 0x7fd318108740 2024-05-04T03:07:26.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.722+0000 7fd2fffff700 1 -- 172.21.15.39:0/544965710 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7fd310009cb0 con 0x7fd318108740 2024-05-04T03:07:26.723 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:26.724 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-04T03:07:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd3000b2880 msgr2=0x7fd3000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd3000b2880 0x7fd3000b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd308005fd0 tx=0x7fd308005e30 comp rx=0 tx=0).stop 2024-05-04T03:07:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 msgr2=0x7fd31819c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:26.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 0x7fd31819c590 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd310007d10 tx=0x7fd310007df0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 shutdown_connections 2024-05-04T03:07:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd3000b2880 0x7fd3000b4d90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd318108740 0x7fd31819c590 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3181034c0 0x7fd31819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 --2- 172.21.15.39:0/544965710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318107df0 0x7fd31819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 >> 172.21.15.39:0/544965710 conn(0x7fd3180fedc0 msgr2=0x7fd318107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 shutdown_connections 2024-05-04T03:07:26.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:26.725+0000 7fd31d4cd700 1 -- 172.21.15.39:0/544965710 wait complete. 2024-05-04T03:07:26.727 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 13 2024-05-04T03:07:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:26 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/544965710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:26 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:27.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:26 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:27.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:26 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/544965710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:27.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:27.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:26 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:27.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-04T03:07:27.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:26 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/544965710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:07:27.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:26 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:27.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:26 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:27.685 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:27 smithi039 ceph-mon[29715]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:07:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:27 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:28.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:27 smithi066 ceph-mon[33264]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:28.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:28.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:07:28.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:27 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:27 smithi167 ceph-mon[33614]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:07:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:27 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:07:29.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 -- 172.21.15.39:0/880481216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c108760 msgr2=0x7f636c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:29.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 --2- 172.21.15.39:0/880481216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c108760 0x7f636c1100c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f636400b0d0 tx=0x7f636400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 -- 172.21.15.39:0/880481216 shutdown_connections 2024-05-04T03:07:29.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 --2- 172.21.15.39:0/880481216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c108760 0x7f636c1100c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 --2- 172.21.15.39:0/880481216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c1034e0 0x7f636c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 --2- 172.21.15.39:0/880481216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c107e10 0x7f636c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 -- 172.21.15.39:0/880481216 >> 172.21.15.39:0/880481216 conn(0x7f636c0fef30 msgr2=0x7f636c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 -- 172.21.15.39:0/880481216 shutdown_connections 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.439+0000 7f6374547700 1 -- 172.21.15.39:0/880481216 wait complete. 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.440+0000 7f6374547700 1 Processor -- start 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.440+0000 7f6374547700 1 -- start start 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.440+0000 7f6374547700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c1034e0 0x7f636c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.440+0000 7f6374547700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c107e10 0x7f636c19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.440+0000 7f6374547700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c108760 0x7f636c1a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f6374547700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f636c113290 con 0x7f636c1034e0 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f6374547700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f636c113110 con 0x7f636c108760 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f6374547700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f636c113410 con 0x7f636c107e10 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c1034e0 0x7f636c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c1034e0 0x7f636c19bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33612/0 (socket says 172.21.15.39:33612) 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 -- 172.21.15.39:0/2462625432 learned_addr learned my addr 172.21.15.39:0/2462625432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f6372ae4700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c108760 0x7f636c1a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f6371ae2700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c107e10 0x7f636c19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 -- 172.21.15.39:0/2462625432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c107e10 msgr2=0x7f636c19c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c107e10 0x7f636c19c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 -- 172.21.15.39:0/2462625432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c108760 msgr2=0x7f636c1a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c108760 0x7f636c1a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 -- 172.21.15.39:0/2462625432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c19c960 con 0x7f636c1034e0 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63722e3700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c1034e0 0x7f636c19bb70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f636800c920 tx=0x7f636800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63637fe700 1 -- 172.21.15.39:0/2462625432 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368004a30 con 0x7f636c1034e0 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c1a7250 con 0x7f636c1034e0 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63637fe700 1 -- 172.21.15.39:0/2462625432 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63680054b0 con 0x7f636c1034e0 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.441+0000 7f63637fe700 1 -- 172.21.15.39:0/2462625432 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368007b70 con 0x7f636c1034e0 2024-05-04T03:07:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.442+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c1a7760 con 0x7f636c1034e0 2024-05-04T03:07:29.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.442+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c0512c0 con 0x7f636c1034e0 2024-05-04T03:07:29.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.444+0000 7f63637fe700 1 -- 172.21.15.39:0/2462625432 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6368007d10 con 0x7f636c1034e0 2024-05-04T03:07:29.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.445+0000 7f63637fe700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f63580b2880 0x7f63580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:29.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.445+0000 7f6371ae2700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f63580b2880 0x7f63580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:29.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.445+0000 7f63637fe700 1 -- 172.21.15.39:0/2462625432 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f63680e1c90 con 0x7f636c1034e0 2024-05-04T03:07:29.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.445+0000 7f6371ae2700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f63580b2880 0x7f63580b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f636c102620 tx=0x7f635c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:29.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.448+0000 7f63637fe700 1 -- 172.21.15.39:0/2462625432 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6368014030 con 0x7f636c1034e0 2024-05-04T03:07:29.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.665+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f636c02e5f0 con 0x7f636c1034e0 2024-05-04T03:07:29.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.665+0000 7f63637fe700 1 -- 172.21.15.39:0/2462625432 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f6368005660 con 0x7f636c1034e0 2024-05-04T03:07:29.666 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:29.666 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-05-04T03:07:29.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:07:29.666 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-05-04T03:07:29.666 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-05-04T03:07:29.666 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:07:29.666 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-05-04T03:07:29.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.667+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f63580b2880 msgr2=0x7f63580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:29.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.667+0000 7f6374547700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f63580b2880 0x7f63580b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f636c102620 tx=0x7f635c00b410 comp rx=0 tx=0).stop 2024-05-04T03:07:29.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.667+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c1034e0 msgr2=0x7f636c19bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:29.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c1034e0 0x7f636c19bb70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f636800c920 tx=0x7f636800cc30 comp rx=0 tx=0).stop 2024-05-04T03:07:29.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 shutdown_connections 2024-05-04T03:07:29.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f63580b2880 0x7f63580b4d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636c1034e0 0x7f636c19bb70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c108760 0x7f636c1a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 --2- 172.21.15.39:0/2462625432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c107e10 0x7f636c19c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:29.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 >> 172.21.15.39:0/2462625432 conn(0x7f636c0fef30 msgr2=0x7f636c107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:29.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 shutdown_connections 2024-05-04T03:07:29.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:29.668+0000 7f6374547700 1 -- 172.21.15.39:0/2462625432 wait complete. 2024-05-04T03:07:30.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:29 smithi066 ceph-mon[33264]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:30.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:29 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2462625432' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-04T03:07:30.363 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:29 smithi039 ceph-mon[29715]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:30.363 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:29 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2462625432' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-04T03:07:30.396 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-04T03:07:30.397 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-04T03:07:30.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:29 smithi167 ceph-mon[33614]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:30.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:29 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2462625432' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-04T03:07:30.651 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:32.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:31 smithi039 ceph-mon[29715]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:32.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:31 smithi066 ceph-mon[33264]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:32.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:31 smithi167 ceph-mon[33614]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 -- 172.21.15.39:0/2858926171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc108740 msgr2=0x7f5fdc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 --2- 172.21.15.39:0/2858926171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc108740 0x7f5fdc1100a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0099c0 tx=0x7f5fcc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 -- 172.21.15.39:0/2858926171 shutdown_connections 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 --2- 172.21.15.39:0/2858926171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc108740 0x7f5fdc1100a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 --2- 172.21.15.39:0/2858926171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fdc1034c0 0x7f5fdc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 --2- 172.21.15.39:0/2858926171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc107df0 0x7f5fdc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 -- 172.21.15.39:0/2858926171 >> 172.21.15.39:0/2858926171 conn(0x7f5fdc0fedc0 msgr2=0x7f5fdc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 -- 172.21.15.39:0/2858926171 shutdown_connections 2024-05-04T03:07:32.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.499+0000 7f5fe1531700 1 -- 172.21.15.39:0/2858926171 wait complete. 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.500+0000 7f5fe1531700 1 Processor -- start 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.500+0000 7f5fe1531700 1 -- start start 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fe1531700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc1034c0 0x7f5fdc110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fe1531700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fdc107df0 0x7f5fdc110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fe1531700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc108740 0x7f5fdc111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fe1531700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc114390 con 0x7f5fdc1034c0 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fe1531700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc114210 con 0x7f5fdc107df0 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fe1531700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc114510 con 0x7f5fdc108740 2024-05-04T03:07:32.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc1034c0 0x7f5fdc110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc1034c0 0x7f5fdc110a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33628/0 (socket says 172.21.15.39:33628) 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 -- 172.21.15.39:0/50436716 learned_addr learned my addr 172.21.15.39:0/50436716 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdb7fe700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc108740 0x7f5fdc111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 -- 172.21.15.39:0/50436716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc108740 msgr2=0x7f5fdc111500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc108740 0x7f5fdc111500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 -- 172.21.15.39:0/50436716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fdc107df0 msgr2=0x7f5fdc110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fda7fc700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fdc107df0 0x7f5fdc110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fdc107df0 0x7f5fdc110fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 -- 172.21.15.39:0/50436716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc1a90c0 con 0x7f5fdc1034c0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.501+0000 7f5fdaffd700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc1034c0 0x7f5fdc110a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40060b0 tx=0x7f5fd4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.502+0000 7f5fd3fff700 1 -- 172.21.15.39:0/50436716 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd401a070 con 0x7f5fdc1034c0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.502+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc1a93b0 con 0x7f5fdc1034c0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.502+0000 7f5fd3fff700 1 -- 172.21.15.39:0/50436716 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fd400de20 con 0x7f5fdc1034c0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.502+0000 7f5fd3fff700 1 -- 172.21.15.39:0/50436716 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4014690 con 0x7f5fdc1034c0 2024-05-04T03:07:32.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.502+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1a98f0 con 0x7f5fdc1034c0 2024-05-04T03:07:32.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.503+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc0512c0 con 0x7f5fdc1034c0 2024-05-04T03:07:32.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.505+0000 7f5fd3fff700 1 -- 172.21.15.39:0/50436716 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5fd401d070 con 0x7f5fdc1034c0 2024-05-04T03:07:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.506+0000 7f5fd3fff700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5fc80b2880 0x7f5fc80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.506+0000 7f5fda7fc700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5fc80b2880 0x7f5fc80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.506+0000 7f5fd3fff700 1 -- 172.21.15.39:0/50436716 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5fd40e1b70 con 0x7f5fdc1034c0 2024-05-04T03:07:32.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.506+0000 7f5fda7fc700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5fc80b2880 0x7f5fc80b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4006fd0 tx=0x7f5fc4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:32.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.510+0000 7f5fd3fff700 1 -- 172.21.15.39:0/50436716 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5fd4010070 con 0x7f5fdc1034c0 2024-05-04T03:07:32.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.722+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5fdc02e7b0 con 0x7f5fdc1034c0 2024-05-04T03:07:32.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:32.997+0000 7f5fd3fff700 1 -- 172.21.15.39:0/50436716 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f5fd40a9d30 con 0x7f5fdc1034c0 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5fc80b2880 msgr2=0x7f5fc80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5fc80b2880 0x7f5fc80b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4006fd0 tx=0x7f5fc4008040 comp rx=0 tx=0).stop 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc1034c0 msgr2=0x7f5fdc110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc1034c0 0x7f5fdc110a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40060b0 tx=0x7f5fd4007f90 comp rx=0 tx=0).stop 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 shutdown_connections 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5fc80b2880 0x7f5fc80b4d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fdc1034c0 0x7f5fdc110a80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc108740 0x7f5fdc111500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 --2- 172.21.15.39:0/50436716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fdc107df0 0x7f5fdc110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 >> 172.21.15.39:0/50436716 conn(0x7f5fdc0fedc0 msgr2=0x7f5fdc1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 shutdown_connections 2024-05-04T03:07:33.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:33.000+0000 7f5fe1531700 1 -- 172.21.15.39:0/50436716 wait complete. 2024-05-04T03:07:33.002 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-05-04T03:07:33.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:32 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/50436716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-04T03:07:33.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:32 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/50436716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-04T03:07:33.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:32 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/50436716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-04T03:07:33.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-04T03:07:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:34.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:33 smithi039 ceph-mon[29715]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:34.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:33 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/50436716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-04T03:07:34.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:33 smithi039 ceph-mon[29715]: osdmap e72: 12 total, 12 up, 12 in 2024-05-04T03:07:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:34 smithi066 ceph-mon[33264]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:34 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/50436716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-04T03:07:34.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:34 smithi066 ceph-mon[33264]: osdmap e72: 12 total, 12 up, 12 in 2024-05-04T03:07:34.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:34 smithi167 ceph-mon[33614]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:34.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:34 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/50436716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-04T03:07:34.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:34 smithi167 ceph-mon[33614]: osdmap e72: 12 total, 12 up, 12 in 2024-05-04T03:07:35.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:35 smithi066 ceph-mon[33264]: osdmap e73: 12 total, 12 up, 12 in 2024-05-04T03:07:35.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:35 smithi039 ceph-mon[29715]: osdmap e73: 12 total, 12 up, 12 in 2024-05-04T03:07:35.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:35 smithi167 ceph-mon[33614]: osdmap e73: 12 total, 12 up, 12 in 2024-05-04T03:07:35.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 -- 172.21.15.39:0/583053549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc0109310 msgr2=0x7f1fc0109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:35.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/583053549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc0109310 0x7f1fc0109720 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009990 tx=0x7f1fb0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 -- 172.21.15.39:0/583053549 shutdown_connections 2024-05-04T03:07:35.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/583053549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc0109310 0x7f1fc0109720 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/583053549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc01030f0 0x7f1fc0111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/583053549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc01027a0 0x7f1fc0102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 -- 172.21.15.39:0/583053549 >> 172.21.15.39:0/583053549 conn(0x7f1fc00fe4f0 msgr2=0x7f1fc0100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:35.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 -- 172.21.15.39:0/583053549 shutdown_connections 2024-05-04T03:07:35.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.780+0000 7f1fc6c15700 1 -- 172.21.15.39:0/583053549 wait complete. 2024-05-04T03:07:35.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc6c15700 1 Processor -- start 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc6c15700 1 -- start start 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc6c15700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc01027a0 0x7f1fc0103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc6c15700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc01030f0 0x7f1fc0103ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc6c15700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc0109310 0x7f1fc0104400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc6c15700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0114450 con 0x7f1fc01030f0 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc6c15700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01142d0 con 0x7f1fc01027a0 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc6c15700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01145d0 con 0x7f1fc0109310 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc01030f0 0x7f1fc0103ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc01030f0 0x7f1fc0103ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36792/0 (socket says 172.21.15.39:36792) 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fbffff700 1 -- 172.21.15.39:0/2562054030 learned_addr learned my addr 172.21.15.39:0/2562054030 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc49b1700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc01027a0 0x7f1fc0103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fc51b2700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc0109310 0x7f1fc0104400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fbffff700 1 -- 172.21.15.39:0/2562054030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc0109310 msgr2=0x7f1fc0104400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:35.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fbffff700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc0109310 0x7f1fc0104400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fbffff700 1 -- 172.21.15.39:0/2562054030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc01027a0 msgr2=0x7f1fc0103980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.781+0000 7f1fbffff700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc01027a0 0x7f1fc0103980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.782+0000 7f1fbffff700 1 -- 172.21.15.39:0/2562054030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc01a91e0 con 0x7f1fc01030f0 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.782+0000 7f1fc51b2700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc0109310 0x7f1fc0104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.782+0000 7f1fbffff700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc01030f0 0x7f1fc0103ec0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c8f0 tx=0x7f1fb400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.782+0000 7f1fbdffb700 1 -- 172.21.15.39:0/2562054030 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4007940 con 0x7f1fc01030f0 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.782+0000 7f1fbdffb700 1 -- 172.21.15.39:0/2562054030 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fb400f450 con 0x7f1fc01030f0 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.782+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc01a94d0 con 0x7f1fc01030f0 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.782+0000 7f1fbdffb700 1 -- 172.21.15.39:0/2562054030 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4018660 con 0x7f1fc01030f0 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.782+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01a9a60 con 0x7f1fc01030f0 2024-05-04T03:07:35.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.783+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc00512c0 con 0x7f1fc01030f0 2024-05-04T03:07:35.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.785+0000 7f1fbdffb700 1 -- 172.21.15.39:0/2562054030 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1fb400f900 con 0x7f1fc01030f0 2024-05-04T03:07:35.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.786+0000 7f1fbdffb700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1fa80b2900 0x7f1fa80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:35.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.786+0000 7f1fc49b1700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1fa80b2900 0x7f1fa80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:35.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.786+0000 7f1fbdffb700 1 -- 172.21.15.39:0/2562054030 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f1fb40e1d70 con 0x7f1fc01030f0 2024-05-04T03:07:35.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.786+0000 7f1fc49b1700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1fa80b2900 0x7f1fa80b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00059d0 tx=0x7f1fb0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:35.791+0000 7f1fbdffb700 1 -- 172.21.15.39:0/2562054030 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1fb4014030 con 0x7f1fc01030f0 2024-05-04T03:07:36.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.005+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1fc0104eb0 con 0x7f1fc01030f0 2024-05-04T03:07:36.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.021+0000 7f1fbdffb700 1 -- 172.21.15.39:0/2562054030 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f1fb40a9dc0 con 0x7f1fc01030f0 2024-05-04T03:07:36.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.023+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1fa80b2900 msgr2=0x7f1fa80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:36.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.023+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1fa80b2900 0x7f1fa80b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00059d0 tx=0x7f1fb0005960 comp rx=0 tx=0).stop 2024-05-04T03:07:36.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.023+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc01030f0 msgr2=0x7f1fc0103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:36.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.023+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc01030f0 0x7f1fc0103ec0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c8f0 tx=0x7f1fb400cc00 comp rx=0 tx=0).stop 2024-05-04T03:07:36.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.024+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 shutdown_connections 2024-05-04T03:07:36.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.024+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1fa80b2900 0x7f1fa80b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.024+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc0109310 0x7f1fc0104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.024+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc01027a0 0x7f1fc0103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.024+0000 7f1fc6c15700 1 --2- 172.21.15.39:0/2562054030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fc01030f0 0x7f1fc0103ec0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.024+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 >> 172.21.15.39:0/2562054030 conn(0x7f1fc00fe4f0 msgr2=0x7f1fc01072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.024+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 shutdown_connections 2024-05-04T03:07:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:36.024+0000 7f1fc6c15700 1 -- 172.21.15.39:0/2562054030 wait complete. 2024-05-04T03:07:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-05-04T03:07:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:36 smithi066 ceph-mon[33264]: pgmap v283: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:36 smithi066 ceph-mon[33264]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:07:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:36 smithi066 ceph-mon[33264]: osdmap e74: 12 total, 12 up, 12 in 2024-05-04T03:07:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:36 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2562054030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:07:36.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:36 smithi039 ceph-mon[29715]: pgmap v283: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:36.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:36 smithi039 ceph-mon[29715]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:07:36.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:36 smithi039 ceph-mon[29715]: osdmap e74: 12 total, 12 up, 12 in 2024-05-04T03:07:36.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:36 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2562054030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:07:36.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:36 smithi167 ceph-mon[33614]: pgmap v283: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:36.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:36 smithi167 ceph-mon[33614]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:07:36.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:36 smithi167 ceph-mon[33614]: osdmap e74: 12 total, 12 up, 12 in 2024-05-04T03:07:36.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:36 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2562054030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:07:36.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-04T03:07:37.008 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:37.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:37 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2562054030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-04T03:07:37.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:37 smithi039 ceph-mon[29715]: osdmap e75: 12 total, 12 up, 12 in 2024-05-04T03:07:37.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:37 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2562054030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-04T03:07:37.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:37 smithi066 ceph-mon[33264]: osdmap e75: 12 total, 12 up, 12 in 2024-05-04T03:07:37.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:37 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2562054030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-04T03:07:37.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:37 smithi167 ceph-mon[33614]: osdmap e75: 12 total, 12 up, 12 in 2024-05-04T03:07:38.174 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:38 smithi039 ceph-mon[29715]: pgmap v286: 97 pgs: 11 creating+peering, 11 active+clean, 75 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:38.174 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:38 smithi039 ceph-mon[29715]: osdmap e76: 12 total, 12 up, 12 in 2024-05-04T03:07:38.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:38 smithi066 ceph-mon[33264]: pgmap v286: 97 pgs: 11 creating+peering, 11 active+clean, 75 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:38.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:38 smithi066 ceph-mon[33264]: osdmap e76: 12 total, 12 up, 12 in 2024-05-04T03:07:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:38 smithi167 ceph-mon[33614]: pgmap v286: 97 pgs: 11 creating+peering, 11 active+clean, 75 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:38 smithi167 ceph-mon[33614]: osdmap e76: 12 total, 12 up, 12 in 2024-05-04T03:07:38.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 -- 172.21.15.39:0/1620773262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf341034c0 msgr2=0x7fdf341038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:38.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1620773262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf341034c0 0x7fdf341038d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdf240099c0 tx=0x7fdf24009cd0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 -- 172.21.15.39:0/1620773262 shutdown_connections 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1620773262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf341034c0 0x7fdf341038d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1620773262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34108740 0x7fdf341100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1620773262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf34107df0 0x7fdf34108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 -- 172.21.15.39:0/1620773262 >> 172.21.15.39:0/1620773262 conn(0x7fdf340fedf0 msgr2=0x7fdf34101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 -- 172.21.15.39:0/1620773262 shutdown_connections 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.813+0000 7fdf3925f700 1 -- 172.21.15.39:0/1620773262 wait complete. 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.814+0000 7fdf3925f700 1 Processor -- start 2024-05-04T03:07:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.814+0000 7fdf3925f700 1 -- start start 2024-05-04T03:07:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.814+0000 7fdf3925f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf341034c0 0x7fdf34199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.814+0000 7fdf3925f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107df0 0x7fdf3419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.814+0000 7fdf3925f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf34108740 0x7fdf3419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.814+0000 7fdf3925f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341132e0 con 0x7fdf34108740 2024-05-04T03:07:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.814+0000 7fdf3925f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34113160 con 0x7fdf341034c0 2024-05-04T03:07:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.814+0000 7fdf3925f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34113460 con 0x7fdf34107df0 2024-05-04T03:07:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf34108740 0x7fdf3419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf34108740 0x7fdf3419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36806/0 (socket says 172.21.15.39:36806) 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 -- 172.21.15.39:0/1083459565 learned_addr learned my addr 172.21.15.39:0/1083459565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf32d9d700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf341034c0 0x7fdf34199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3259c700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107df0 0x7fdf3419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 -- 172.21.15.39:0/1083459565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107df0 msgr2=0x7fdf3419a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107df0 0x7fdf3419a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 -- 172.21.15.39:0/1083459565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf341034c0 msgr2=0x7fdf34199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf341034c0 0x7fdf34199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 -- 172.21.15.39:0/1083459565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3419eba0 con 0x7fdf34108740 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3259c700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107df0 0x7fdf3419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3359e700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf34108740 0x7fdf3419e5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c007d70 tx=0x7fdf2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf1bfff700 1 -- 172.21.15.39:0/1083459565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c016020 con 0x7fdf34108740 2024-05-04T03:07:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3419ee90 con 0x7fdf34108740 2024-05-04T03:07:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf1bfff700 1 -- 172.21.15.39:0/1083459565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf2c00ad40 con 0x7fdf34108740 2024-05-04T03:07:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf1bfff700 1 -- 172.21.15.39:0/1083459565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0115e0 con 0x7fdf34108740 2024-05-04T03:07:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.815+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3419f380 con 0x7fdf34108740 2024-05-04T03:07:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.817+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf340512c0 con 0x7fdf34108740 2024-05-04T03:07:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.818+0000 7fdf1bfff700 1 -- 172.21.15.39:0/1083459565 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdf2c019020 con 0x7fdf34108740 2024-05-04T03:07:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.819+0000 7fdf1bfff700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf1c0b2880 0x7fdf1c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.819+0000 7fdf32d9d700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf1c0b2880 0x7fdf1c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.819+0000 7fdf1bfff700 1 -- 172.21.15.39:0/1083459565 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7fdf2c0e2500 con 0x7fdf34108740 2024-05-04T03:07:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.819+0000 7fdf32d9d700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf1c0b2880 0x7fdf1c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf24000c00 tx=0x7fdf24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:38.823+0000 7fdf1bfff700 1 -- 172.21.15.39:0/1083459565 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf2c020020 con 0x7fdf34108740 2024-05-04T03:07:39.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:39.086+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdf3419b0e0 con 0x7fdf34108740 2024-05-04T03:07:39.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:39 smithi039 ceph-mon[29715]: osdmap e77: 12 total, 12 up, 12 in 2024-05-04T03:07:39.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:39 smithi066 ceph-mon[33264]: osdmap e77: 12 total, 12 up, 12 in 2024-05-04T03:07:39.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:39 smithi167 ceph-mon[33614]: osdmap e77: 12 total, 12 up, 12 in 2024-05-04T03:07:40.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.064+0000 7fdf1bfff700 1 -- 172.21.15.39:0/1083459565 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fdf2c0aa3d0 con 0x7fdf34108740 2024-05-04T03:07:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf1c0b2880 msgr2=0x7fdf1c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf1c0b2880 0x7fdf1c0b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf24000c00 tx=0x7fdf24005fb0 comp rx=0 tx=0).stop 2024-05-04T03:07:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf34108740 msgr2=0x7fdf3419e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf34108740 0x7fdf3419e5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c007d70 tx=0x7fdf2c00a340 comp rx=0 tx=0).stop 2024-05-04T03:07:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 shutdown_connections 2024-05-04T03:07:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf1c0b2880 0x7fdf1c0b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf34108740 0x7fdf3419e5b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf341034c0 0x7fdf34199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 --2- 172.21.15.39:0/1083459565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107df0 0x7fdf3419a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 >> 172.21.15.39:0/1083459565 conn(0x7fdf340fedf0 msgr2=0x7fdf34101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 shutdown_connections 2024-05-04T03:07:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:40.067+0000 7fdf3925f700 1 -- 172.21.15.39:0/1083459565 wait complete. 2024-05-04T03:07:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2024-05-04T03:07:40.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:40 smithi066 ceph-mon[33264]: pgmap v289: 97 pgs: 7 creating+peering, 65 active+clean, 25 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:40.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:40 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1083459565' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-04T03:07:40.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:40 smithi066 ceph-mon[33264]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-04T03:07:40.407 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:40 smithi039 ceph-mon[29715]: pgmap v289: 97 pgs: 7 creating+peering, 65 active+clean, 25 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:40.407 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:40 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1083459565' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-04T03:07:40.407 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:40 smithi039 ceph-mon[29715]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-04T03:07:40.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:40 smithi167 ceph-mon[33614]: pgmap v289: 97 pgs: 7 creating+peering, 65 active+clean, 25 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:07:40.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:40 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1083459565' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-04T03:07:40.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:40 smithi167 ceph-mon[33614]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-04T03:07:40.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:07:41.079 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:41.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:41 smithi066 ceph-mon[33264]: osdmap e78: 12 total, 12 up, 12 in 2024-05-04T03:07:41.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:41 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1083459565' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-04T03:07:41.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:41 smithi066 ceph-mon[33264]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-04T03:07:41.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:41 smithi066 ceph-mon[33264]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-04T03:07:41.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:41 smithi066 ceph-mon[33264]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:07:41.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:41 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] up:active 2024-05-04T03:07:41.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:41 smithi066 ceph-mon[33264]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:41.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:41 smithi039 ceph-mon[29715]: osdmap e78: 12 total, 12 up, 12 in 2024-05-04T03:07:41.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:41 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1083459565' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-04T03:07:41.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:41 smithi039 ceph-mon[29715]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-04T03:07:41.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:41 smithi039 ceph-mon[29715]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-04T03:07:41.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:41 smithi039 ceph-mon[29715]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:07:41.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:41 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] up:active 2024-05-04T03:07:41.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:41 smithi039 ceph-mon[29715]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:41.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:41 smithi167 ceph-mon[33614]: osdmap e78: 12 total, 12 up, 12 in 2024-05-04T03:07:41.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:41 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1083459565' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-04T03:07:41.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:41 smithi167 ceph-mon[33614]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-04T03:07:41.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:41 smithi167 ceph-mon[33614]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-04T03:07:41.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:41 smithi167 ceph-mon[33614]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:07:41.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:41 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] up:active 2024-05-04T03:07:41.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:41 smithi167 ceph-mon[33614]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:41.817 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:41 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b[63850]: 2024-05-04T03:07:41.460+0000 7f8fafbe1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-04T03:07:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:42 smithi066 ceph-mon[33264]: pgmap v291: 97 pgs: 7 creating+peering, 90 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 2 op/s 2024-05-04T03:07:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:42 smithi066 ceph-mon[33264]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-04T03:07:42.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:42 smithi066 ceph-mon[33264]: Cluster is now healthy 2024-05-04T03:07:42.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:42 smithi039 ceph-mon[29715]: pgmap v291: 97 pgs: 7 creating+peering, 90 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 2 op/s 2024-05-04T03:07:42.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:42 smithi039 ceph-mon[29715]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-04T03:07:42.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:42 smithi039 ceph-mon[29715]: Cluster is now healthy 2024-05-04T03:07:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:42 smithi167 ceph-mon[33614]: pgmap v291: 97 pgs: 7 creating+peering, 90 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 2 op/s 2024-05-04T03:07:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:42 smithi167 ceph-mon[33614]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-04T03:07:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:42 smithi167 ceph-mon[33614]: Cluster is now healthy 2024-05-04T03:07:42.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.902+0000 7f026181d700 1 -- 172.21.15.39:0/118238754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 msgr2=0x7f025c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.902+0000 7f026181d700 1 --2- 172.21.15.39:0/118238754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 0x7f025c1038d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f02500099c0 tx=0x7f0250009cd0 comp rx=0 tx=0).stop 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.902+0000 7f026181d700 1 -- 172.21.15.39:0/118238754 shutdown_connections 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.902+0000 7f026181d700 1 --2- 172.21.15.39:0/118238754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 0x7f025c1038d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.902+0000 7f026181d700 1 --2- 172.21.15.39:0/118238754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025c108740 0x7f025c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.902+0000 7f026181d700 1 --2- 172.21.15.39:0/118238754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f025c107df0 0x7f025c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.902+0000 7f026181d700 1 -- 172.21.15.39:0/118238754 >> 172.21.15.39:0/118238754 conn(0x7f025c0fedc0 msgr2=0x7f025c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.903+0000 7f026181d700 1 -- 172.21.15.39:0/118238754 shutdown_connections 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.903+0000 7f026181d700 1 -- 172.21.15.39:0/118238754 wait complete. 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.903+0000 7f026181d700 1 Processor -- start 2024-05-04T03:07:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.903+0000 7f026181d700 1 -- start start 2024-05-04T03:07:42.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f026181d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 0x7f025c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:42.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f026181d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025c107df0 0x7f025c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:42.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f026181d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f025c108740 0x7f025c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:42.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f026181d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1132a0 con 0x7f025c1034c0 2024-05-04T03:07:42.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f026181d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f025c113120 con 0x7f025c107df0 2024-05-04T03:07:42.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f026181d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f025c113420 con 0x7f025c108740 2024-05-04T03:07:42.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 0x7f025c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:42.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 0x7f025c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36822/0 (socket says 172.21.15.39:36822) 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 -- 172.21.15.39:0/25042616 learned_addr learned my addr 172.21.15.39:0/25042616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025b7fe700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f025c108740 0x7f025c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025a7fc700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025c107df0 0x7f025c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 -- 172.21.15.39:0/25042616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f025c108740 msgr2=0x7f025c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f025c108740 0x7f025c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 -- 172.21.15.39:0/25042616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025c107df0 msgr2=0x7f025c19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025c107df0 0x7f025c19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 -- 172.21.15.39:0/25042616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c19fc50 con 0x7f025c1034c0 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.904+0000 7f025affd700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 0x7f025c19ad80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f02500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.905+0000 7f026081b700 1 -- 172.21.15.39:0/25042616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250004830 con 0x7f025c1034c0 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.905+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c19fee0 con 0x7f025c1034c0 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.905+0000 7f026081b700 1 -- 172.21.15.39:0/25042616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f025000f7f0 con 0x7f025c1034c0 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.905+0000 7f026081b700 1 -- 172.21.15.39:0/25042616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02500053a0 con 0x7f025c1034c0 2024-05-04T03:07:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.905+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1a03b0 con 0x7f025c1034c0 2024-05-04T03:07:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.906+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c0512c0 con 0x7f025c1034c0 2024-05-04T03:07:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.907+0000 7f026081b700 1 -- 172.21.15.39:0/25042616 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0250029060 con 0x7f025c1034c0 2024-05-04T03:07:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.908+0000 7f026081b700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f02440b2880 0x7f02440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.908+0000 7f025a7fc700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f02440b2880 0x7f02440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.908+0000 7f026081b700 1 -- 172.21.15.39:0/25042616 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f02500e2dd0 con 0x7f025c1034c0 2024-05-04T03:07:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.909+0000 7f025a7fc700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f02440b2880 0x7f02440b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f025c102080 tx=0x7f024c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:42.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:42.912+0000 7f026081b700 1 -- 172.21.15.39:0/25042616 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0250013070 con 0x7f025c1034c0 2024-05-04T03:07:43.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.133+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f025c02e5f0 con 0x7f025c1034c0 2024-05-04T03:07:43.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.135+0000 7f026081b700 1 -- 172.21.15.39:0/25042616 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22146 (secure 0 0 0) 0x7f02500aacd0 con 0x7f025c1034c0 2024-05-04T03:07:43.135 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:43.135 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":78,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:07:39.087999+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:07:43.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f02440b2880 msgr2=0x7f02440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:43.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f02440b2880 0x7f02440b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f025c102080 tx=0x7f024c00b410 comp rx=0 tx=0).stop 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 msgr2=0x7f025c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 0x7f025c19ad80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f02500047b0 comp rx=0 tx=0).stop 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 shutdown_connections 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f02440b2880 0x7f02440b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c1034c0 0x7f025c19ad80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f025c108740 0x7f025c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 --2- 172.21.15.39:0/25042616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f025c107df0 0x7f025c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 >> 172.21.15.39:0/25042616 conn(0x7f025c0fedc0 msgr2=0x7f025c1063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 shutdown_connections 2024-05-04T03:07:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:43.137+0000 7f026181d700 1 -- 172.21.15.39:0/25042616 wait complete. 2024-05-04T03:07:43.951 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:07:44.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:44 smithi039 ceph-mon[29715]: pgmap v292: 97 pgs: 97 active+clean; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-04T03:07:44.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:44 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/25042616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:07:44.205 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:44 smithi066 ceph-mon[33264]: pgmap v292: 97 pgs: 97 active+clean; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-04T03:07:44.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:44 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/25042616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:07:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:44 smithi167 ceph-mon[33614]: pgmap v292: 97 pgs: 97 active+clean; 578 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-04T03:07:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:44 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/25042616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:07:46.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 -- 172.21.15.39:0/1912321797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09601045e0 msgr2=0x7f09601049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:46.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 --2- 172.21.15.39:0/1912321797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09601045e0 0x7f09601049f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f095c0099c0 tx=0x7f095c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 -- 172.21.15.39:0/1912321797 shutdown_connections 2024-05-04T03:07:46.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 --2- 172.21.15.39:0/1912321797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09601045e0 0x7f09601049f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 --2- 172.21.15.39:0/1912321797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0960109860 0x7f09601111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 --2- 172.21.15.39:0/1912321797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0960108f10 0x7f0960109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 -- 172.21.15.39:0/1912321797 >> 172.21.15.39:0/1912321797 conn(0x7f09600ffef0 msgr2=0x7f0960102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:46.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 -- 172.21.15.39:0/1912321797 shutdown_connections 2024-05-04T03:07:46.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.021+0000 7f0967503700 1 -- 172.21.15.39:0/1912321797 wait complete. 2024-05-04T03:07:46.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.022+0000 7f0967503700 1 Processor -- start 2024-05-04T03:07:46.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.022+0000 7f0967503700 1 -- start start 2024-05-04T03:07:46.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.022+0000 7f0967503700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09601045e0 0x7f096019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:46.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.022+0000 7f0967503700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0960108f10 0x7f096019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:46.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.022+0000 7f0967503700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0960109860 0x7f096019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:46.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.022+0000 7f0967503700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0960114340 con 0x7f0960108f10 2024-05-04T03:07:46.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.022+0000 7f0967503700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09601141c0 con 0x7f0960109860 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.022+0000 7f0967503700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09601144c0 con 0x7f09601045e0 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f096529f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09601045e0 0x7f096019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0965aa0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0960109860 0x7f096019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f096529f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09601045e0 0x7f096019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:45540/0 (socket says 172.21.15.39:45540) 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f096529f700 1 -- 172.21.15.39:0/491108248 learned_addr learned my addr 172.21.15.39:0/491108248 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0964a9e700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0960108f10 0x7f096019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0964a9e700 1 -- 172.21.15.39:0/491108248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09601045e0 msgr2=0x7f096019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0964a9e700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09601045e0 0x7f096019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0964a9e700 1 -- 172.21.15.39:0/491108248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0960109860 msgr2=0x7f096019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0964a9e700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0960109860 0x7f096019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0964a9e700 1 -- 172.21.15.39:0/491108248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096019fb50 con 0x7f0960108f10 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0965aa0700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0960109860 0x7f096019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0964a9e700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0960108f10 0x7f096019b2d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f095000c940 tx=0x7f095000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f09567fc700 1 -- 172.21.15.39:0/491108248 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950014070 con 0x7f0960108f10 2024-05-04T03:07:46.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f09567fc700 1 -- 172.21.15.39:0/491108248 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0950004440 con 0x7f0960108f10 2024-05-04T03:07:46.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096019fe40 con 0x7f0960108f10 2024-05-04T03:07:46.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f09567fc700 1 -- 172.21.15.39:0/491108248 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500079e0 con 0x7f0960108f10 2024-05-04T03:07:46.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.023+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09601a0380 con 0x7f0960108f10 2024-05-04T03:07:46.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.024+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09600512c0 con 0x7f0960108f10 2024-05-04T03:07:46.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.027+0000 7f09567fc700 1 -- 172.21.15.39:0/491108248 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0950017070 con 0x7f0960108f10 2024-05-04T03:07:46.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.028+0000 7f09567fc700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f094c0b2880 0x7f094c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:46.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.028+0000 7f096529f700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f094c0b2880 0x7f094c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:46.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.028+0000 7f09567fc700 1 -- 172.21.15.39:0/491108248 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f09500e2060 con 0x7f0960108f10 2024-05-04T03:07:46.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.028+0000 7f096529f700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f094c0b2880 0x7f094c0b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f095c009f50 tx=0x7f095c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:46.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.031+0000 7f09567fc700 1 -- 172.21.15.39:0/491108248 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f095001e020 con 0x7f0960108f10 2024-05-04T03:07:46.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.251+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f096019c090 con 0x7f0960108f10 2024-05-04T03:07:46.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.252+0000 7f09567fc700 1 -- 172.21.15.39:0/491108248 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22146 (secure 0 0 0) 0x7f09500a9ed0 con 0x7f0960108f10 2024-05-04T03:07:46.253 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:07:46.253 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":78,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:07:39.087999+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:07:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f094c0b2880 msgr2=0x7f094c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f094c0b2880 0x7f094c0b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f095c009f50 tx=0x7f095c005fb0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0960108f10 msgr2=0x7f096019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0960108f10 0x7f096019b2d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f095000c940 tx=0x7f095000cc50 comp rx=0 tx=0).stop 2024-05-04T03:07:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 shutdown_connections 2024-05-04T03:07:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f094c0b2880 0x7f094c0b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0960109860 0x7f096019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09601045e0 0x7f096019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 --2- 172.21.15.39:0/491108248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0960108f10 0x7f096019b2d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:46.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.254+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 >> 172.21.15.39:0/491108248 conn(0x7f09600ffef0 msgr2=0x7f0960102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:46.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.255+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 shutdown_connections 2024-05-04T03:07:46.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:46.255+0000 7f0967503700 1 -- 172.21.15.39:0/491108248 wait complete. 2024-05-04T03:07:46.290 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:46 smithi039 ceph-mon[29715]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-04T03:07:46.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:46 smithi167 ceph-mon[33614]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-04T03:07:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:46 smithi066 ceph-mon[33264]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-04T03:07:46.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-04T03:07:47.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:47 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/491108248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:07:47.217 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:47.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:47 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/491108248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:07:47.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:47 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/491108248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:07:48.364 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:48 smithi039 ceph-mon[29715]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-04T03:07:48.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:48 smithi167 ceph-mon[33614]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-04T03:07:48.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:48 smithi066 ceph-mon[33264]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-04T03:07:48.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 -- 172.21.15.39:0/1906781915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04108740 msgr2=0x7fae041100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:48.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 --2- 172.21.15.39:0/1906781915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04108740 0x7fae041100a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00b0d0 tx=0x7fadfc00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:07:48.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 -- 172.21.15.39:0/1906781915 shutdown_connections 2024-05-04T03:07:48.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 --2- 172.21.15.39:0/1906781915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae04108740 0x7fae041100a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:48.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 --2- 172.21.15.39:0/1906781915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae041034c0 0x7fae041038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:48.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 --2- 172.21.15.39:0/1906781915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae04107df0 0x7fae04108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:48.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 -- 172.21.15.39:0/1906781915 >> 172.21.15.39:0/1906781915 conn(0x7fae040fedc0 msgr2=0x7fae041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:48.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 -- 172.21.15.39:0/1906781915 shutdown_connections 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.964+0000 7fae08b10700 1 -- 172.21.15.39:0/1906781915 wait complete. 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae08b10700 1 Processor -- start 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae08b10700 1 -- start start 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae08b10700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae041034c0 0x7fae0419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae08b10700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae04107df0 0x7fae0419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae08b10700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae04108740 0x7fae0419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae08b10700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae04113220 con 0x7fae041034c0 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae08b10700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae041130a0 con 0x7fae04107df0 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae08b10700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae041133a0 con 0x7fae04108740 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae0259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae041034c0 0x7fae0419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae0259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae041034c0 0x7fae0419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43472/0 (socket says 172.21.15.39:43472) 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.965+0000 7fae0259c700 1 -- 172.21.15.39:0/3162524036 learned_addr learned my addr 172.21.15.39:0/3162524036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae02d9d700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae04108740 0x7fae0419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:48.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae01d9b700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae04107df0 0x7fae0419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae0259c700 1 -- 172.21.15.39:0/3162524036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae04108740 msgr2=0x7fae0419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae0259c700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae04108740 0x7fae0419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae0259c700 1 -- 172.21.15.39:0/3162524036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae04107df0 msgr2=0x7fae0419c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae0259c700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae04107df0 0x7fae0419c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae0259c700 1 -- 172.21.15.39:0/3162524036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae041a6ef0 con 0x7fae041034c0 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae02d9d700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae04108740 0x7fae0419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae0259c700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae041034c0 0x7fae0419bba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fadf400d950 tx=0x7fadf400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fadf37fe700 1 -- 172.21.15.39:0/3162524036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4016070 con 0x7fae041034c0 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae041a71e0 con 0x7fae041034c0 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fadf37fe700 1 -- 172.21.15.39:0/3162524036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf4010460 con 0x7fae041034c0 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fadf37fe700 1 -- 172.21.15.39:0/3162524036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf400f620 con 0x7fae041034c0 2024-05-04T03:07:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.966+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae041a7770 con 0x7fae041034c0 2024-05-04T03:07:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.967+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae040512c0 con 0x7fae041034c0 2024-05-04T03:07:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.969+0000 7fadf37fe700 1 -- 172.21.15.39:0/3162524036 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fadf4019070 con 0x7fae041034c0 2024-05-04T03:07:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.969+0000 7fadf37fe700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fadec0b2880 0x7fadec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:48.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.970+0000 7fae01d9b700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fadec0b2880 0x7fadec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:48.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.970+0000 7fadf37fe700 1 -- 172.21.15.39:0/3162524036 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fadf40e2120 con 0x7fae041034c0 2024-05-04T03:07:48.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.970+0000 7fae01d9b700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fadec0b2880 0x7fadec0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005fd0 tx=0x7fadf8005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:48.973+0000 7fadf37fe700 1 -- 172.21.15.39:0/3162524036 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadf40a9ef0 con 0x7fae041034c0 2024-05-04T03:07:49.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:49.236+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fae0402e5f0 con 0x7fae041034c0 2024-05-04T03:07:50.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.092+0000 7fadf37fe700 1 -- 172.21.15.39:0/3162524036 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fadf40a97f0 con 0x7fae041034c0 2024-05-04T03:07:50.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fadec0b2880 msgr2=0x7fadec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fadec0b2880 0x7fadec0b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fadf8005fd0 tx=0x7fadf8005db0 comp rx=0 tx=0).stop 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae041034c0 msgr2=0x7fae0419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae041034c0 0x7fae0419bba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fadf400d950 tx=0x7fadf400dd10 comp rx=0 tx=0).stop 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 shutdown_connections 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fadec0b2880 0x7fadec0b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae041034c0 0x7fae0419bba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae04108740 0x7fae0419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 --2- 172.21.15.39:0/3162524036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae04107df0 0x7fae0419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 >> 172.21.15.39:0/3162524036 conn(0x7fae040fedc0 msgr2=0x7fae041075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 shutdown_connections 2024-05-04T03:07:50.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:50.095+0000 7fae08b10700 1 -- 172.21.15.39:0/3162524036 wait complete. 2024-05-04T03:07:50.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:50 smithi167 ceph-mon[33614]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-04T03:07:50.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:50 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3162524036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-04T03:07:50.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:50 smithi039 ceph-mon[29715]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-04T03:07:50.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:50 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3162524036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-04T03:07:50.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:50 smithi066 ceph-mon[33264]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-04T03:07:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:50 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3162524036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-04T03:07:50.877 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-04T03:07:50.877 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-04T03:07:51.133 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:51.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:51 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3162524036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-04T03:07:51.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:51 smithi167 ceph-mon[33614]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:51.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:51 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3162524036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-04T03:07:51.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:51 smithi039 ceph-mon[29715]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:51.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:51 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3162524036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-04T03:07:51.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:51 smithi066 ceph-mon[33264]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:52 smithi167 ceph-mon[33614]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-04T03:07:52.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:52 smithi039 ceph-mon[29715]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-04T03:07:52.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:52 smithi066 ceph-mon[33264]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-04T03:07:53.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 -- 172.21.15.39:0/1810274867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 msgr2=0x7f150c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 --2- 172.21.15.39:0/1810274867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 0x7f150c1038d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15000099c0 tx=0x7f1500009cd0 comp rx=0 tx=0).stop 2024-05-04T03:07:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 -- 172.21.15.39:0/1810274867 shutdown_connections 2024-05-04T03:07:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 --2- 172.21.15.39:0/1810274867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 0x7f150c1038d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 --2- 172.21.15.39:0/1810274867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c108740 0x7f150c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 --2- 172.21.15.39:0/1810274867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c107df0 0x7f150c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 -- 172.21.15.39:0/1810274867 >> 172.21.15.39:0/1810274867 conn(0x7f150c0fedf0 msgr2=0x7f150c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 -- 172.21.15.39:0/1810274867 shutdown_connections 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.041+0000 7f1512c06700 1 -- 172.21.15.39:0/1810274867 wait complete. 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f1512c06700 1 Processor -- start 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f1512c06700 1 -- start start 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f1512c06700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 0x7f150c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f1512c06700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c107df0 0x7f150c19c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f1512c06700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c108740 0x7f150c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f1512c06700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f150c113220 con 0x7f150c1034c0 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f1512c06700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1130a0 con 0x7f150c108740 2024-05-04T03:07:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f1512c06700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1133a0 con 0x7f150c107df0 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15109a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 0x7f150c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15109a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 0x7f150c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43492/0 (socket says 172.21.15.39:43492) 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15109a2700 1 -- 172.21.15.39:0/2830891846 learned_addr learned my addr 172.21.15.39:0/2830891846 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15111a3700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c108740 0x7f150c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f150bfff700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c107df0 0x7f150c19c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15109a2700 1 -- 172.21.15.39:0/2830891846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c107df0 msgr2=0x7f150c19c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15109a2700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c107df0 0x7f150c19c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15109a2700 1 -- 172.21.15.39:0/2830891846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c108740 msgr2=0x7f150c19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15109a2700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c108740 0x7f150c19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15109a2700 1 -- 172.21.15.39:0/2830891846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c1a6ef0 con 0x7f150c1034c0 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f150bfff700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c107df0 0x7f150c19c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.042+0000 7f15111a3700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c108740 0x7f150c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.043+0000 7f15109a2700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 0x7f150c19bb80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1500006010 tx=0x7f1500004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.043+0000 7f1509ffb700 1 -- 172.21.15.39:0/2830891846 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500026070 con 0x7f150c1034c0 2024-05-04T03:07:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.043+0000 7f1509ffb700 1 -- 172.21.15.39:0/2830891846 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f150000f460 con 0x7f150c1034c0 2024-05-04T03:07:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.043+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c1a7180 con 0x7f150c1034c0 2024-05-04T03:07:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.043+0000 7f1509ffb700 1 -- 172.21.15.39:0/2830891846 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500009e60 con 0x7f150c1034c0 2024-05-04T03:07:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.043+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c1a75d0 con 0x7f150c1034c0 2024-05-04T03:07:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.044+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c0512c0 con 0x7f150c1034c0 2024-05-04T03:07:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.046+0000 7f1509ffb700 1 -- 172.21.15.39:0/2830891846 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1500029070 con 0x7f150c1034c0 2024-05-04T03:07:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.046+0000 7f1509ffb700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f14fc0b2880 0x7f14fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.047+0000 7f150bfff700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f14fc0b2880 0x7f14fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.047+0000 7f1509ffb700 1 -- 172.21.15.39:0/2830891846 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f15000e2d00 con 0x7f150c1034c0 2024-05-04T03:07:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.047+0000 7f150bfff700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f14fc0b2880 0x7f14fc0b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f150c106630 tx=0x7f14f8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.050+0000 7f1509ffb700 1 -- 172.21.15.39:0/2830891846 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f150005edf0 con 0x7f150c1034c0 2024-05-04T03:07:53.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:53.311+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f150c02e5f0 con 0x7f150c1034c0 2024-05-04T03:07:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.113+0000 7f1509ffb700 1 -- 172.21.15.39:0/2830891846 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f1500013070 con 0x7f150c1034c0 2024-05-04T03:07:54.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f14fc0b2880 msgr2=0x7f14fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:54.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f14fc0b2880 0x7f14fc0b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f150c106630 tx=0x7f14f8005f50 comp rx=0 tx=0).stop 2024-05-04T03:07:54.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 msgr2=0x7f150c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:54.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 0x7f150c19bb80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1500006010 tx=0x7f1500004690 comp rx=0 tx=0).stop 2024-05-04T03:07:54.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 shutdown_connections 2024-05-04T03:07:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f14fc0b2880 0x7f14fc0b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f150c1034c0 0x7f150c19bb80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f150c108740 0x7f150c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 --2- 172.21.15.39:0/2830891846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150c107df0 0x7f150c19c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.117+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 >> 172.21.15.39:0/2830891846 conn(0x7f150c0fedf0 msgr2=0x7f150c101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.118+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 shutdown_connections 2024-05-04T03:07:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:54.118+0000 7f1512c06700 1 -- 172.21.15.39:0/2830891846 wait complete. 2024-05-04T03:07:54.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:54 smithi167 ceph-mon[33614]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-05-04T03:07:54.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:54 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2830891846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-04T03:07:54.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:54 smithi039 ceph-mon[29715]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-05-04T03:07:54.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:54 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2830891846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-04T03:07:54.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:54 smithi066 ceph-mon[33264]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-05-04T03:07:54.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:54 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2830891846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-04T03:07:54.955 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-04T03:07:55.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:55 smithi039 ceph-mon[29715]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-04T03:07:55.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:55 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2830891846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-04T03:07:55.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:55 smithi039 ceph-mon[29715]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-04T03:07:55.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:55 smithi039 ceph-mon[29715]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:55 smithi039 ceph-mon[29715]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-04T03:07:55.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:55 smithi039 ceph-mon[29715]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-04T03:07:55.222 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:55.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:55 smithi167 ceph-mon[33614]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-04T03:07:55.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:55 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2830891846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-04T03:07:55.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:55 smithi167 ceph-mon[33614]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-04T03:07:55.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:55 smithi167 ceph-mon[33614]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:55.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:55 smithi167 ceph-mon[33614]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-04T03:07:55.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:55 smithi167 ceph-mon[33614]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-04T03:07:55.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:55 smithi066 ceph-mon[33264]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-04T03:07:55.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:55 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2830891846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-04T03:07:55.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:55 smithi066 ceph-mon[33264]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-04T03:07:55.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:55 smithi066 ceph-mon[33264]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:07:55.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:55 smithi066 ceph-mon[33264]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-04T03:07:55.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:55 smithi066 ceph-mon[33264]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-04T03:07:56.355 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:56 smithi039 ceph-mon[29715]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-04T03:07:56.356 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:56 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] up:active 2024-05-04T03:07:56.356 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:56 smithi039 ceph-mon[29715]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-04T03:07:56.356 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:56 smithi039 ceph-mon[29715]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-04T03:07:56.356 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:56 smithi039 ceph-mon[29715]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-04T03:07:56.356 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:56 smithi039 ceph-mon[29715]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-04T03:07:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:56 smithi167 ceph-mon[33614]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-04T03:07:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:56 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] up:active 2024-05-04T03:07:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:56 smithi167 ceph-mon[33614]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-04T03:07:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:56 smithi167 ceph-mon[33614]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-04T03:07:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:56 smithi167 ceph-mon[33614]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-04T03:07:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:56 smithi167 ceph-mon[33614]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-04T03:07:56.461 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:56 smithi066 ceph-mon[33264]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-04T03:07:56.461 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:56 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] up:active 2024-05-04T03:07:56.461 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:56 smithi066 ceph-mon[33264]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-04T03:07:56.461 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:56 smithi066 ceph-mon[33264]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-04T03:07:56.461 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:56 smithi066 ceph-mon[33264]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-04T03:07:56.462 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:56 smithi066 ceph-mon[33264]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-04T03:07:56.817 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:07:56 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b[63850]: 2024-05-04T03:07:56.460+0000 7f8fafbe1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-04T03:07:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 -- 172.21.15.39:0/604333353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac104610 msgr2=0x7f0eac104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/604333353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac104610 0x7f0eac104a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0099c0 tx=0x7f0e9c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 -- 172.21.15.39:0/604333353 shutdown_connections 2024-05-04T03:07:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/604333353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac104610 0x7f0eac104a20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/604333353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eac109890 0x7f0eac1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/604333353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eac108f40 0x7f0eac109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 -- 172.21.15.39:0/604333353 >> 172.21.15.39:0/604333353 conn(0x7f0eac07af70 msgr2=0x7f0eac07b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 -- 172.21.15.39:0/604333353 shutdown_connections 2024-05-04T03:07:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.097+0000 7f0eb28b2700 1 -- 172.21.15.39:0/604333353 wait complete. 2024-05-04T03:07:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb28b2700 1 Processor -- start 2024-05-04T03:07:57.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb28b2700 1 -- start start 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb28b2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eac104610 0x7f0eac19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb28b2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eac108f40 0x7f0eac19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb28b2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac109890 0x7f0eac19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb28b2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac114370 con 0x7f0eac109890 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb28b2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1141f0 con 0x7f0eac108f40 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb28b2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1144f0 con 0x7f0eac104610 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb0e4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac109890 0x7f0eac19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb0e4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac109890 0x7f0eac19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33598/0 (socket says 172.21.15.39:33598) 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eb0e4f700 1 -- 172.21.15.39:0/753158011 learned_addr learned my addr 172.21.15.39:0/753158011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.098+0000 7f0eab7fe700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eac108f40 0x7f0eac19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:57.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eabfff700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eac104610 0x7f0eac19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eb0e4f700 1 -- 172.21.15.39:0/753158011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eac104610 msgr2=0x7f0eac19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eb0e4f700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eac104610 0x7f0eac19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eb0e4f700 1 -- 172.21.15.39:0/753158011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eac108f40 msgr2=0x7f0eac19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eb0e4f700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eac108f40 0x7f0eac19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eb0e4f700 1 -- 172.21.15.39:0/753158011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac19fcd0 con 0x7f0eac109890 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eab7fe700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eac108f40 0x7f0eac19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eb0e4f700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac109890 0x7f0eac19f630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4007ee0 tx=0x7f0ea400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0ea97fa700 1 -- 172.21.15.39:0/753158011 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4004bb0 con 0x7f0eac109890 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0ea97fa700 1 -- 172.21.15.39:0/753158011 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ea4004d50 con 0x7f0eac109890 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac19ffc0 con 0x7f0eac109890 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0ea97fa700 1 -- 172.21.15.39:0/753158011 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40058b0 con 0x7f0eac109890 2024-05-04T03:07:57.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.099+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1a78e0 con 0x7f0eac109890 2024-05-04T03:07:57.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.100+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac0512c0 con 0x7f0eac109890 2024-05-04T03:07:57.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.102+0000 7f0ea97fa700 1 -- 172.21.15.39:0/753158011 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0ea401a020 con 0x7f0eac109890 2024-05-04T03:07:57.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.102+0000 7f0ea97fa700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e980b2880 0x7f0e980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:57.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.103+0000 7f0eabfff700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e980b2880 0x7f0e980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:57.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.103+0000 7f0ea97fa700 1 -- 172.21.15.39:0/753158011 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f0ea40e1db0 con 0x7f0eac109890 2024-05-04T03:07:57.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.103+0000 7f0eabfff700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e980b2880 0x7f0e980b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c000c00 tx=0x7f0e9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:57.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.106+0000 7f0ea97fa700 1 -- 172.21.15.39:0/753158011 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ea4010070 con 0x7f0eac109890 2024-05-04T03:07:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:57.371+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0eac02e5f0 con 0x7f0eac109890 2024-05-04T03:07:57.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:57 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] up:active 2024-05-04T03:07:57.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:57 smithi039 ceph-mon[29715]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-04T03:07:57.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:57 smithi039 ceph-mon[29715]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-04T03:07:57.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:57 smithi039 ceph-mon[29715]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-04T03:07:57.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:57 smithi039 ceph-mon[29715]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-04T03:07:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:57 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] up:active 2024-05-04T03:07:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:57 smithi167 ceph-mon[33614]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-04T03:07:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:57 smithi167 ceph-mon[33614]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-04T03:07:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:57 smithi167 ceph-mon[33614]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-04T03:07:57.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:57 smithi167 ceph-mon[33614]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-04T03:07:57.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:57 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] up:active 2024-05-04T03:07:57.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:57 smithi066 ceph-mon[33264]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-04T03:07:57.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:57 smithi066 ceph-mon[33264]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-04T03:07:57.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:57 smithi066 ceph-mon[33264]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-04T03:07:57.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:57 smithi066 ceph-mon[33264]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-04T03:07:58.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.167+0000 7f0ea97fa700 1 -- 172.21.15.39:0/753158011 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f0ea40a9c20 con 0x7f0eac109890 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e980b2880 msgr2=0x7f0e980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e980b2880 0x7f0e980b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c000c00 tx=0x7f0e9c01a040 comp rx=0 tx=0).stop 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac109890 msgr2=0x7f0eac19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac109890 0x7f0eac19f630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4007ee0 tx=0x7f0ea400d3b0 comp rx=0 tx=0).stop 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 shutdown_connections 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e980b2880 0x7f0e980b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eac109890 0x7f0eac19f630 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0eac104610 0x7f0eac19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 --2- 172.21.15.39:0/753158011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0eac108f40 0x7f0eac19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:58.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.170+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 >> 172.21.15.39:0/753158011 conn(0x7f0eac07af70 msgr2=0x7f0eac101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:58.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.171+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 shutdown_connections 2024-05-04T03:07:58.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:07:58.171+0000 7f0eb28b2700 1 -- 172.21.15.39:0/753158011 wait complete. 2024-05-04T03:07:58.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: pgmap v299: 97 pgs: 97 active+clean; 580 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-04T03:07:58.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] up:active 2024-05-04T03:07:58.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-04T03:07:58.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-04T03:07:58.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: Cluster is now healthy 2024-05-04T03:07:58.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-04T03:07:58.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-04T03:07:58.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-04T03:07:58.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:58 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/753158011' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:07:58.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: pgmap v299: 97 pgs: 97 active+clean; 580 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-04T03:07:58.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] up:active 2024-05-04T03:07:58.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-04T03:07:58.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-04T03:07:58.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: Cluster is now healthy 2024-05-04T03:07:58.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-04T03:07:58.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-04T03:07:58.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-04T03:07:58.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:58 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/753158011' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: pgmap v299: 97 pgs: 97 active+clean; 580 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 1 op/s 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] up:active 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: Cluster is now healthy 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-04T03:07:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:58 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/753158011' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:07:58.975 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-04T03:07:59.235 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:07:59.358 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:59 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/753158011' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-04T03:07:59.358 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:59 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] up:active 2024-05-04T03:07:59.358 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:59 smithi039 ceph-mon[29715]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-04T03:07:59.358 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:07:59 smithi039 ceph-mon[29715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:07:59.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:59 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/753158011' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-04T03:07:59.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:59 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] up:active 2024-05-04T03:07:59.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:59 smithi167 ceph-mon[33614]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-04T03:07:59.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:07:59 smithi167 ceph-mon[33614]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:07:59.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:59 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/753158011' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-04T03:07:59.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:59 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] up:active 2024-05-04T03:07:59.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:59 smithi066 ceph-mon[33264]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-04T03:07:59.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:07:59 smithi066 ceph-mon[33264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:08:00.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:00 smithi039 ceph-mon[29715]: pgmap v300: 97 pgs: 97 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-04T03:08:00.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:00 smithi066 ceph-mon[33264]: pgmap v300: 97 pgs: 97 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-04T03:08:00.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:00 smithi167 ceph-mon[33614]: pgmap v300: 97 pgs: 97 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-04T03:08:01.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.067+0000 7f3437d9f700 1 -- 172.21.15.39:0/1882294468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3430108740 msgr2=0x7f34301100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:01.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.067+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1882294468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3430108740 0x7f34301100a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f342800b0d0 tx=0x7f342800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:08:01.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.067+0000 7f3437d9f700 1 -- 172.21.15.39:0/1882294468 shutdown_connections 2024-05-04T03:08:01.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.067+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1882294468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3430108740 0x7f34301100a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:01.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.067+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1882294468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34301034c0 0x7f34301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:01.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.067+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1882294468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3430107df0 0x7f3430108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:01.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.067+0000 7f3437d9f700 1 -- 172.21.15.39:0/1882294468 >> 172.21.15.39:0/1882294468 conn(0x7f34300fedc0 msgr2=0x7f34301011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:01.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.067+0000 7f3437d9f700 1 -- 172.21.15.39:0/1882294468 shutdown_connections 2024-05-04T03:08:01.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.068+0000 7f3437d9f700 1 -- 172.21.15.39:0/1882294468 wait complete. 2024-05-04T03:08:01.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.068+0000 7f3437d9f700 1 Processor -- start 2024-05-04T03:08:01.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.068+0000 7f3437d9f700 1 -- start start 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3437d9f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34301034c0 0x7f343019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3437d9f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3430107df0 0x7f343019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3437d9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3430108740 0x7f34301a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3437d9f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3430113220 con 0x7f34301034c0 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3437d9f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34301130a0 con 0x7f3430108740 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3437d9f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f34301133a0 con 0x7f3430107df0 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3435b3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34301034c0 0x7f343019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3435b3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34301034c0 0x7f343019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33610/0 (socket says 172.21.15.39:33610) 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3435b3b700 1 -- 172.21.15.39:0/1438452000 learned_addr learned my addr 172.21.15.39:0/1438452000 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f343633c700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3430108740 0x7f34301a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f343533a700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3430107df0 0x7f343019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3435b3b700 1 -- 172.21.15.39:0/1438452000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3430107df0 msgr2=0x7f343019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3435b3b700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3430107df0 0x7f343019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:01.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3435b3b700 1 -- 172.21.15.39:0/1438452000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3430108740 msgr2=0x7f34301a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:01.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3435b3b700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3430108740 0x7f34301a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:01.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.069+0000 7f3435b3b700 1 -- 172.21.15.39:0/1438452000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34301a1d60 con 0x7f34301034c0 2024-05-04T03:08:01.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.070+0000 7f3435b3b700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34301034c0 0x7f343019ce90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f342c00e9a0 tx=0x7f342c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:01.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.070+0000 7f3426ffd700 1 -- 172.21.15.39:0/1438452000 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c00ca10 con 0x7f34301034c0 2024-05-04T03:08:01.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.070+0000 7f3426ffd700 1 -- 172.21.15.39:0/1438452000 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f342c00cbb0 con 0x7f34301034c0 2024-05-04T03:08:01.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.070+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34301a2050 con 0x7f34301034c0 2024-05-04T03:08:01.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.070+0000 7f3426ffd700 1 -- 172.21.15.39:0/1438452000 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c018680 con 0x7f34301034c0 2024-05-04T03:08:01.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.070+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34301a9970 con 0x7f34301034c0 2024-05-04T03:08:01.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.071+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34300512c0 con 0x7f34301034c0 2024-05-04T03:08:01.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.073+0000 7f3426ffd700 1 -- 172.21.15.39:0/1438452000 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f342c010430 con 0x7f34301034c0 2024-05-04T03:08:01.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.073+0000 7f3426ffd700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f341c0b2880 0x7f341c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:01.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.073+0000 7f343533a700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f341c0b2880 0x7f341c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:01.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.073+0000 7f3426ffd700 1 -- 172.21.15.39:0/1438452000 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f342c0e2610 con 0x7f34301034c0 2024-05-04T03:08:01.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.074+0000 7f343533a700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f341c0b2880 0x7f341c0b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3430109910 tx=0x7f342000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:01.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.077+0000 7f3426ffd700 1 -- 172.21.15.39:0/1438452000 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f342c0aa480 con 0x7f34301034c0 2024-05-04T03:08:01.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:01.343+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f34301a9ba0 con 0x7f34301034c0 2024-05-04T03:08:02.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.187+0000 7f3426ffd700 1 -- 172.21.15.39:0/1438452000 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7f342c0a9d80 con 0x7f34301034c0 2024-05-04T03:08:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f341c0b2880 msgr2=0x7f341c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f341c0b2880 0x7f341c0b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3430109910 tx=0x7f342000b410 comp rx=0 tx=0).stop 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34301034c0 msgr2=0x7f343019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34301034c0 0x7f343019ce90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f342c00e9a0 tx=0x7f342c00ecb0 comp rx=0 tx=0).stop 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 shutdown_connections 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f341c0b2880 0x7f341c0b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34301034c0 0x7f343019ce90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3430108740 0x7f34301a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 --2- 172.21.15.39:0/1438452000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3430107df0 0x7f343019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 >> 172.21.15.39:0/1438452000 conn(0x7f34300fedc0 msgr2=0x7f343006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 shutdown_connections 2024-05-04T03:08:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:02.191+0000 7f3437d9f700 1 -- 172.21.15.39:0/1438452000 wait complete. 2024-05-04T03:08:02.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:02 smithi039 ceph-mon[29715]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-04T03:08:02.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:02 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1438452000' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-04T03:08:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:02 smithi066 ceph-mon[33264]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-04T03:08:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:02 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1438452000' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-04T03:08:02.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:02 smithi167 ceph-mon[33614]: pgmap v301: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-05-04T03:08:02.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:02 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1438452000' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-04T03:08:02.919 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-04T03:08:02.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-04T03:08:03.177 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:03.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:03 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1438452000' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-04T03:08:03.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:03 smithi039 ceph-mon[29715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:08:03.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:03 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1438452000' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-04T03:08:03.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:03 smithi066 ceph-mon[33264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:08:03.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:03 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1438452000' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-04T03:08:03.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:03 smithi167 ceph-mon[33614]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:08:04.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:04 smithi039 ceph-mon[29715]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-04T03:08:04.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:04 smithi066 ceph-mon[33264]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-04T03:08:04.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:04 smithi167 ceph-mon[33614]: pgmap v302: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-04T03:08:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 -- 172.21.15.39:0/2848252028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94107e10 msgr2=0x7efc94108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/2848252028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94107e10 0x7efc94108220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efc840099c0 tx=0x7efc84009cd0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 -- 172.21.15.39:0/2848252028 shutdown_connections 2024-05-04T03:08:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/2848252028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc94108760 0x7efc941100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/2848252028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc941034e0 0x7efc941038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/2848252028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94107e10 0x7efc94108220 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 -- 172.21.15.39:0/2848252028 >> 172.21.15.39:0/2848252028 conn(0x7efc940fef30 msgr2=0x7efc94101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 -- 172.21.15.39:0/2848252028 shutdown_connections 2024-05-04T03:08:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 -- 172.21.15.39:0/2848252028 wait complete. 2024-05-04T03:08:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 Processor -- start 2024-05-04T03:08:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.008+0000 7efc9ab4a700 1 -- start start 2024-05-04T03:08:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc9ab4a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc941034e0 0x7efc940ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc9ab4a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc94107e10 0x7efc940ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc9ab4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94108760 0x7efc940ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc9ab4a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc941132b0 con 0x7efc94108760 2024-05-04T03:08:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc9ab4a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc94113130 con 0x7efc94107e10 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc9ab4a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc94113430 con 0x7efc941034e0 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94108760 0x7efc940ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94108760 0x7efc940ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33630/0 (socket says 172.21.15.39:33630) 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 -- 172.21.15.39:0/91871147 learned_addr learned my addr 172.21.15.39:0/91871147 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc988e6700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc941034e0 0x7efc940ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc93fff700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc94107e10 0x7efc940ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 -- 172.21.15.39:0/91871147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc941034e0 msgr2=0x7efc940ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc941034e0 0x7efc940ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 -- 172.21.15.39:0/91871147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc94107e10 msgr2=0x7efc940ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc94107e10 0x7efc940ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 -- 172.21.15.39:0/91871147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc941a9150 con 0x7efc94108760 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc93fff700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc94107e10 0x7efc940ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.009+0000 7efc990e7700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94108760 0x7efc940ffe90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efc8c007ee0 tx=0x7efc8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.010+0000 7efc91ffb700 1 -- 172.21.15.39:0/91871147 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00dbd0 con 0x7efc94108760 2024-05-04T03:08:05.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.010+0000 7efc91ffb700 1 -- 172.21.15.39:0/91871147 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc8c00dd70 con 0x7efc94108760 2024-05-04T03:08:05.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.010+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc941a9440 con 0x7efc94108760 2024-05-04T03:08:05.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.010+0000 7efc91ffb700 1 -- 172.21.15.39:0/91871147 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0058d0 con 0x7efc94108760 2024-05-04T03:08:05.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.010+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc941a9980 con 0x7efc94108760 2024-05-04T03:08:05.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.011+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc940512c0 con 0x7efc94108760 2024-05-04T03:08:05.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.013+0000 7efc91ffb700 1 -- 172.21.15.39:0/91871147 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efc8c01a020 con 0x7efc94108760 2024-05-04T03:08:05.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.014+0000 7efc91ffb700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc800b2880 0x7efc800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:05.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.014+0000 7efc988e6700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc800b2880 0x7efc800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:05.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.014+0000 7efc91ffb700 1 -- 172.21.15.39:0/91871147 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7efc8c0e1f50 con 0x7efc94108760 2024-05-04T03:08:05.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.014+0000 7efc988e6700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc800b2880 0x7efc800b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc7c006fd0 tx=0x7efc7c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:05.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.019+0000 7efc91ffb700 1 -- 172.21.15.39:0/91871147 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc8c010070 con 0x7efc94108760 2024-05-04T03:08:05.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.267+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efc94063c00 con 0x7efc800b2880 2024-05-04T03:08:05.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.298+0000 7efc91ffb700 1 -- 172.21.15.39:0/91871147 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc94063c00 con 0x7efc800b2880 2024-05-04T03:08:05.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc800b2880 msgr2=0x7efc800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:05.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc800b2880 0x7efc800b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc7c006fd0 tx=0x7efc7c008040 comp rx=0 tx=0).stop 2024-05-04T03:08:05.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94108760 msgr2=0x7efc940ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:05.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94108760 0x7efc940ffe90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efc8c007ee0 tx=0x7efc8c00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 shutdown_connections 2024-05-04T03:08:05.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efc800b2880 0x7efc800b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94108760 0x7efc940ffe90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc941034e0 0x7efc940ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 --2- 172.21.15.39:0/91871147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc94107e10 0x7efc940ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:05.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 >> 172.21.15.39:0/91871147 conn(0x7efc940fef30 msgr2=0x7efc9406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:05.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 shutdown_connections 2024-05-04T03:08:05.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:05.300+0000 7efc9ab4a700 1 -- 172.21.15.39:0/91871147 wait complete. 2024-05-04T03:08:05.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:08:05 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:08:05.272+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:08:05.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:08:05 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:08:05.272+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:08:05.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:08:05 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:08:05.272+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:08:05.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:08:05 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:08:05.272+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:08:05.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:08:05 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:08:05.272+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:08:06.036 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-04T03:08:06.291 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:06.415 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:06 smithi039 ceph-mon[29715]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-04T03:08:06.415 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:06 smithi039 ceph-mon[29715]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:06.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:06 smithi167 ceph-mon[33614]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-04T03:08:06.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:06 smithi167 ceph-mon[33614]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:06.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:06 smithi066 ceph-mon[33264]: pgmap v303: 97 pgs: 97 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-04T03:08:06.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:06 smithi066 ceph-mon[33264]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.142+0000 7ff957813700 1 -- 172.21.15.39:0/219329480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950109860 msgr2=0x7ff9501111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.142+0000 7ff957813700 1 --2- 172.21.15.39:0/219329480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950109860 0x7ff9501111c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b0d0 tx=0x7ff94800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.142+0000 7ff957813700 1 -- 172.21.15.39:0/219329480 shutdown_connections 2024-05-04T03:08:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.142+0000 7ff957813700 1 --2- 172.21.15.39:0/219329480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950109860 0x7ff9501111c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.142+0000 7ff957813700 1 --2- 172.21.15.39:0/219329480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9501045e0 0x7ff9501049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.142+0000 7ff957813700 1 --2- 172.21.15.39:0/219329480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff950108f10 0x7ff950109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.142+0000 7ff957813700 1 -- 172.21.15.39:0/219329480 >> 172.21.15.39:0/219329480 conn(0x7ff9500ffef0 msgr2=0x7ff950102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.142+0000 7ff957813700 1 -- 172.21.15.39:0/219329480 shutdown_connections 2024-05-04T03:08:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.143+0000 7ff957813700 1 -- 172.21.15.39:0/219329480 wait complete. 2024-05-04T03:08:08.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.143+0000 7ff957813700 1 Processor -- start 2024-05-04T03:08:08.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.143+0000 7ff957813700 1 -- start start 2024-05-04T03:08:08.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff957813700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9501045e0 0x7ff9501a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:08.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff957813700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108f10 0x7ff9501a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff957813700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff950109860 0x7ff95019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff957813700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff950114340 con 0x7ff950108f10 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff957813700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501141c0 con 0x7ff9501045e0 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff957813700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501144c0 con 0x7ff950109860 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108f10 0x7ff9501a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108f10 0x7ff9501a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52270/0 (socket says 172.21.15.39:52270) 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 -- 172.21.15.39:0/1750146020 learned_addr learned my addr 172.21.15.39:0/1750146020 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff9555af700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9501045e0 0x7ff9501a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff955db0700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff950109860 0x7ff95019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 -- 172.21.15.39:0/1750146020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff950109860 msgr2=0x7ff95019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff950109860 0x7ff95019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 -- 172.21.15.39:0/1750146020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9501045e0 msgr2=0x7ff9501a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9501045e0 0x7ff9501a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 -- 172.21.15.39:0/1750146020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95019b1e0 con 0x7ff950108f10 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff955db0700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff950109860 0x7ff95019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff9555af700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9501045e0 0x7ff9501a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.144+0000 7ff954dae700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108f10 0x7ff9501a0e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff94000c8f0 tx=0x7ff94000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.145+0000 7ff9467fc700 1 -- 172.21.15.39:0/1750146020 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940007b00 con 0x7ff950108f10 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.145+0000 7ff9467fc700 1 -- 172.21.15.39:0/1750146020 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff940007ca0 con 0x7ff950108f10 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.145+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95019b4d0 con 0x7ff950108f10 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.145+0000 7ff9467fc700 1 -- 172.21.15.39:0/1750146020 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940018660 con 0x7ff950108f10 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.145+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501a7770 con 0x7ff950108f10 2024-05-04T03:08:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.146+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9500512c0 con 0x7ff950108f10 2024-05-04T03:08:08.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.148+0000 7ff9467fc700 1 -- 172.21.15.39:0/1750146020 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff94000f450 con 0x7ff950108f10 2024-05-04T03:08:08.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.149+0000 7ff9467fc700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff93c0b2910 0x7ff93c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:08.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.149+0000 7ff9555af700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff93c0b2910 0x7ff93c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:08.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.149+0000 7ff9467fc700 1 -- 172.21.15.39:0/1750146020 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7ff9400e2230 con 0x7ff950108f10 2024-05-04T03:08:08.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.150+0000 7ff9555af700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff93c0b2910 0x7ff93c0b4e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff94c005fd0 tx=0x7ff94c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.153+0000 7ff9467fc700 1 -- 172.21.15.39:0/1750146020 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff940014030 con 0x7ff950108f10 2024-05-04T03:08:08.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.398+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff950063c00 con 0x7ff93c0b2910 2024-05-04T03:08:08.399 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:08 smithi039 ceph-mon[29715]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.7 KiB/s wr, 21 op/s 2024-05-04T03:08:08.399 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:08 smithi039 ceph-mon[29715]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-04T03:08:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.427+0000 7ff9467fc700 1 -- 172.21.15.39:0/1750146020 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff950063c00 con 0x7ff93c0b2910 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff93c0b2910 msgr2=0x7ff93c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff93c0b2910 0x7ff93c0b4e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff94c005fd0 tx=0x7ff94c005f60 comp rx=0 tx=0).stop 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108f10 msgr2=0x7ff9501a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108f10 0x7ff9501a0e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff94000c8f0 tx=0x7ff94000ccb0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 shutdown_connections 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff93c0b2910 0x7ff93c0b4e20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff950109860 0x7ff95019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9501045e0 0x7ff9501a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 --2- 172.21.15.39:0/1750146020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108f10 0x7ff9501a0e20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 >> 172.21.15.39:0/1750146020 conn(0x7ff9500ffef0 msgr2=0x7ff9501086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:08.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 shutdown_connections 2024-05-04T03:08:08.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:08.429+0000 7ff957813700 1 -- 172.21.15.39:0/1750146020 wait complete. 2024-05-04T03:08:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:08 smithi066 ceph-mon[33264]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.7 KiB/s wr, 21 op/s 2024-05-04T03:08:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:08 smithi066 ceph-mon[33264]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-04T03:08:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:08 smithi167 ceph-mon[33614]: pgmap v304: 97 pgs: 97 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.7 KiB/s wr, 21 op/s 2024-05-04T03:08:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:08 smithi167 ceph-mon[33614]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-04T03:08:09.125 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-04T03:08:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:09.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:09 smithi039 ceph-mon[29715]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:09.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:09 smithi066 ceph-mon[33264]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:09.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:09 smithi167 ceph-mon[33614]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:10.536 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:10 smithi039 ceph-mon[29715]: pgmap v305: 97 pgs: 97 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 22 op/s 2024-05-04T03:08:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:10 smithi066 ceph-mon[33264]: pgmap v305: 97 pgs: 97 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 22 op/s 2024-05-04T03:08:10.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:10 smithi167 ceph-mon[33614]: pgmap v305: 97 pgs: 97 active+clean; 591 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 22 op/s 2024-05-04T03:08:11.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.118+0000 7f08a3095700 1 -- 172.21.15.39:0/2419323027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105e70 msgr2=0x7f089c10e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:11.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.118+0000 7f08a3095700 1 --2- 172.21.15.39:0/2419323027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105e70 0x7f089c10e3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0888009990 tx=0x7f0888009ca0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.119+0000 7f08a3095700 1 -- 172.21.15.39:0/2419323027 shutdown_connections 2024-05-04T03:08:11.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.119+0000 7f08a3095700 1 --2- 172.21.15.39:0/2419323027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f089c10e900 0x7f089c110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.119+0000 7f08a3095700 1 --2- 172.21.15.39:0/2419323027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089c105520 0x7f089c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.119+0000 7f08a3095700 1 --2- 172.21.15.39:0/2419323027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105e70 0x7f089c10e3c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.119+0000 7f08a3095700 1 -- 172.21.15.39:0/2419323027 >> 172.21.15.39:0/2419323027 conn(0x7f089c0fedc0 msgr2=0x7f089c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:11.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.119+0000 7f08a3095700 1 -- 172.21.15.39:0/2419323027 shutdown_connections 2024-05-04T03:08:11.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.119+0000 7f08a3095700 1 -- 172.21.15.39:0/2419323027 wait complete. 2024-05-04T03:08:11.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.120+0000 7f08a3095700 1 Processor -- start 2024-05-04T03:08:11.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.120+0000 7f08a3095700 1 -- start start 2024-05-04T03:08:11.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.120+0000 7f08a3095700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105520 0x7f089c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:11.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.120+0000 7f08a3095700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f089c105e70 0x7f089c1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:11.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.120+0000 7f08a3095700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089c10e900 0x7f089c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:11.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.120+0000 7f08a3095700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1142f0 con 0x7f089c105520 2024-05-04T03:08:11.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a3095700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f089c114170 con 0x7f089c105e70 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a3095700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f089c114470 con 0x7f089c10e900 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105520 0x7f089c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105520 0x7f089c1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52282/0 (socket says 172.21.15.39:52282) 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 -- 172.21.15.39:0/2192496671 learned_addr learned my addr 172.21.15.39:0/2192496671 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a1632700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089c10e900 0x7f089c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f089bfff700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f089c105e70 0x7f089c1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 -- 172.21.15.39:0/2192496671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089c10e900 msgr2=0x7f089c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089c10e900 0x7f089c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 -- 172.21.15.39:0/2192496671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f089c105e70 msgr2=0x7f089c1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f089c105e70 0x7f089c1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 -- 172.21.15.39:0/2192496671 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c19b1d0 con 0x7f089c105520 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f089bfff700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f089c105e70 0x7f089c1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f08a0e31700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105520 0x7f089c1a09a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f089000d950 tx=0x7f089000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.121+0000 7f0899ffb700 1 -- 172.21.15.39:0/2192496671 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890016070 con 0x7f089c105520 2024-05-04T03:08:11.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.122+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c19b4c0 con 0x7f089c105520 2024-05-04T03:08:11.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.122+0000 7f0899ffb700 1 -- 172.21.15.39:0/2192496671 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0890009dd0 con 0x7f089c105520 2024-05-04T03:08:11.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.122+0000 7f0899ffb700 1 -- 172.21.15.39:0/2192496671 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089000f3c0 con 0x7f089c105520 2024-05-04T03:08:11.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.122+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1a7750 con 0x7f089c105520 2024-05-04T03:08:11.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.123+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089c06be30 con 0x7f089c105520 2024-05-04T03:08:11.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.124+0000 7f0899ffb700 1 -- 172.21.15.39:0/2192496671 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0890019070 con 0x7f089c105520 2024-05-04T03:08:11.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.125+0000 7f0899ffb700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f088c0b2990 0x7f088c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:11.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.125+0000 7f089bfff700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f088c0b2990 0x7f088c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:11.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.125+0000 7f0899ffb700 1 -- 172.21.15.39:0/2192496671 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f08900e2160 con 0x7f089c105520 2024-05-04T03:08:11.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.125+0000 7f089bfff700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f088c0b2990 0x7f088c0b4ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08880059d0 tx=0x7f0888005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:11.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.129+0000 7f0899ffb700 1 -- 172.21.15.39:0/2192496671 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08900a9ed0 con 0x7f089c105520 2024-05-04T03:08:11.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.379+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f089c19bfd0 con 0x7f088c0b2990 2024-05-04T03:08:11.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.386+0000 7f0899ffb700 1 -- 172.21.15.39:0/2192496671 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f089c19bfd0 con 0x7f088c0b2990 2024-05-04T03:08:11.387 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/qa/sv_0/e7a5b914-81ab-44ab-942c-66f330eaa990 2024-05-04T03:08:11.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f088c0b2990 msgr2=0x7f088c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:11.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f088c0b2990 0x7f088c0b4ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08880059d0 tx=0x7f0888005940 comp rx=0 tx=0).stop 2024-05-04T03:08:11.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105520 msgr2=0x7f089c1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:11.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105520 0x7f089c1a09a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f089000d950 tx=0x7f089000dc60 comp rx=0 tx=0).stop 2024-05-04T03:08:11.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 shutdown_connections 2024-05-04T03:08:11.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f088c0b2990 0x7f088c0b4ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f089c105520 0x7f089c1a09a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089c10e900 0x7f089c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 --2- 172.21.15.39:0/2192496671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f089c105e70 0x7f089c1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 >> 172.21.15.39:0/2192496671 conn(0x7f089c0fedc0 msgr2=0x7f089c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 shutdown_connections 2024-05-04T03:08:11.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:11.389+0000 7f08a3095700 1 -- 172.21.15.39:0/2192496671 wait complete. 2024-05-04T03:08:12.063 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-04T03:08:12.320 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:12.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:12 smithi039 ceph-mon[29715]: pgmap v306: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 16 op/s 2024-05-04T03:08:12.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:12 smithi039 ceph-mon[29715]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:12 smithi066 ceph-mon[33264]: pgmap v306: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 16 op/s 2024-05-04T03:08:12.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:12 smithi066 ceph-mon[33264]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:12.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:12 smithi167 ceph-mon[33614]: pgmap v306: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 16 op/s 2024-05-04T03:08:12.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:12 smithi167 ceph-mon[33614]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 -- 172.21.15.39:0/2289026351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 msgr2=0x7f2a941038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 --2- 172.21.15.39:0/2289026351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 0x7f2a941038d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400b0d0 tx=0x7f2a8400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 -- 172.21.15.39:0/2289026351 shutdown_connections 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 --2- 172.21.15.39:0/2289026351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 0x7f2a941038d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 --2- 172.21.15.39:0/2289026351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a94108740 0x7f2a941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 --2- 172.21.15.39:0/2289026351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a94107df0 0x7f2a94108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 -- 172.21.15.39:0/2289026351 >> 172.21.15.39:0/2289026351 conn(0x7f2a940fedf0 msgr2=0x7f2a94101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 -- 172.21.15.39:0/2289026351 shutdown_connections 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.190+0000 7f2a99757700 1 -- 172.21.15.39:0/2289026351 wait complete. 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.191+0000 7f2a99757700 1 Processor -- start 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.191+0000 7f2a99757700 1 -- start start 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.191+0000 7f2a99757700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 0x7f2a9419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.191+0000 7f2a99757700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a94107df0 0x7f2a9419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.191+0000 7f2a99757700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a94108740 0x7f2a9419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.191+0000 7f2a99757700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a94113220 con 0x7f2a941034c0 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.191+0000 7f2a99757700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941130a0 con 0x7f2a94107df0 2024-05-04T03:08:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.191+0000 7f2a99757700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a941133a0 con 0x7f2a94108740 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 0x7f2a9419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 0x7f2a9419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52298/0 (socket says 172.21.15.39:52298) 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 -- 172.21.15.39:0/4185586455 learned_addr learned my addr 172.21.15.39:0/4185586455 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a927fc700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a94107df0 0x7f2a9419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a937fe700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a94108740 0x7f2a9419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 -- 172.21.15.39:0/4185586455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a94108740 msgr2=0x7f2a9419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a94108740 0x7f2a9419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 -- 172.21.15.39:0/4185586455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a94107df0 msgr2=0x7f2a9419c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a94107df0 0x7f2a9419c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 -- 172.21.15.39:0/4185586455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a941a6ef0 con 0x7f2a941034c0 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a927fc700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a94107df0 0x7f2a9419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a92ffd700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 0x7f2a9419bba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a84007a20 tx=0x7f2a84007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a8bfff700 1 -- 172.21.15.39:0/4185586455 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8401f070 con 0x7f2a941034c0 2024-05-04T03:08:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a8bfff700 1 -- 172.21.15.39:0/4185586455 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a84005500 con 0x7f2a941034c0 2024-05-04T03:08:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a941a7180 con 0x7f2a941034c0 2024-05-04T03:08:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.192+0000 7f2a8bfff700 1 -- 172.21.15.39:0/4185586455 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8401b830 con 0x7f2a941034c0 2024-05-04T03:08:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.193+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a941a75d0 con 0x7f2a941034c0 2024-05-04T03:08:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.193+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a940512c0 con 0x7f2a941034c0 2024-05-04T03:08:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.195+0000 7f2a8bfff700 1 -- 172.21.15.39:0/4185586455 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2a84022070 con 0x7f2a941034c0 2024-05-04T03:08:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.196+0000 7f2a8bfff700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a800b2990 0x7f2a800b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.196+0000 7f2a927fc700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a800b2990 0x7f2a800b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.196+0000 7f2a8bfff700 1 -- 172.21.15.39:0/4185586455 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f2a8400f020 con 0x7f2a941034c0 2024-05-04T03:08:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.196+0000 7f2a927fc700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a800b2990 0x7f2a800b4ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2a94106a50 tx=0x7f2a7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.199+0000 7f2a8bfff700 1 -- 172.21.15.39:0/4185586455 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a84029020 con 0x7f2a941034c0 2024-05-04T03:08:14.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:14 smithi039 ceph-mon[29715]: pgmap v307: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-05-04T03:08:14.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.449+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2a94063c00 con 0x7f2a800b2990 2024-05-04T03:08:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.479+0000 7f2a8bfff700 1 -- 172.21.15.39:0/4185586455 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2a94063c00 con 0x7f2a800b2990 2024-05-04T03:08:14.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a800b2990 msgr2=0x7f2a800b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:14.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a800b2990 0x7f2a800b4ea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2a94106a50 tx=0x7f2a7c009260 comp rx=0 tx=0).stop 2024-05-04T03:08:14.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 msgr2=0x7f2a9419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:14.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 0x7f2a9419bba0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a84007a20 tx=0x7f2a84007b00 comp rx=0 tx=0).stop 2024-05-04T03:08:14.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 shutdown_connections 2024-05-04T03:08:14.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a800b2990 0x7f2a800b4ea0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a941034c0 0x7f2a9419bba0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a94108740 0x7f2a9419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 --2- 172.21.15.39:0/4185586455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a94107df0 0x7f2a9419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:14.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.481+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 >> 172.21.15.39:0/4185586455 conn(0x7f2a940fedf0 msgr2=0x7f2a94101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:14.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.482+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 shutdown_connections 2024-05-04T03:08:14.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:14.482+0000 7f2a99757700 1 -- 172.21.15.39:0/4185586455 wait complete. 2024-05-04T03:08:14.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:14 smithi066 ceph-mon[33264]: pgmap v307: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-05-04T03:08:14.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:14 smithi167 ceph-mon[33614]: pgmap v307: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-05-04T03:08:15.145 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-04T03:08:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-04T03:08:15.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:15 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-04T03:08:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-04T03:08:15.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-04T03:08:15.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-04T03:08:15.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-04T03:08:15.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-05-04T03:08:15.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-04T03:08:15.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-04T03:08:15.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:15 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-04T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:15 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-04T03:08:16.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: pgmap v308: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-04T03:08:16.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:16 smithi039 ceph-mon[29715]: osdmap e79: 12 total, 12 up, 12 in 2024-05-04T03:08:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: pgmap v308: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-04T03:08:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-04T03:08:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-04T03:08:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-04T03:08:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-04T03:08:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-05-04T03:08:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-04T03:08:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-04T03:08:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-04T03:08:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:16 smithi066 ceph-mon[33264]: osdmap e79: 12 total, 12 up, 12 in 2024-05-04T03:08:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: pgmap v308: 97 pgs: 97 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8, 5, 9]}]': finished 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-04T03:08:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:16 smithi167 ceph-mon[33614]: osdmap e79: 12 total, 12 up, 12 in 2024-05-04T03:08:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.168+0000 7fd5a07c1700 1 -- 172.21.15.39:0/3997540006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598107df0 msgr2=0x7fd598108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.168+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/3997540006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598107df0 0x7fd598108200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0099c0 tx=0x7fd58c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.168+0000 7fd5a07c1700 1 -- 172.21.15.39:0/3997540006 shutdown_connections 2024-05-04T03:08:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.168+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/3997540006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd598108740 0x7fd5981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.168+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/3997540006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5981034c0 0x7fd5981038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.168+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/3997540006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598107df0 0x7fd598108200 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.168+0000 7fd5a07c1700 1 -- 172.21.15.39:0/3997540006 >> 172.21.15.39:0/3997540006 conn(0x7fd5980fedc0 msgr2=0x7fd5981011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.169+0000 7fd5a07c1700 1 -- 172.21.15.39:0/3997540006 shutdown_connections 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.169+0000 7fd5a07c1700 1 -- 172.21.15.39:0/3997540006 wait complete. 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.169+0000 7fd5a07c1700 1 Processor -- start 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.169+0000 7fd5a07c1700 1 -- start start 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd5a07c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981034c0 0x7fd59819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd5a07c1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd598107df0 0x7fd59819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd5a07c1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd598108740 0x7fd59819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd5a07c1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd598114370 con 0x7fd5981034c0 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd5a07c1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981141f0 con 0x7fd598107df0 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd5a07c1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981144f0 con 0x7fd598108740 2024-05-04T03:08:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981034c0 0x7fd59819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981034c0 0x7fd59819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33546/0 (socket says 172.21.15.39:33546) 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 -- 172.21.15.39:0/1528741452 learned_addr learned my addr 172.21.15.39:0/1528741452 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59dd5c700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd598107df0 0x7fd59819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59ed5e700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd598108740 0x7fd59819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 -- 172.21.15.39:0/1528741452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd598108740 msgr2=0x7fd59819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd598108740 0x7fd59819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 -- 172.21.15.39:0/1528741452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd598107df0 msgr2=0x7fd59819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd598107df0 0x7fd59819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 -- 172.21.15.39:0/1528741452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59819fc30 con 0x7fd5981034c0 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.170+0000 7fd59e55d700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981034c0 0x7fd59819ad60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5900060b0 tx=0x7fd59000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.171+0000 7fd5977fe700 1 -- 172.21.15.39:0/1528741452 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59001a070 con 0x7fd5981034c0 2024-05-04T03:08:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.171+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59819ff20 con 0x7fd5981034c0 2024-05-04T03:08:17.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.171+0000 7fd5977fe700 1 -- 172.21.15.39:0/1528741452 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd59000dd20 con 0x7fd5981034c0 2024-05-04T03:08:17.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.171+0000 7fd5977fe700 1 -- 172.21.15.39:0/1528741452 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590014600 con 0x7fd5981034c0 2024-05-04T03:08:17.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.171+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5981a7840 con 0x7fd5981034c0 2024-05-04T03:08:17.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.172+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5980512c0 con 0x7fd5981034c0 2024-05-04T03:08:17.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.173+0000 7fd5977fe700 1 -- 172.21.15.39:0/1528741452 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd59001d070 con 0x7fd5981034c0 2024-05-04T03:08:17.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.174+0000 7fd5977fe700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5800b2910 0x7fd5800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:17.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.174+0000 7fd59dd5c700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5800b2910 0x7fd5800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:17.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.174+0000 7fd5977fe700 1 -- 172.21.15.39:0/1528741452 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd5900e1af0 con 0x7fd5981034c0 2024-05-04T03:08:17.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.174+0000 7fd59dd5c700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5800b2910 0x7fd5800b4e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009990 tx=0x7fd58c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:17.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.179+0000 7fd5977fe700 1 -- 172.21.15.39:0/1528741452 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd590010070 con 0x7fd5981034c0 2024-05-04T03:08:17.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.428+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd59819c0c0 con 0x7fd5800b2910 2024-05-04T03:08:17.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:17 smithi039 ceph-mon[29715]: osdmap e80: 12 total, 12 up, 12 in 2024-05-04T03:08:17.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.433+0000 7fd5977fe700 1 -- 172.21.15.39:0/1528741452 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd59819c0c0 con 0x7fd5800b2910 2024-05-04T03:08:17.435 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/qa/sv_1/38b2a878-d93c-40a9-bbaf-bb51df7b7f02 2024-05-04T03:08:17.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.435+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5800b2910 msgr2=0x7fd5800b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:17.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.435+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5800b2910 0x7fd5800b4e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009990 tx=0x7fd58c005fb0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.435+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981034c0 msgr2=0x7fd59819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:17.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.435+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981034c0 0x7fd59819ad60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5900060b0 tx=0x7fd59000d3b0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.436+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 shutdown_connections 2024-05-04T03:08:17.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.436+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5800b2910 0x7fd5800b4e20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.436+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981034c0 0x7fd59819ad60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.436+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd598108740 0x7fd59819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.436+0000 7fd5a07c1700 1 --2- 172.21.15.39:0/1528741452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd598107df0 0x7fd59819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.436+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 >> 172.21.15.39:0/1528741452 conn(0x7fd5980fedc0 msgr2=0x7fd598101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.436+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 shutdown_connections 2024-05-04T03:08:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:17.436+0000 7fd5a07c1700 1 -- 172.21.15.39:0/1528741452 wait complete. 2024-05-04T03:08:17.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:17 smithi066 ceph-mon[33264]: osdmap e80: 12 total, 12 up, 12 in 2024-05-04T03:08:17.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:17 smithi167 ceph-mon[33614]: osdmap e80: 12 total, 12 up, 12 in 2024-05-04T03:08:18.155 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:08:18.412 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:18.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:18 smithi039 ceph-mon[29715]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-05-04T03:08:18.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:18 smithi039 ceph-mon[29715]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:18 smithi066 ceph-mon[33264]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-05-04T03:08:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:18 smithi066 ceph-mon[33264]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:18.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:18 smithi167 ceph-mon[33614]: pgmap v311: 97 pgs: 1 peering, 96 active+clean; 625 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-05-04T03:08:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:18 smithi167 ceph-mon[33614]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:20.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 -- 172.21.15.39:0/1457861373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8109850 msgr2=0x7fe6a81111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/1457861373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8109850 0x7fe6a81111b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000b0d0 tx=0x7fe6a000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 -- 172.21.15.39:0/1457861373 shutdown_connections 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/1457861373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8109850 0x7fe6a81111b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/1457861373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a81045d0 0x7fe6a81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/1457861373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6a8108f00 0x7fe6a8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 -- 172.21.15.39:0/1457861373 >> 172.21.15.39:0/1457861373 conn(0x7fe6a80ffea0 msgr2=0x7fe6a81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 -- 172.21.15.39:0/1457861373 shutdown_connections 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.146+0000 7fe6adf0b700 1 -- 172.21.15.39:0/1457861373 wait complete. 2024-05-04T03:08:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.147+0000 7fe6adf0b700 1 Processor -- start 2024-05-04T03:08:20.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6adf0b700 1 -- start start 2024-05-04T03:08:20.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6adf0b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6a81045d0 0x7fe6a8076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:20.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6adf0b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8108f00 0x7fe6a8077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:20.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6adf0b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a8109850 0x7fe6a8077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:20.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6adf0b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81143a0 con 0x7fe6a8108f00 2024-05-04T03:08:20.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6adf0b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8114220 con 0x7fe6a81045d0 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6adf0b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8114520 con 0x7fe6a8109850 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8108f00 0x7fe6a8077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8108f00 0x7fe6a8077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33570/0 (socket says 172.21.15.39:33570) 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a6ffd700 1 -- 172.21.15.39:0/417776950 learned_addr learned my addr 172.21.15.39:0/417776950 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a7fff700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a8109850 0x7fe6a8077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a77fe700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6a81045d0 0x7fe6a8076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a6ffd700 1 -- 172.21.15.39:0/417776950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a8109850 msgr2=0x7fe6a8077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a6ffd700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a8109850 0x7fe6a8077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a6ffd700 1 -- 172.21.15.39:0/417776950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6a81045d0 msgr2=0x7fe6a8076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a6ffd700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6a81045d0 0x7fe6a8076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.148+0000 7fe6a6ffd700 1 -- 172.21.15.39:0/417776950 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a81a90b0 con 0x7fe6a8108f00 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.149+0000 7fe6a6ffd700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8108f00 0x7fe6a8077110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b500 tx=0x7fe69800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.149+0000 7fe6a4ff9700 1 -- 172.21.15.39:0/417776950 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698015020 con 0x7fe6a8108f00 2024-05-04T03:08:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.149+0000 7fe6a4ff9700 1 -- 172.21.15.39:0/417776950 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe698011cc0 con 0x7fe6a8108f00 2024-05-04T03:08:20.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.149+0000 7fe6a4ff9700 1 -- 172.21.15.39:0/417776950 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698010ba0 con 0x7fe6a8108f00 2024-05-04T03:08:20.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.149+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a81a93a0 con 0x7fe6a8108f00 2024-05-04T03:08:20.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.149+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a81a9930 con 0x7fe6a8108f00 2024-05-04T03:08:20.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.150+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a80512c0 con 0x7fe6a8108f00 2024-05-04T03:08:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.152+0000 7fe6a4ff9700 1 -- 172.21.15.39:0/417776950 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe698018020 con 0x7fe6a8108f00 2024-05-04T03:08:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.152+0000 7fe6a4ff9700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6900b2990 0x7fe6900b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.152+0000 7fe6a77fe700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6900b2990 0x7fe6900b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.152+0000 7fe6a4ff9700 1 -- 172.21.15.39:0/417776950 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe6980e1f00 con 0x7fe6a8108f00 2024-05-04T03:08:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.153+0000 7fe6a77fe700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6900b2990 0x7fe6900b4ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe69c005950 tx=0x7fe69c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:20.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.156+0000 7fe6a4ff9700 1 -- 172.21.15.39:0/417776950 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe69801f020 con 0x7fe6a8108f00 2024-05-04T03:08:20.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.423+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6a8078100 con 0x7fe6a8108f00 2024-05-04T03:08:20.424 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:20 smithi039 ceph-mon[29715]: pgmap v312: 97 pgs: 1 peering, 96 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:20.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.424+0000 7fe6a4ff9700 1 -- 172.21.15.39:0/417776950 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7fe6980a9be0 con 0x7fe6a8108f00 2024-05-04T03:08:20.425 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:08:20.425 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:08:01.344495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24455,24490,24466,24472,24506]},"id":1}]} 2024-05-04T03:08:20.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6900b2990 msgr2=0x7fe6900b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6900b2990 0x7fe6900b4ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe69c005950 tx=0x7fe69c00b410 comp rx=0 tx=0).stop 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8108f00 msgr2=0x7fe6a8077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8108f00 0x7fe6a8077110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b500 tx=0x7fe69800b810 comp rx=0 tx=0).stop 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 shutdown_connections 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6900b2990 0x7fe6900b4ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a8109850 0x7fe6a8077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6a81045d0 0x7fe6a8076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 --2- 172.21.15.39:0/417776950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a8108f00 0x7fe6a8077110 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 >> 172.21.15.39:0/417776950 conn(0x7fe6a80ffea0 msgr2=0x7fe6a81086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 shutdown_connections 2024-05-04T03:08:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:20.426+0000 7fe6adf0b700 1 -- 172.21.15.39:0/417776950 wait complete. 2024-05-04T03:08:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-04T03:08:20.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:20 smithi066 ceph-mon[33264]: pgmap v312: 97 pgs: 1 peering, 96 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:20.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:20 smithi167 ceph-mon[33614]: pgmap v312: 97 pgs: 1 peering, 96 active+clean; 625 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:21.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:08:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:21.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:21 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/417776950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:08:21.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:21 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/417776950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:08:21.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:21 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/417776950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:08:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:22 smithi066 ceph-mon[33264]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 12 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:22.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:22 smithi039 ceph-mon[29715]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 12 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:22 smithi167 ceph-mon[33614]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 12 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:23.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/1262121971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c41045e0 msgr2=0x7fa0c41049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/1262121971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c41045e0 0x7fa0c41049f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40099c0 tx=0x7fa0b4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/1262121971 shutdown_connections 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/1262121971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c41045e0 0x7fa0c41049f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/1262121971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c4109860 0x7fa0c41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/1262121971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c4108f10 0x7fa0c4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/1262121971 >> 172.21.15.39:0/1262121971 conn(0x7fa0c40ffed0 msgr2=0x7fa0c4102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/1262121971 shutdown_connections 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/1262121971 wait complete. 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 Processor -- start 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.174+0000 7fa0ca8a7700 1 -- start start 2024-05-04T03:08:23.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0ca8a7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c41045e0 0x7fa0c4076b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0ca8a7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c4108f10 0x7fa0c40770d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0ca8a7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c4109860 0x7fa0c4077610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0ca8a7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4114430 con 0x7fa0c4108f10 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0ca8a7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c41142b0 con 0x7fa0c4109860 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0ca8a7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c41145b0 con 0x7fa0c41045e0 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c4108f10 0x7fa0c40770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c4108f10 0x7fa0c40770d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33588/0 (socket says 172.21.15.39:33588) 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c37fe700 1 -- 172.21.15.39:0/3798595289 learned_addr learned my addr 172.21.15.39:0/3798595289 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c3fff700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c41045e0 0x7fa0c4076b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c8e44700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c4109860 0x7fa0c4077610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c37fe700 1 -- 172.21.15.39:0/3798595289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c41045e0 msgr2=0x7fa0c4076b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c37fe700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c41045e0 0x7fa0c4076b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c37fe700 1 -- 172.21.15.39:0/3798595289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c4109860 msgr2=0x7fa0c4077610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c37fe700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c4109860 0x7fa0c4077610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.175+0000 7fa0c37fe700 1 -- 172.21.15.39:0/3798595289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c41a91d0 con 0x7fa0c4108f10 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.176+0000 7fa0c37fe700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c4108f10 0x7fa0c40770d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b500 tx=0x7fa0b800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.176+0000 7fa0c17fa700 1 -- 172.21.15.39:0/3798595289 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8015020 con 0x7fa0c4108f10 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.176+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c41a94c0 con 0x7fa0c4108f10 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.176+0000 7fa0c17fa700 1 -- 172.21.15.39:0/3798595289 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0b8011cc0 con 0x7fa0c4108f10 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.176+0000 7fa0c17fa700 1 -- 172.21.15.39:0/3798595289 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8010cb0 con 0x7fa0c4108f10 2024-05-04T03:08:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.176+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c41a9a00 con 0x7fa0c4108f10 2024-05-04T03:08:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.177+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c40512c0 con 0x7fa0c4108f10 2024-05-04T03:08:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.179+0000 7fa0c17fa700 1 -- 172.21.15.39:0/3798595289 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa0b8018020 con 0x7fa0c4108f10 2024-05-04T03:08:23.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.180+0000 7fa0c17fa700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0ac0b2910 0x7fa0ac0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:23.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.180+0000 7fa0c3fff700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0ac0b2910 0x7fa0ac0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:23.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.180+0000 7fa0c17fa700 1 -- 172.21.15.39:0/3798595289 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa0b80e1f00 con 0x7fa0c4108f10 2024-05-04T03:08:23.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.181+0000 7fa0c3fff700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0ac0b2910 0x7fa0ac0b4e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:23.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.184+0000 7fa0c17fa700 1 -- 172.21.15.39:0/3798595289 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0b801f020 con 0x7fa0c4108f10 2024-05-04T03:08:23.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.399+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0c40780c0 con 0x7fa0c4108f10 2024-05-04T03:08:23.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.400+0000 7fa0c17fa700 1 -- 172.21.15.39:0/3798595289 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22541 (secure 0 0 0) 0x7fa0b80a9be0 con 0x7fa0c4108f10 2024-05-04T03:08:23.401 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:08:23.401 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:08:16.237923+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:08:23.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.402+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0ac0b2910 msgr2=0x7fa0ac0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:23.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.402+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0ac0b2910 0x7fa0ac0b4e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4005fb0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.402+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c4108f10 msgr2=0x7fa0c40770d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:23.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.402+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c4108f10 0x7fa0c40770d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b500 tx=0x7fa0b800b810 comp rx=0 tx=0).stop 2024-05-04T03:08:23.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.403+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 shutdown_connections 2024-05-04T03:08:23.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.403+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0ac0b2910 0x7fa0ac0b4e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.403+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c4109860 0x7fa0c4077610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.403+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c41045e0 0x7fa0c4076b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.403+0000 7fa0ca8a7700 1 --2- 172.21.15.39:0/3798595289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0c4108f10 0x7fa0c40770d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:23.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.403+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 >> 172.21.15.39:0/3798595289 conn(0x7fa0c40ffed0 msgr2=0x7fa0c41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:23.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.403+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 shutdown_connections 2024-05-04T03:08:23.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:23.403+0000 7fa0ca8a7700 1 -- 172.21.15.39:0/3798595289 wait complete. 2024-05-04T03:08:24.082 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-04T03:08:24.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:08:24.335 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:24.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:24 smithi039 ceph-mon[29715]: pgmap v314: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:24.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:24 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3798595289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:24 smithi066 ceph-mon[33264]: pgmap v314: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:24 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3798595289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:24.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:24 smithi167 ceph-mon[33614]: pgmap v314: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:24.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:24 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3798595289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 -- 172.21.15.39:0/1686938897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978102ae0 msgr2=0x7f8978102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 --2- 172.21.15.39:0/1686938897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978102ae0 0x7f8978102ef0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f89680099c0 tx=0x7f8968009cd0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 -- 172.21.15.39:0/1686938897 shutdown_connections 2024-05-04T03:08:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 --2- 172.21.15.39:0/1686938897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978102ae0 0x7f8978102ef0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 --2- 172.21.15.39:0/1686938897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8978109850 0x7f89781111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 --2- 172.21.15.39:0/1686938897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8978108f00 0x7f8978109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 -- 172.21.15.39:0/1686938897 >> 172.21.15.39:0/1686938897 conn(0x7f89780fe4e0 msgr2=0x7f8978100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 -- 172.21.15.39:0/1686938897 shutdown_connections 2024-05-04T03:08:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.195+0000 7f897cb29700 1 -- 172.21.15.39:0/1686938897 wait complete. 2024-05-04T03:08:26.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897cb29700 1 Processor -- start 2024-05-04T03:08:26.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897cb29700 1 -- start start 2024-05-04T03:08:26.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897cb29700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8978102ae0 0x7f897819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:26.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897cb29700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8978108f00 0x7f897819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:26.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897cb29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978109850 0x7f897819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:26.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897cb29700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8978114330 con 0x7f8978109850 2024-05-04T03:08:26.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897cb29700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89781141b0 con 0x7f8978102ae0 2024-05-04T03:08:26.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897cb29700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89781144b0 con 0x7f8978108f00 2024-05-04T03:08:26.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f8976d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978109850 0x7f897819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f8976d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978109850 0x7f897819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58232/0 (socket says 172.21.15.39:58232) 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f8976d9d700 1 -- 172.21.15.39:0/600883812 learned_addr learned my addr 172.21.15.39:0/600883812 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.196+0000 7f897659c700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8978102ae0 0x7f897819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f8975d9b700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8978108f00 0x7f897819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f8976d9d700 1 -- 172.21.15.39:0/600883812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8978108f00 msgr2=0x7f897819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f8976d9d700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8978108f00 0x7f897819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f8976d9d700 1 -- 172.21.15.39:0/600883812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8978102ae0 msgr2=0x7f897819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f8976d9d700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8978102ae0 0x7f897819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f8976d9d700 1 -- 172.21.15.39:0/600883812 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897819fcb0 con 0x7f8978109850 2024-05-04T03:08:26.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f8976d9d700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978109850 0x7f897819f610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f897000ada0 tx=0x7f897000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:26.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f89677fe700 1 -- 172.21.15.39:0/600883812 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970017020 con 0x7f8978109850 2024-05-04T03:08:26.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f89677fe700 1 -- 172.21.15.39:0/600883812 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f897000cd60 con 0x7f8978109850 2024-05-04T03:08:26.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f89677fe700 1 -- 172.21.15.39:0/600883812 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970005280 con 0x7f8978109850 2024-05-04T03:08:26.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897819ffa0 con 0x7f8978109850 2024-05-04T03:08:26.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.197+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89781a78c0 con 0x7f8978109850 2024-05-04T03:08:26.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.198+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89780512c0 con 0x7f8978109850 2024-05-04T03:08:26.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.200+0000 7f89677fe700 1 -- 172.21.15.39:0/600883812 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f897000c8d0 con 0x7f8978109850 2024-05-04T03:08:26.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.200+0000 7f89677fe700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89600b2990 0x7f89600b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:26.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.200+0000 7f897659c700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89600b2990 0x7f89600b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:26.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.201+0000 7f89677fe700 1 -- 172.21.15.39:0/600883812 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f89700e1f90 con 0x7f8978109850 2024-05-04T03:08:26.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.201+0000 7f897659c700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89600b2990 0x7f89600b4ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8968000c00 tx=0x7f8968005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:26.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.204+0000 7f89677fe700 1 -- 172.21.15.39:0/600883812 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8970021020 con 0x7f8978109850 2024-05-04T03:08:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.412+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f897819c140 con 0x7f89600b2990 2024-05-04T03:08:26.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.420+0000 7f89677fe700 1 -- 172.21.15.39:0/600883812 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258918 (secure 0 0 0) 0x7f8968067380 con 0x7f89600b2990 2024-05-04T03:08:26.428 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:08:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89600b2990 msgr2=0x7f89600b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89600b2990 0x7f89600b4ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8968000c00 tx=0x7f8968005fb0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978109850 msgr2=0x7f897819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978109850 0x7f897819f610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f897000ada0 tx=0x7f897000c330 comp rx=0 tx=0).stop 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 shutdown_connections 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89600b2990 0x7f89600b4ea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8978109850 0x7f897819f610 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8978102ae0 0x7f897819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 --2- 172.21.15.39:0/600883812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8978108f00 0x7f897819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.424+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 >> 172.21.15.39:0/600883812 conn(0x7f89780fe4e0 msgr2=0x7f897810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.425+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 shutdown_connections 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:26.425+0000 7f897cb29700 1 -- 172.21.15.39:0/600883812 wait complete. 2024-05-04T03:08:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-04T03:08:26.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:26 smithi066 ceph-mon[33264]: pgmap v315: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 12 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:26.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:26 smithi167 ceph-mon[33614]: pgmap v315: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 12 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:26.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:26 smithi039 ceph-mon[29715]: pgmap v315: 97 pgs: 97 active+clean; 633 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 12 op/s; 6 B/s, 1 keys/s, 1 objects/s recovering 2024-05-04T03:08:27.124 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-05-04T03:08:25.004308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648110,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":354,"num_read_kb":266,"num_write":363,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376624,"kb_used_data":10784,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565200,"statfs":{"total":1151940427776,"available":1151554764800,"internally_reserved":0,"allocated":11042816,"data_stored":7049984,"data_compressed":109203,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28482,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":103,"num_read_kb":79,"num_write":19,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.764108"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247282+0000","last_change":"2024-05-04T03:07:37.070347+0000","last_active":"2024-05-04T03:08:16.247282+0000","last_peered":"2024-05-04T03:08:16.247282+0000","last_clean":"2024-05-04T03:08:16.247282+0000","last_became_active":"2024-05-04T03:07:37.069997+0000","last_became_peered":"2024-05-04T03:07:37.069997+0000","last_unstale":"2024-05-04T03:08:16.247282+0000","last_undegraded":"2024-05-04T03:08:16.247282+0000","last_fullsized":"2024-05-04T03:08:16.247282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:24:11.362228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247260+0000","last_change":"2024-05-04T03:07:37.071598+0000","last_active":"2024-05-04T03:08:16.247260+0000","last_peered":"2024-05-04T03:08:16.247260+0000","last_clean":"2024-05-04T03:08:16.247260+0000","last_became_active":"2024-05-04T03:07:37.071059+0000","last_became_peered":"2024-05-04T03:07:37.071059+0000","last_unstale":"2024-05-04T03:08:16.247260+0000","last_undegraded":"2024-05-04T03:08:16.247260+0000","last_fullsized":"2024-05-04T03:08:16.247260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:33:33.929246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246145+0000","last_change":"2024-05-04T03:07:37.069951+0000","last_active":"2024-05-04T03:08:16.246145+0000","last_peered":"2024-05-04T03:08:16.246145+0000","last_clean":"2024-05-04T03:08:16.246145+0000","last_became_active":"2024-05-04T03:07:37.069558+0000","last_became_peered":"2024-05-04T03:07:37.069558+0000","last_unstale":"2024-05-04T03:08:16.246145+0000","last_undegraded":"2024-05-04T03:08:16.246145+0000","last_fullsized":"2024-05-04T03:08:16.246145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:37:52.596455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367260+0000","last_change":"2024-05-04T03:07:37.068820+0000","last_active":"2024-05-04T03:08:16.367260+0000","last_peered":"2024-05-04T03:08:16.367260+0000","last_clean":"2024-05-04T03:08:16.367260+0000","last_became_active":"2024-05-04T03:07:37.068323+0000","last_became_peered":"2024-05-04T03:07:37.068323+0000","last_unstale":"2024-05-04T03:08:16.367260+0000","last_undegraded":"2024-05-04T03:08:16.367260+0000","last_fullsized":"2024-05-04T03:08:16.367260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:52:54.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245207+0000","last_change":"2024-05-04T03:07:37.061982+0000","last_active":"2024-05-04T03:08:16.245207+0000","last_peered":"2024-05-04T03:08:16.245207+0000","last_clean":"2024-05-04T03:08:16.245207+0000","last_became_active":"2024-05-04T03:07:37.061621+0000","last_became_peered":"2024-05-04T03:07:37.061621+0000","last_unstale":"2024-05-04T03:08:16.245207+0000","last_undegraded":"2024-05-04T03:08:16.245207+0000","last_fullsized":"2024-05-04T03:08:16.245207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:17:47.421011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248166+0000","last_change":"2024-05-04T03:07:37.058641+0000","last_active":"2024-05-04T03:08:16.248166+0000","last_peered":"2024-05-04T03:08:16.248166+0000","last_clean":"2024-05-04T03:08:16.248166+0000","last_became_active":"2024-05-04T03:07:37.058235+0000","last_became_peered":"2024-05-04T03:07:37.058235+0000","last_unstale":"2024-05-04T03:08:16.248166+0000","last_undegraded":"2024-05-04T03:08:16.248166+0000","last_fullsized":"2024-05-04T03:08:16.248166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:17:45.441777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244522+0000","last_change":"2024-05-04T03:07:37.065959+0000","last_active":"2024-05-04T03:08:16.244522+0000","last_peered":"2024-05-04T03:08:16.244522+0000","last_clean":"2024-05-04T03:08:16.244522+0000","last_became_active":"2024-05-04T03:07:37.065508+0000","last_became_peered":"2024-05-04T03:07:37.065508+0000","last_unstale":"2024-05-04T03:08:16.244522+0000","last_undegraded":"2024-05-04T03:08:16.244522+0000","last_fullsized":"2024-05-04T03:08:16.244522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:17.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243478+0000","last_change":"2024-05-04T03:07:37.065620+0000","last_active":"2024-05-04T03:08:16.243478+0000","last_peered":"2024-05-04T03:08:16.243478+0000","last_clean":"2024-05-04T03:08:16.243478+0000","last_became_active":"2024-05-04T03:07:37.065261+0000","last_became_peered":"2024-05-04T03:07:37.065261+0000","last_unstale":"2024-05-04T03:08:16.243478+0000","last_undegraded":"2024-05-04T03:08:16.243478+0000","last_fullsized":"2024-05-04T03:08:16.243478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:27.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242930+0000","last_change":"2024-05-04T03:07:37.072204+0000","last_active":"2024-05-04T03:08:16.242930+0000","last_peered":"2024-05-04T03:08:16.242930+0000","last_clean":"2024-05-04T03:08:16.242930+0000","last_became_active":"2024-05-04T03:07:37.070622+0000","last_became_peered":"2024-05-04T03:07:37.070622+0000","last_unstale":"2024-05-04T03:08:16.242930+0000","last_undegraded":"2024-05-04T03:08:16.242930+0000","last_fullsized":"2024-05-04T03:08:16.242930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:55:25.931369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.253125+0000","last_change":"2024-05-04T03:08:16.253125+0000","last_active":"2024-05-04T03:08:16.253125+0000","last_peered":"2024-05-04T03:08:16.253125+0000","last_clean":"2024-05-04T03:08:16.253125+0000","last_became_active":"2024-05-04T03:08:16.252682+0000","last_became_peered":"2024-05-04T03:08:16.252682+0000","last_unstale":"2024-05-04T03:08:16.253125+0000","last_undegraded":"2024-05-04T03:08:16.253125+0000","last_fullsized":"2024-05-04T03:08:16.253125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:05.933491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244798+0000","last_change":"2024-05-04T03:07:37.071614+0000","last_active":"2024-05-04T03:08:16.244798+0000","last_peered":"2024-05-04T03:08:16.244798+0000","last_clean":"2024-05-04T03:08:16.244798+0000","last_became_active":"2024-05-04T03:07:37.070927+0000","last_became_peered":"2024-05-04T03:07:37.070927+0000","last_unstale":"2024-05-04T03:08:16.244798+0000","last_undegraded":"2024-05-04T03:08:16.244798+0000","last_fullsized":"2024-05-04T03:08:16.244798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:30.432959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367227+0000","last_change":"2024-05-04T03:07:37.069305+0000","last_active":"2024-05-04T03:08:16.367227+0000","last_peered":"2024-05-04T03:08:16.367227+0000","last_clean":"2024-05-04T03:08:16.367227+0000","last_became_active":"2024-05-04T03:07:37.068804+0000","last_became_peered":"2024-05-04T03:07:37.068804+0000","last_unstale":"2024-05-04T03:08:16.367227+0000","last_undegraded":"2024-05-04T03:08:16.367227+0000","last_fullsized":"2024-05-04T03:08:16.367227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:43.693161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248895+0000","last_change":"2024-05-04T03:07:37.062848+0000","last_active":"2024-05-04T03:08:16.248895+0000","last_peered":"2024-05-04T03:08:16.248895+0000","last_clean":"2024-05-04T03:08:16.248895+0000","last_became_active":"2024-05-04T03:07:37.062548+0000","last_became_peered":"2024-05-04T03:07:37.062548+0000","last_unstale":"2024-05-04T03:08:16.248895+0000","last_undegraded":"2024-05-04T03:08:16.248895+0000","last_fullsized":"2024-05-04T03:08:16.248895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246059+0000","last_change":"2024-05-04T03:07:37.062598+0000","last_active":"2024-05-04T03:08:16.246059+0000","last_peered":"2024-05-04T03:08:16.246059+0000","last_clean":"2024-05-04T03:08:16.246059+0000","last_became_active":"2024-05-04T03:07:37.061711+0000","last_became_peered":"2024-05-04T03:07:37.061711+0000","last_unstale":"2024-05-04T03:08:16.246059+0000","last_undegraded":"2024-05-04T03:08:16.246059+0000","last_fullsized":"2024-05-04T03:08:16.246059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:29:03.153680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246387+0000","last_change":"2024-05-04T03:07:37.067889+0000","last_active":"2024-05-04T03:08:16.246387+0000","last_peered":"2024-05-04T03:08:16.246387+0000","last_clean":"2024-05-04T03:08:16.246387+0000","last_became_active":"2024-05-04T03:07:37.066768+0000","last_became_peered":"2024-05-04T03:07:37.066768+0000","last_unstale":"2024-05-04T03:08:16.246387+0000","last_undegraded":"2024-05-04T03:08:16.246387+0000","last_fullsized":"2024-05-04T03:08:16.246387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:01:18.136379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242305+0000","last_change":"2024-05-04T03:07:37.070438+0000","last_active":"2024-05-04T03:08:16.242305+0000","last_peered":"2024-05-04T03:08:16.242305+0000","last_clean":"2024-05-04T03:08:16.242305+0000","last_became_active":"2024-05-04T03:07:37.070007+0000","last_became_peered":"2024-05-04T03:07:37.070007+0000","last_unstale":"2024-05-04T03:08:16.242305+0000","last_undegraded":"2024-05-04T03:08:16.242305+0000","last_fullsized":"2024-05-04T03:08:16.242305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:22.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247556+0000","last_change":"2024-05-04T03:07:37.070020+0000","last_active":"2024-05-04T03:08:16.247556+0000","last_peered":"2024-05-04T03:08:16.247556+0000","last_clean":"2024-05-04T03:08:16.247556+0000","last_became_active":"2024-05-04T03:07:37.069629+0000","last_became_peered":"2024-05-04T03:07:37.069629+0000","last_unstale":"2024-05-04T03:08:16.247556+0000","last_undegraded":"2024-05-04T03:08:16.247556+0000","last_fullsized":"2024-05-04T03:08:16.247556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:28.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245734+0000","last_change":"2024-05-04T03:07:37.068651+0000","last_active":"2024-05-04T03:08:16.245734+0000","last_peered":"2024-05-04T03:08:16.245734+0000","last_clean":"2024-05-04T03:08:16.245734+0000","last_became_active":"2024-05-04T03:07:37.067135+0000","last_became_peered":"2024-05-04T03:07:37.067135+0000","last_unstale":"2024-05-04T03:08:16.245734+0000","last_undegraded":"2024-05-04T03:08:16.245734+0000","last_fullsized":"2024-05-04T03:08:16.245734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:00:59.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250832+0000","last_change":"2024-05-04T03:07:37.055419+0000","last_active":"2024-05-04T03:08:16.250832+0000","last_peered":"2024-05-04T03:08:16.250832+0000","last_clean":"2024-05-04T03:08:16.250832+0000","last_became_active":"2024-05-04T03:07:37.055024+0000","last_became_peered":"2024-05-04T03:07:37.055024+0000","last_unstale":"2024-05-04T03:08:16.250832+0000","last_undegraded":"2024-05-04T03:08:16.250832+0000","last_fullsized":"2024-05-04T03:08:16.250832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:55:05.506505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:22.218342+0000","last_change":"2024-05-04T03:07:34.019975+0000","last_active":"2024-05-04T03:08:22.218342+0000","last_peered":"2024-05-04T03:08:22.218342+0000","last_clean":"2024-05-04T03:08:22.218342+0000","last_became_active":"2024-05-04T03:07:34.019532+0000","last_became_peered":"2024-05-04T03:07:34.019532+0000","last_unstale":"2024-05-04T03:08:22.218342+0000","last_undegraded":"2024-05-04T03:08:22.218342+0000","last_fullsized":"2024-05-04T03:08:22.218342+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:11:03.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243957+0000","last_change":"2024-05-04T03:07:37.064725+0000","last_active":"2024-05-04T03:08:16.243957+0000","last_peered":"2024-05-04T03:08:16.243957+0000","last_clean":"2024-05-04T03:08:16.243957+0000","last_became_active":"2024-05-04T03:07:37.064385+0000","last_became_peered":"2024-05-04T03:07:37.064385+0000","last_unstale":"2024-05-04T03:08:16.243957+0000","last_undegraded":"2024-05-04T03:08:16.243957+0000","last_fullsized":"2024-05-04T03:08:16.243957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:22:12.024328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244081+0000","last_change":"2024-05-04T03:07:34.157179+0000","last_active":"2024-05-04T03:08:16.244081+0000","last_peered":"2024-05-04T03:08:16.244081+0000","last_clean":"2024-05-04T03:08:16.244081+0000","last_became_active":"2024-05-04T03:07:34.156790+0000","last_became_peered":"2024-05-04T03:07:34.156790+0000","last_unstale":"2024-05-04T03:08:16.244081+0000","last_undegraded":"2024-05-04T03:08:16.244081+0000","last_fullsized":"2024-05-04T03:08:16.244081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:32.066410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246454+0000","last_change":"2024-05-04T03:07:37.070481+0000","last_active":"2024-05-04T03:08:16.246454+0000","last_peered":"2024-05-04T03:08:16.246454+0000","last_clean":"2024-05-04T03:08:16.246454+0000","last_became_active":"2024-05-04T03:07:37.069903+0000","last_became_peered":"2024-05-04T03:07:37.069903+0000","last_unstale":"2024-05-04T03:08:16.246454+0000","last_undegraded":"2024-05-04T03:08:16.246454+0000","last_fullsized":"2024-05-04T03:08:16.246454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:45:40.764716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245723+0000","last_change":"2024-05-04T03:07:34.023978+0000","last_active":"2024-05-04T03:08:16.245723+0000","last_peered":"2024-05-04T03:08:16.245723+0000","last_clean":"2024-05-04T03:08:16.245723+0000","last_became_active":"2024-05-04T03:07:34.023199+0000","last_became_peered":"2024-05-04T03:07:34.023199+0000","last_unstale":"2024-05-04T03:08:16.245723+0000","last_undegraded":"2024-05-04T03:08:16.245723+0000","last_fullsized":"2024-05-04T03:08:16.245723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:17:43.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242450+0000","last_change":"2024-05-04T03:07:37.072661+0000","last_active":"2024-05-04T03:08:16.242450+0000","last_peered":"2024-05-04T03:08:16.242450+0000","last_clean":"2024-05-04T03:08:16.242450+0000","last_became_active":"2024-05-04T03:07:37.071393+0000","last_became_peered":"2024-05-04T03:07:37.071393+0000","last_unstale":"2024-05-04T03:08:16.242450+0000","last_undegraded":"2024-05-04T03:08:16.242450+0000","last_fullsized":"2024-05-04T03:08:16.242450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:39:39.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245493+0000","last_change":"2024-05-04T03:07:34.018543+0000","last_active":"2024-05-04T03:08:16.245493+0000","last_peered":"2024-05-04T03:08:16.245493+0000","last_clean":"2024-05-04T03:08:16.245493+0000","last_became_active":"2024-05-04T03:07:34.018145+0000","last_became_peered":"2024-05-04T03:07:34.018145+0000","last_unstale":"2024-05-04T03:08:16.245493+0000","last_undegraded":"2024-05-04T03:08:16.245493+0000","last_fullsized":"2024-05-04T03:08:16.245493+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:44.529492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248677+0000","last_change":"2024-05-04T03:07:37.070811+0000","last_active":"2024-05-04T03:08:16.248677+0000","last_peered":"2024-05-04T03:08:16.248677+0000","last_clean":"2024-05-04T03:08:16.248677+0000","last_became_active":"2024-05-04T03:07:37.070441+0000","last_became_peered":"2024-05-04T03:07:37.070441+0000","last_unstale":"2024-05-04T03:08:16.248677+0000","last_undegraded":"2024-05-04T03:08:16.248677+0000","last_fullsized":"2024-05-04T03:08:16.248677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:01:48.146339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":56,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:19.215648+0000","last_change":"2024-05-04T03:07:34.020508+0000","last_active":"2024-05-04T03:08:19.215648+0000","last_peered":"2024-05-04T03:08:19.215648+0000","last_clean":"2024-05-04T03:08:19.215648+0000","last_became_active":"2024-05-04T03:07:34.020076+0000","last_became_peered":"2024-05-04T03:07:34.020076+0000","last_unstale":"2024-05-04T03:08:19.215648+0000","last_undegraded":"2024-05-04T03:08:19.215648+0000","last_fullsized":"2024-05-04T03:08:19.215648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:27.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245429+0000","last_change":"2024-05-04T03:07:37.060004+0000","last_active":"2024-05-04T03:08:16.245429+0000","last_peered":"2024-05-04T03:08:16.245429+0000","last_clean":"2024-05-04T03:08:16.245429+0000","last_became_active":"2024-05-04T03:07:37.059350+0000","last_became_peered":"2024-05-04T03:07:37.059350+0000","last_unstale":"2024-05-04T03:08:16.245429+0000","last_undegraded":"2024-05-04T03:08:16.245429+0000","last_fullsized":"2024-05-04T03:08:16.245429+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:37:07.315935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248068+0000","last_change":"2024-05-04T03:07:34.020174+0000","last_active":"2024-05-04T03:08:16.248068+0000","last_peered":"2024-05-04T03:08:16.248068+0000","last_clean":"2024-05-04T03:08:16.248068+0000","last_became_active":"2024-05-04T03:07:34.019790+0000","last_became_peered":"2024-05-04T03:07:34.019790+0000","last_unstale":"2024-05-04T03:08:16.248068+0000","last_undegraded":"2024-05-04T03:08:16.248068+0000","last_fullsized":"2024-05-04T03:08:16.248068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:37:37.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.252143+0000","last_change":"2024-05-04T03:08:16.251277+0000","last_active":"2024-05-04T03:08:16.252143+0000","last_peered":"2024-05-04T03:08:16.252143+0000","last_clean":"2024-05-04T03:08:16.252143+0000","last_became_active":"2024-05-04T03:08:16.250836+0000","last_became_peered":"2024-05-04T03:08:16.250836+0000","last_unstale":"2024-05-04T03:08:16.252143+0000","last_undegraded":"2024-05-04T03:08:16.252143+0000","last_fullsized":"2024-05-04T03:08:16.252143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:59:47.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248397+0000","last_change":"2024-05-04T03:07:34.013188+0000","last_active":"2024-05-04T03:08:16.248397+0000","last_peered":"2024-05-04T03:08:16.248397+0000","last_clean":"2024-05-04T03:08:16.248397+0000","last_became_active":"2024-05-04T03:07:34.012759+0000","last_became_peered":"2024-05-04T03:07:34.012759+0000","last_unstale":"2024-05-04T03:08:16.248397+0000","last_undegraded":"2024-05-04T03:08:16.248397+0000","last_fullsized":"2024-05-04T03:08:16.248397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:54:29.621250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243425+0000","last_change":"2024-05-04T03:07:37.056265+0000","last_active":"2024-05-04T03:08:16.243425+0000","last_peered":"2024-05-04T03:08:16.243425+0000","last_clean":"2024-05-04T03:08:16.243425+0000","last_became_active":"2024-05-04T03:07:37.055957+0000","last_became_peered":"2024-05-04T03:07:37.055957+0000","last_unstale":"2024-05-04T03:08:16.243425+0000","last_undegraded":"2024-05-04T03:08:16.243425+0000","last_fullsized":"2024-05-04T03:08:16.243425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:41.739034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247855+0000","last_change":"2024-05-04T03:07:34.020722+0000","last_active":"2024-05-04T03:08:16.247855+0000","last_peered":"2024-05-04T03:08:16.247855+0000","last_clean":"2024-05-04T03:08:16.247855+0000","last_became_active":"2024-05-04T03:07:34.020028+0000","last_became_peered":"2024-05-04T03:07:34.020028+0000","last_unstale":"2024-05-04T03:08:16.247855+0000","last_undegraded":"2024-05-04T03:08:16.247855+0000","last_fullsized":"2024-05-04T03:08:16.247855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:23:20.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245364+0000","last_change":"2024-05-04T03:07:37.059680+0000","last_active":"2024-05-04T03:08:16.245364+0000","last_peered":"2024-05-04T03:08:16.245364+0000","last_clean":"2024-05-04T03:08:16.245364+0000","last_became_active":"2024-05-04T03:07:37.059294+0000","last_became_peered":"2024-05-04T03:07:37.059294+0000","last_unstale":"2024-05-04T03:08:16.245364+0000","last_undegraded":"2024-05-04T03:08:16.245364+0000","last_fullsized":"2024-05-04T03:08:16.245364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:50:06.251706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":76,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:20.216105+0000","last_change":"2024-05-04T03:07:34.019055+0000","last_active":"2024-05-04T03:08:20.216105+0000","last_peered":"2024-05-04T03:08:20.216105+0000","last_clean":"2024-05-04T03:08:20.216105+0000","last_became_active":"2024-05-04T03:07:34.018638+0000","last_became_peered":"2024-05-04T03:07:34.018638+0000","last_unstale":"2024-05-04T03:08:20.216105+0000","last_undegraded":"2024-05-04T03:08:20.216105+0000","last_fullsized":"2024-05-04T03:08:20.216105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:58:01.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242844+0000","last_change":"2024-05-04T03:07:37.063037+0000","last_active":"2024-05-04T03:08:16.242844+0000","last_peered":"2024-05-04T03:08:16.242844+0000","last_clean":"2024-05-04T03:08:16.242844+0000","last_became_active":"2024-05-04T03:07:37.062350+0000","last_became_peered":"2024-05-04T03:07:37.062350+0000","last_unstale":"2024-05-04T03:08:16.242844+0000","last_undegraded":"2024-05-04T03:08:16.242844+0000","last_fullsized":"2024-05-04T03:08:16.242844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:41:15.898241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.373765+0000","last_change":"2024-05-04T03:08:16.373765+0000","last_active":"2024-05-04T03:08:16.373765+0000","last_peered":"2024-05-04T03:08:16.373765+0000","last_clean":"2024-05-04T03:08:16.373765+0000","last_became_active":"2024-05-04T03:08:16.368588+0000","last_became_peered":"2024-05-04T03:08:16.368588+0000","last_unstale":"2024-05-04T03:08:16.373765+0000","last_undegraded":"2024-05-04T03:08:16.373765+0000","last_fullsized":"2024-05-04T03:08:16.373765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:55:07.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246604+0000","last_change":"2024-05-04T03:07:37.071330+0000","last_active":"2024-05-04T03:08:16.246604+0000","last_peered":"2024-05-04T03:08:16.246604+0000","last_clean":"2024-05-04T03:08:16.246604+0000","last_became_active":"2024-05-04T03:07:37.070743+0000","last_became_peered":"2024-05-04T03:07:37.070743+0000","last_unstale":"2024-05-04T03:08:16.246604+0000","last_undegraded":"2024-05-04T03:08:16.246604+0000","last_fullsized":"2024-05-04T03:08:16.246604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:10:00.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248817+0000","last_change":"2024-05-04T03:07:34.157432+0000","last_active":"2024-05-04T03:08:16.248817+0000","last_peered":"2024-05-04T03:08:16.248817+0000","last_clean":"2024-05-04T03:08:16.248817+0000","last_became_active":"2024-05-04T03:07:34.156976+0000","last_became_peered":"2024-05-04T03:07:34.156976+0000","last_unstale":"2024-05-04T03:08:16.248817+0000","last_undegraded":"2024-05-04T03:08:16.248817+0000","last_fullsized":"2024-05-04T03:08:16.248817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:56.082287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242517+0000","last_change":"2024-05-04T03:07:37.064301+0000","last_active":"2024-05-04T03:08:16.242517+0000","last_peered":"2024-05-04T03:08:16.242517+0000","last_clean":"2024-05-04T03:08:16.242517+0000","last_became_active":"2024-05-04T03:07:37.063691+0000","last_became_peered":"2024-05-04T03:07:37.063691+0000","last_unstale":"2024-05-04T03:08:16.242517+0000","last_undegraded":"2024-05-04T03:08:16.242517+0000","last_fullsized":"2024-05-04T03:08:16.242517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:22:13.844703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'27","reported_seq":106,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:20.217298+0000","last_change":"2024-05-04T03:07:34.021595+0000","last_active":"2024-05-04T03:08:20.217298+0000","last_peered":"2024-05-04T03:08:20.217298+0000","last_clean":"2024-05-04T03:08:20.217298+0000","last_became_active":"2024-05-04T03:07:34.021126+0000","last_became_peered":"2024-05-04T03:07:34.021126+0000","last_unstale":"2024-05-04T03:08:20.217298+0000","last_undegraded":"2024-05-04T03:08:20.217298+0000","last_fullsized":"2024-05-04T03:08:20.217298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:55.094800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50695,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":75,"num_write":29,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245497+0000","last_change":"2024-05-04T03:07:37.066365+0000","last_active":"2024-05-04T03:08:16.245497+0000","last_peered":"2024-05-04T03:08:16.245497+0000","last_clean":"2024-05-04T03:08:16.245497+0000","last_became_active":"2024-05-04T03:07:37.065851+0000","last_became_peered":"2024-05-04T03:07:37.065851+0000","last_unstale":"2024-05-04T03:08:16.245497+0000","last_undegraded":"2024-05-04T03:08:16.245497+0000","last_fullsized":"2024-05-04T03:08:16.245497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:03:43.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.263114+0000","last_change":"2024-05-04T03:08:16.262600+0000","last_active":"2024-05-04T03:08:16.263114+0000","last_peered":"2024-05-04T03:08:16.263114+0000","last_clean":"2024-05-04T03:08:16.263114+0000","last_became_active":"2024-05-04T03:08:16.250619+0000","last_became_peered":"2024-05-04T03:08:16.250619+0000","last_unstale":"2024-05-04T03:08:16.263114+0000","last_undegraded":"2024-05-04T03:08:16.263114+0000","last_fullsized":"2024-05-04T03:08:16.263114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:51:21.596467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248552+0000","last_change":"2024-05-04T03:07:37.055356+0000","last_active":"2024-05-04T03:08:16.248552+0000","last_peered":"2024-05-04T03:08:16.248552+0000","last_clean":"2024-05-04T03:08:16.248552+0000","last_became_active":"2024-05-04T03:07:37.055045+0000","last_became_peered":"2024-05-04T03:07:37.055045+0000","last_unstale":"2024-05-04T03:08:16.248552+0000","last_undegraded":"2024-05-04T03:08:16.248552+0000","last_fullsized":"2024-05-04T03:08:16.248552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:06.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247382+0000","last_change":"2024-05-04T03:07:34.021525+0000","last_active":"2024-05-04T03:08:16.247382+0000","last_peered":"2024-05-04T03:08:16.247382+0000","last_clean":"2024-05-04T03:08:16.247382+0000","last_became_active":"2024-05-04T03:07:34.021042+0000","last_became_peered":"2024-05-04T03:07:34.021042+0000","last_unstale":"2024-05-04T03:08:16.247382+0000","last_undegraded":"2024-05-04T03:08:16.247382+0000","last_fullsized":"2024-05-04T03:08:16.247382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:44.090176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244794+0000","last_change":"2024-05-04T03:07:37.066471+0000","last_active":"2024-05-04T03:08:16.244794+0000","last_peered":"2024-05-04T03:08:16.244794+0000","last_clean":"2024-05-04T03:08:16.244794+0000","last_became_active":"2024-05-04T03:07:37.066059+0000","last_became_peered":"2024-05-04T03:07:37.066059+0000","last_unstale":"2024-05-04T03:08:16.244794+0000","last_undegraded":"2024-05-04T03:08:16.244794+0000","last_fullsized":"2024-05-04T03:08:16.244794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:16.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":59,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:21.217046+0000","last_change":"2024-05-04T03:07:34.021531+0000","last_active":"2024-05-04T03:08:21.217046+0000","last_peered":"2024-05-04T03:08:21.217046+0000","last_clean":"2024-05-04T03:08:21.217046+0000","last_became_active":"2024-05-04T03:07:34.021112+0000","last_became_peered":"2024-05-04T03:07:34.021112+0000","last_unstale":"2024-05-04T03:08:21.217046+0000","last_undegraded":"2024-05-04T03:08:21.217046+0000","last_fullsized":"2024-05-04T03:08:21.217046+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:28.826426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244611+0000","last_change":"2024-05-04T03:07:37.067155+0000","last_active":"2024-05-04T03:08:16.244611+0000","last_peered":"2024-05-04T03:08:16.244611+0000","last_clean":"2024-05-04T03:08:16.244611+0000","last_became_active":"2024-05-04T03:07:37.066783+0000","last_became_peered":"2024-05-04T03:07:37.066783+0000","last_unstale":"2024-05-04T03:08:16.244611+0000","last_undegraded":"2024-05-04T03:08:16.244611+0000","last_fullsized":"2024-05-04T03:08:16.244611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:23.217930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245013+0000","last_change":"2024-05-04T03:07:34.158227+0000","last_active":"2024-05-04T03:08:16.245013+0000","last_peered":"2024-05-04T03:08:16.245013+0000","last_clean":"2024-05-04T03:08:16.245013+0000","last_became_active":"2024-05-04T03:07:34.156905+0000","last_became_peered":"2024-05-04T03:07:34.156905+0000","last_unstale":"2024-05-04T03:08:16.245013+0000","last_undegraded":"2024-05-04T03:08:16.245013+0000","last_fullsized":"2024-05-04T03:08:16.245013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:40:41.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247601+0000","last_change":"2024-05-04T03:07:37.075559+0000","last_active":"2024-05-04T03:08:16.247601+0000","last_peered":"2024-05-04T03:08:16.247601+0000","last_clean":"2024-05-04T03:08:16.247601+0000","last_became_active":"2024-05-04T03:07:37.075220+0000","last_became_peered":"2024-05-04T03:07:37.075220+0000","last_unstale":"2024-05-04T03:08:16.247601+0000","last_undegraded":"2024-05-04T03:08:16.247601+0000","last_fullsized":"2024-05-04T03:08:16.247601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:51:23.942514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246548+0000","last_change":"2024-05-04T03:07:34.020326+0000","last_active":"2024-05-04T03:08:16.246548+0000","last_peered":"2024-05-04T03:08:16.246548+0000","last_clean":"2024-05-04T03:08:16.246548+0000","last_became_active":"2024-05-04T03:07:34.019472+0000","last_became_peered":"2024-05-04T03:07:34.019472+0000","last_unstale":"2024-05-04T03:08:16.246548+0000","last_undegraded":"2024-05-04T03:08:16.246548+0000","last_fullsized":"2024-05-04T03:08:16.246548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:33:28.375901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247007+0000","last_change":"2024-05-04T03:07:37.066223+0000","last_active":"2024-05-04T03:08:16.247007+0000","last_peered":"2024-05-04T03:08:16.247007+0000","last_clean":"2024-05-04T03:08:16.247007+0000","last_became_active":"2024-05-04T03:07:37.065859+0000","last_became_peered":"2024-05-04T03:07:37.065859+0000","last_unstale":"2024-05-04T03:08:16.247007+0000","last_undegraded":"2024-05-04T03:08:16.247007+0000","last_fullsized":"2024-05-04T03:08:16.247007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:44.902986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245474+0000","last_change":"2024-05-04T03:07:34.158674+0000","last_active":"2024-05-04T03:08:16.245474+0000","last_peered":"2024-05-04T03:08:16.245474+0000","last_clean":"2024-05-04T03:08:16.245474+0000","last_became_active":"2024-05-04T03:07:34.157952+0000","last_became_peered":"2024-05-04T03:07:34.157952+0000","last_unstale":"2024-05-04T03:08:16.245474+0000","last_undegraded":"2024-05-04T03:08:16.245474+0000","last_fullsized":"2024-05-04T03:08:16.245474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:19:11.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248252+0000","last_change":"2024-05-04T03:07:37.053897+0000","last_active":"2024-05-04T03:08:16.248252+0000","last_peered":"2024-05-04T03:08:16.248252+0000","last_clean":"2024-05-04T03:08:16.248252+0000","last_became_active":"2024-05-04T03:07:37.053466+0000","last_became_peered":"2024-05-04T03:07:37.053466+0000","last_unstale":"2024-05-04T03:08:16.248252+0000","last_undegraded":"2024-05-04T03:08:16.248252+0000","last_fullsized":"2024-05-04T03:08:16.248252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:50.650805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":57,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:21.217671+0000","last_change":"2024-05-04T03:07:34.021494+0000","last_active":"2024-05-04T03:08:21.217671+0000","last_peered":"2024-05-04T03:08:21.217671+0000","last_clean":"2024-05-04T03:08:21.217671+0000","last_became_active":"2024-05-04T03:07:34.021113+0000","last_became_peered":"2024-05-04T03:07:34.021113+0000","last_unstale":"2024-05-04T03:08:21.217671+0000","last_undegraded":"2024-05-04T03:08:21.217671+0000","last_fullsized":"2024-05-04T03:08:21.217671+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:02:09.514961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248164+0000","last_change":"2024-05-04T03:07:37.068199+0000","last_active":"2024-05-04T03:08:16.248164+0000","last_peered":"2024-05-04T03:08:16.248164+0000","last_clean":"2024-05-04T03:08:16.248164+0000","last_became_active":"2024-05-04T03:07:37.067828+0000","last_became_peered":"2024-05-04T03:07:37.067828+0000","last_unstale":"2024-05-04T03:08:16.248164+0000","last_undegraded":"2024-05-04T03:08:16.248164+0000","last_fullsized":"2024-05-04T03:08:16.248164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:28.882920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248113+0000","last_change":"2024-05-04T03:07:37.069753+0000","last_active":"2024-05-04T03:08:16.248113+0000","last_peered":"2024-05-04T03:08:16.248113+0000","last_clean":"2024-05-04T03:08:16.248113+0000","last_became_active":"2024-05-04T03:07:37.069338+0000","last_became_peered":"2024-05-04T03:07:37.069338+0000","last_unstale":"2024-05-04T03:08:16.248113+0000","last_undegraded":"2024-05-04T03:08:16.248113+0000","last_fullsized":"2024-05-04T03:08:16.248113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:37:11.355749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":56,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:20.215206+0000","last_change":"2024-05-04T03:07:34.157229+0000","last_active":"2024-05-04T03:08:20.215206+0000","last_peered":"2024-05-04T03:08:20.215206+0000","last_clean":"2024-05-04T03:08:20.215206+0000","last_became_active":"2024-05-04T03:07:34.156776+0000","last_became_peered":"2024-05-04T03:07:34.156776+0000","last_unstale":"2024-05-04T03:08:20.215206+0000","last_undegraded":"2024-05-04T03:08:20.215206+0000","last_fullsized":"2024-05-04T03:08:20.215206+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:26.879754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243715+0000","last_change":"2024-05-04T03:07:37.067025+0000","last_active":"2024-05-04T03:08:16.243715+0000","last_peered":"2024-05-04T03:08:16.243715+0000","last_clean":"2024-05-04T03:08:16.243715+0000","last_became_active":"2024-05-04T03:07:37.066359+0000","last_became_peered":"2024-05-04T03:07:37.066359+0000","last_unstale":"2024-05-04T03:08:16.243715+0000","last_undegraded":"2024-05-04T03:08:16.243715+0000","last_fullsized":"2024-05-04T03:08:16.243715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:44:35.604859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367108+0000","last_change":"2024-05-04T03:07:34.018019+0000","last_active":"2024-05-04T03:08:16.367108+0000","last_peered":"2024-05-04T03:08:16.367108+0000","last_clean":"2024-05-04T03:08:16.367108+0000","last_became_active":"2024-05-04T03:07:34.017556+0000","last_became_peered":"2024-05-04T03:07:34.017556+0000","last_unstale":"2024-05-04T03:08:16.367108+0000","last_undegraded":"2024-05-04T03:08:16.367108+0000","last_fullsized":"2024-05-04T03:08:16.367108+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:31:34.158164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244461+0000","last_change":"2024-05-04T03:06:06.704755+0000","last_active":"2024-05-04T03:08:16.244461+0000","last_peered":"2024-05-04T03:08:16.244461+0000","last_clean":"2024-05-04T03:08:16.244461+0000","last_became_active":"2024-05-04T03:06:06.698747+0000","last_became_peered":"2024-05-04T03:06:06.698747+0000","last_unstale":"2024-05-04T03:08:16.244461+0000","last_undegraded":"2024-05-04T03:08:16.244461+0000","last_fullsized":"2024-05-04T03:08:16.244461+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_clean_scrub_stamp":"2024-05-04T03:02:43.385377+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:04:50.012414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248266+0000","last_change":"2024-05-04T03:07:37.057043+0000","last_active":"2024-05-04T03:08:16.248266+0000","last_peered":"2024-05-04T03:08:16.248266+0000","last_clean":"2024-05-04T03:08:16.248266+0000","last_became_active":"2024-05-04T03:07:37.056725+0000","last_became_peered":"2024-05-04T03:07:37.056725+0000","last_unstale":"2024-05-04T03:08:16.248266+0000","last_undegraded":"2024-05-04T03:08:16.248266+0000","last_fullsized":"2024-05-04T03:08:16.248266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:13:28.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.376510+0000","last_change":"2024-05-04T03:08:16.376510+0000","last_active":"2024-05-04T03:08:16.376510+0000","last_peered":"2024-05-04T03:08:16.376510+0000","last_clean":"2024-05-04T03:08:16.376510+0000","last_became_active":"2024-05-04T03:08:16.368625+0000","last_became_peered":"2024-05-04T03:08:16.368625+0000","last_unstale":"2024-05-04T03:08:16.376510+0000","last_undegraded":"2024-05-04T03:08:16.376510+0000","last_fullsized":"2024-05-04T03:08:16.376510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:27:50.504487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245592+0000","last_change":"2024-05-04T03:07:34.157346+0000","last_active":"2024-05-04T03:08:16.245592+0000","last_peered":"2024-05-04T03:08:16.245592+0000","last_clean":"2024-05-04T03:08:16.245592+0000","last_became_active":"2024-05-04T03:07:34.156914+0000","last_became_peered":"2024-05-04T03:07:34.156914+0000","last_unstale":"2024-05-04T03:08:16.245592+0000","last_undegraded":"2024-05-04T03:08:16.245592+0000","last_fullsized":"2024-05-04T03:08:16.245592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:54:17.059644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248699+0000","last_change":"2024-05-04T03:07:37.054373+0000","last_active":"2024-05-04T03:08:16.248699+0000","last_peered":"2024-05-04T03:08:16.248699+0000","last_clean":"2024-05-04T03:08:16.248699+0000","last_became_active":"2024-05-04T03:07:37.053764+0000","last_became_peered":"2024-05-04T03:07:37.053764+0000","last_unstale":"2024-05-04T03:08:16.248699+0000","last_undegraded":"2024-05-04T03:08:16.248699+0000","last_fullsized":"2024-05-04T03:08:16.248699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:13:47.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:18.078893+0000","last_change":"2024-05-04T03:07:34.020190+0000","last_active":"2024-05-04T03:08:18.078893+0000","last_peered":"2024-05-04T03:08:18.078893+0000","last_clean":"2024-05-04T03:08:18.078893+0000","last_became_active":"2024-05-04T03:07:34.019786+0000","last_became_peered":"2024-05-04T03:07:34.019786+0000","last_unstale":"2024-05-04T03:08:18.078893+0000","last_undegraded":"2024-05-04T03:08:18.078893+0000","last_fullsized":"2024-05-04T03:08:18.078893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:11.243414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244079+0000","last_change":"2024-05-04T03:07:37.055274+0000","last_active":"2024-05-04T03:08:16.244079+0000","last_peered":"2024-05-04T03:08:16.244079+0000","last_clean":"2024-05-04T03:08:16.244079+0000","last_became_active":"2024-05-04T03:07:37.054968+0000","last_became_peered":"2024-05-04T03:07:37.054968+0000","last_unstale":"2024-05-04T03:08:16.244079+0000","last_undegraded":"2024-05-04T03:08:16.244079+0000","last_fullsized":"2024-05-04T03:08:16.244079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.229266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367455+0000","last_change":"2024-05-04T03:07:34.132966+0000","last_active":"2024-05-04T03:08:16.367455+0000","last_peered":"2024-05-04T03:08:16.367455+0000","last_clean":"2024-05-04T03:08:16.367455+0000","last_became_active":"2024-05-04T03:07:34.132529+0000","last_became_peered":"2024-05-04T03:07:34.132529+0000","last_unstale":"2024-05-04T03:08:16.367455+0000","last_undegraded":"2024-05-04T03:08:16.367455+0000","last_fullsized":"2024-05-04T03:08:16.367455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:31:37.122097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.368747+0000","last_change":"2024-05-04T03:08:16.368747+0000","last_active":"2024-05-04T03:08:16.368747+0000","last_peered":"2024-05-04T03:08:16.368747+0000","last_clean":"2024-05-04T03:08:16.368747+0000","last_became_active":"2024-05-04T03:08:16.368376+0000","last_became_peered":"2024-05-04T03:08:16.368376+0000","last_unstale":"2024-05-04T03:08:16.368747+0000","last_undegraded":"2024-05-04T03:08:16.368747+0000","last_fullsized":"2024-05-04T03:08:16.368747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:42:32.085650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242445+0000","last_change":"2024-05-04T03:07:34.157219+0000","last_active":"2024-05-04T03:08:16.242445+0000","last_peered":"2024-05-04T03:08:16.242445+0000","last_clean":"2024-05-04T03:08:16.242445+0000","last_became_active":"2024-05-04T03:07:34.156866+0000","last_became_peered":"2024-05-04T03:07:34.156866+0000","last_unstale":"2024-05-04T03:08:16.242445+0000","last_undegraded":"2024-05-04T03:08:16.242445+0000","last_fullsized":"2024-05-04T03:08:16.242445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:15:34.138825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245287+0000","last_change":"2024-05-04T03:07:37.058733+0000","last_active":"2024-05-04T03:08:16.245287+0000","last_peered":"2024-05-04T03:08:16.245287+0000","last_clean":"2024-05-04T03:08:16.245287+0000","last_became_active":"2024-05-04T03:07:37.058396+0000","last_became_peered":"2024-05-04T03:07:37.058396+0000","last_unstale":"2024-05-04T03:08:16.245287+0000","last_undegraded":"2024-05-04T03:08:16.245287+0000","last_fullsized":"2024-05-04T03:08:16.245287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:35.563728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243514+0000","last_change":"2024-05-04T03:07:34.016458+0000","last_active":"2024-05-04T03:08:16.243514+0000","last_peered":"2024-05-04T03:08:16.243514+0000","last_clean":"2024-05-04T03:08:16.243514+0000","last_became_active":"2024-05-04T03:07:34.016038+0000","last_became_peered":"2024-05-04T03:07:34.016038+0000","last_unstale":"2024-05-04T03:08:16.243514+0000","last_undegraded":"2024-05-04T03:08:16.243514+0000","last_fullsized":"2024-05-04T03:08:16.243514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:39:25.156047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247376+0000","last_change":"2024-05-04T03:07:37.064024+0000","last_active":"2024-05-04T03:08:16.247376+0000","last_peered":"2024-05-04T03:08:16.247376+0000","last_clean":"2024-05-04T03:08:16.247376+0000","last_became_active":"2024-05-04T03:07:37.063642+0000","last_became_peered":"2024-05-04T03:07:37.063642+0000","last_unstale":"2024-05-04T03:08:16.247376+0000","last_undegraded":"2024-05-04T03:08:16.247376+0000","last_fullsized":"2024-05-04T03:08:16.247376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:59:58.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.267243+0000","last_change":"2024-05-04T03:08:16.266763+0000","last_active":"2024-05-04T03:08:16.267243+0000","last_peered":"2024-05-04T03:08:16.267243+0000","last_clean":"2024-05-04T03:08:16.267243+0000","last_became_active":"2024-05-04T03:08:16.251212+0000","last_became_peered":"2024-05-04T03:08:16.251212+0000","last_unstale":"2024-05-04T03:08:16.267243+0000","last_undegraded":"2024-05-04T03:08:16.267243+0000","last_fullsized":"2024-05-04T03:08:16.267243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:31:27.444610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243667+0000","last_change":"2024-05-04T03:07:37.063753+0000","last_active":"2024-05-04T03:08:16.243667+0000","last_peered":"2024-05-04T03:08:16.243667+0000","last_clean":"2024-05-04T03:08:16.243667+0000","last_became_active":"2024-05-04T03:07:37.063318+0000","last_became_peered":"2024-05-04T03:07:37.063318+0000","last_unstale":"2024-05-04T03:08:16.243667+0000","last_undegraded":"2024-05-04T03:08:16.243667+0000","last_fullsized":"2024-05-04T03:08:16.243667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:48.217136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245115+0000","last_change":"2024-05-04T03:07:34.022522+0000","last_active":"2024-05-04T03:08:16.245115+0000","last_peered":"2024-05-04T03:08:16.245115+0000","last_clean":"2024-05-04T03:08:16.245115+0000","last_became_active":"2024-05-04T03:07:34.021018+0000","last_became_peered":"2024-05-04T03:07:34.021018+0000","last_unstale":"2024-05-04T03:08:16.245115+0000","last_undegraded":"2024-05-04T03:08:16.245115+0000","last_fullsized":"2024-05-04T03:08:16.245115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:47:04.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244631+0000","last_change":"2024-05-04T03:07:37.069930+0000","last_active":"2024-05-04T03:08:16.244631+0000","last_peered":"2024-05-04T03:08:16.244631+0000","last_clean":"2024-05-04T03:08:16.244631+0000","last_became_active":"2024-05-04T03:07:37.069580+0000","last_became_peered":"2024-05-04T03:07:37.069580+0000","last_unstale":"2024-05-04T03:08:16.244631+0000","last_undegraded":"2024-05-04T03:08:16.244631+0000","last_fullsized":"2024-05-04T03:08:16.244631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:15:33.815970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":57,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:21.217722+0000","last_change":"2024-05-04T03:07:34.157969+0000","last_active":"2024-05-04T03:08:21.217722+0000","last_peered":"2024-05-04T03:08:21.217722+0000","last_clean":"2024-05-04T03:08:21.217722+0000","last_became_active":"2024-05-04T03:07:34.157563+0000","last_became_peered":"2024-05-04T03:07:34.157563+0000","last_unstale":"2024-05-04T03:08:21.217722+0000","last_undegraded":"2024-05-04T03:08:21.217722+0000","last_fullsized":"2024-05-04T03:08:21.217722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:00:43.562109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245064+0000","last_change":"2024-05-04T03:07:37.071279+0000","last_active":"2024-05-04T03:08:16.245064+0000","last_peered":"2024-05-04T03:08:16.245064+0000","last_clean":"2024-05-04T03:08:16.245064+0000","last_became_active":"2024-05-04T03:07:37.070449+0000","last_became_peered":"2024-05-04T03:07:37.070449+0000","last_unstale":"2024-05-04T03:08:16.245064+0000","last_undegraded":"2024-05-04T03:08:16.245064+0000","last_fullsized":"2024-05-04T03:08:16.245064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:13:22.535788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248538+0000","last_change":"2024-05-04T03:07:34.017010+0000","last_active":"2024-05-04T03:08:16.248538+0000","last_peered":"2024-05-04T03:08:16.248538+0000","last_clean":"2024-05-04T03:08:16.248538+0000","last_became_active":"2024-05-04T03:07:34.016521+0000","last_became_peered":"2024-05-04T03:07:34.016521+0000","last_unstale":"2024-05-04T03:08:16.248538+0000","last_undegraded":"2024-05-04T03:08:16.248538+0000","last_fullsized":"2024-05-04T03:08:16.248538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:36.684656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245167+0000","last_change":"2024-05-04T03:07:37.066526+0000","last_active":"2024-05-04T03:08:16.245167+0000","last_peered":"2024-05-04T03:08:16.245167+0000","last_clean":"2024-05-04T03:08:16.245167+0000","last_became_active":"2024-05-04T03:07:37.065312+0000","last_became_peered":"2024-05-04T03:07:37.065312+0000","last_unstale":"2024-05-04T03:08:16.245167+0000","last_undegraded":"2024-05-04T03:08:16.245167+0000","last_fullsized":"2024-05-04T03:08:16.245167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:10.077295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248927+0000","last_change":"2024-05-04T03:07:34.016898+0000","last_active":"2024-05-04T03:08:16.248927+0000","last_peered":"2024-05-04T03:08:16.248927+0000","last_clean":"2024-05-04T03:08:16.248927+0000","last_became_active":"2024-05-04T03:07:34.016524+0000","last_became_peered":"2024-05-04T03:07:34.016524+0000","last_unstale":"2024-05-04T03:08:16.248927+0000","last_undegraded":"2024-05-04T03:08:16.248927+0000","last_fullsized":"2024-05-04T03:08:16.248927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:01.236207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245122+0000","last_change":"2024-05-04T03:07:37.061501+0000","last_active":"2024-05-04T03:08:16.245122+0000","last_peered":"2024-05-04T03:08:16.245122+0000","last_clean":"2024-05-04T03:08:16.245122+0000","last_became_active":"2024-05-04T03:07:37.061077+0000","last_became_peered":"2024-05-04T03:07:37.061077+0000","last_unstale":"2024-05-04T03:08:16.245122+0000","last_undegraded":"2024-05-04T03:08:16.245122+0000","last_fullsized":"2024-05-04T03:08:16.245122+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:32.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247502+0000","last_change":"2024-05-04T03:07:37.064612+0000","last_active":"2024-05-04T03:08:16.247502+0000","last_peered":"2024-05-04T03:08:16.247502+0000","last_clean":"2024-05-04T03:08:16.247502+0000","last_became_active":"2024-05-04T03:07:37.064303+0000","last_became_peered":"2024-05-04T03:07:37.064303+0000","last_unstale":"2024-05-04T03:08:16.247502+0000","last_undegraded":"2024-05-04T03:08:16.247502+0000","last_fullsized":"2024-05-04T03:08:16.247502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:47:07.811765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250296+0000","last_change":"2024-05-04T03:07:37.061577+0000","last_active":"2024-05-04T03:08:16.250296+0000","last_peered":"2024-05-04T03:08:16.250296+0000","last_clean":"2024-05-04T03:08:16.250296+0000","last_became_active":"2024-05-04T03:07:37.061214+0000","last_became_peered":"2024-05-04T03:07:37.061214+0000","last_unstale":"2024-05-04T03:08:16.250296+0000","last_undegraded":"2024-05-04T03:08:16.250296+0000","last_fullsized":"2024-05-04T03:08:16.250296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:52:11.733257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.251332+0000","last_change":"2024-05-04T03:07:37.062906+0000","last_active":"2024-05-04T03:08:16.251332+0000","last_peered":"2024-05-04T03:08:16.251332+0000","last_clean":"2024-05-04T03:08:16.251332+0000","last_became_active":"2024-05-04T03:07:37.062510+0000","last_became_peered":"2024-05-04T03:07:37.062510+0000","last_unstale":"2024-05-04T03:08:16.251332+0000","last_undegraded":"2024-05-04T03:08:16.251332+0000","last_fullsized":"2024-05-04T03:08:16.251332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:57:58.910490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246507+0000","last_change":"2024-05-04T03:07:37.068367+0000","last_active":"2024-05-04T03:08:16.246507+0000","last_peered":"2024-05-04T03:08:16.246507+0000","last_clean":"2024-05-04T03:08:16.246507+0000","last_became_active":"2024-05-04T03:07:37.068038+0000","last_became_peered":"2024-05-04T03:07:37.068038+0000","last_unstale":"2024-05-04T03:08:16.246507+0000","last_undegraded":"2024-05-04T03:08:16.246507+0000","last_fullsized":"2024-05-04T03:08:16.246507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:11:19.195002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250759+0000","last_change":"2024-05-04T03:08:16.250759+0000","last_active":"2024-05-04T03:08:16.250759+0000","last_peered":"2024-05-04T03:08:16.250759+0000","last_clean":"2024-05-04T03:08:16.250759+0000","last_became_active":"2024-05-04T03:08:16.250379+0000","last_became_peered":"2024-05-04T03:08:16.250379+0000","last_unstale":"2024-05-04T03:08:16.250759+0000","last_undegraded":"2024-05-04T03:08:16.250759+0000","last_fullsized":"2024-05-04T03:08:16.250759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:58:09.419009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246960+0000","last_change":"2024-05-04T03:07:37.064584+0000","last_active":"2024-05-04T03:08:16.246960+0000","last_peered":"2024-05-04T03:08:16.246960+0000","last_clean":"2024-05-04T03:08:16.246960+0000","last_became_active":"2024-05-04T03:07:37.064183+0000","last_became_peered":"2024-05-04T03:07:37.064183+0000","last_unstale":"2024-05-04T03:08:16.246960+0000","last_undegraded":"2024-05-04T03:08:16.246960+0000","last_fullsized":"2024-05-04T03:08:16.246960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:17.121081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250902+0000","last_change":"2024-05-04T03:07:37.066742+0000","last_active":"2024-05-04T03:08:16.250902+0000","last_peered":"2024-05-04T03:08:16.250902+0000","last_clean":"2024-05-04T03:08:16.250902+0000","last_became_active":"2024-05-04T03:07:37.066126+0000","last_became_peered":"2024-05-04T03:07:37.066126+0000","last_unstale":"2024-05-04T03:08:16.250902+0000","last_undegraded":"2024-05-04T03:08:16.250902+0000","last_fullsized":"2024-05-04T03:08:16.250902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:57.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244050+0000","last_change":"2024-05-04T03:07:37.068122+0000","last_active":"2024-05-04T03:08:16.244050+0000","last_peered":"2024-05-04T03:08:16.244050+0000","last_clean":"2024-05-04T03:08:16.244050+0000","last_became_active":"2024-05-04T03:07:37.066321+0000","last_became_peered":"2024-05-04T03:07:37.066321+0000","last_unstale":"2024-05-04T03:08:16.244050+0000","last_undegraded":"2024-05-04T03:08:16.244050+0000","last_fullsized":"2024-05-04T03:08:16.244050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:25:09.321853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247940+0000","last_change":"2024-05-04T03:07:37.060322+0000","last_active":"2024-05-04T03:08:16.247940+0000","last_peered":"2024-05-04T03:08:16.247940+0000","last_clean":"2024-05-04T03:08:16.247940+0000","last_became_active":"2024-05-04T03:07:37.059944+0000","last_became_peered":"2024-05-04T03:07:37.059944+0000","last_unstale":"2024-05-04T03:08:16.247940+0000","last_undegraded":"2024-05-04T03:08:16.247940+0000","last_fullsized":"2024-05-04T03:08:16.247940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.105085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243469+0000","last_change":"2024-05-04T03:07:37.056269+0000","last_active":"2024-05-04T03:08:16.243469+0000","last_peered":"2024-05-04T03:08:16.243469+0000","last_clean":"2024-05-04T03:08:16.243469+0000","last_became_active":"2024-05-04T03:07:37.055886+0000","last_became_peered":"2024-05-04T03:07:37.055886+0000","last_unstale":"2024-05-04T03:08:16.243469+0000","last_undegraded":"2024-05-04T03:08:16.243469+0000","last_fullsized":"2024-05-04T03:08:16.243469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:55:11.292577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242733+0000","last_change":"2024-05-04T03:07:37.071013+0000","last_active":"2024-05-04T03:08:16.242733+0000","last_peered":"2024-05-04T03:08:16.242733+0000","last_clean":"2024-05-04T03:08:16.242733+0000","last_became_active":"2024-05-04T03:07:37.069346+0000","last_became_peered":"2024-05-04T03:07:37.069346+0000","last_unstale":"2024-05-04T03:08:16.242733+0000","last_undegraded":"2024-05-04T03:08:16.242733+0000","last_fullsized":"2024-05-04T03:08:16.242733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:19:11.898766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.249009+0000","last_change":"2024-05-04T03:07:37.064962+0000","last_active":"2024-05-04T03:08:16.249009+0000","last_peered":"2024-05-04T03:08:16.249009+0000","last_clean":"2024-05-04T03:08:16.249009+0000","last_became_active":"2024-05-04T03:07:37.064637+0000","last_became_peered":"2024-05-04T03:07:37.064637+0000","last_unstale":"2024-05-04T03:08:16.249009+0000","last_undegraded":"2024-05-04T03:08:16.249009+0000","last_fullsized":"2024-05-04T03:08:16.249009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:10:10.659621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246565+0000","last_change":"2024-05-04T03:07:37.056721+0000","last_active":"2024-05-04T03:08:16.246565+0000","last_peered":"2024-05-04T03:08:16.246565+0000","last_clean":"2024-05-04T03:08:16.246565+0000","last_became_active":"2024-05-04T03:07:37.056353+0000","last_became_peered":"2024-05-04T03:07:37.056353+0000","last_unstale":"2024-05-04T03:08:16.246565+0000","last_undegraded":"2024-05-04T03:08:16.246565+0000","last_fullsized":"2024-05-04T03:08:16.246565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:30:47.476579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286351,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57508,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":278,"num_read_kb":202,"num_write":238,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710745,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":880640,"data_stored":425832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1067879,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":425694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":426072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1018590,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":425411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":482414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":426750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":482566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":425236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":425663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":1017877,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":52822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:08:27.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:08:27.380 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:27.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:27 smithi039 ceph-mon[29715]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:08:27.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:27 smithi066 ceph-mon[33264]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:08:27.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:27 smithi167 ceph-mon[33614]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:08:28.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:28 smithi039 ceph-mon[29715]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s; 6 B/s, 0 keys/s, 1 objects/s recovering 2024-05-04T03:08:28.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:28 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:28.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:28 smithi066 ceph-mon[33264]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s; 6 B/s, 0 keys/s, 1 objects/s recovering 2024-05-04T03:08:28.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:28 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:28.652 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:28 smithi167 ceph-mon[33614]: pgmap v316: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s; 6 B/s, 0 keys/s, 1 objects/s recovering 2024-05-04T03:08:28.652 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:28 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1961160488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc109300 msgr2=0x7fe0dc109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1961160488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc109300 0x7fe0dc109710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009990 tx=0x7fe0cc009ca0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1961160488 shutdown_connections 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1961160488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc109300 0x7fe0dc109710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1961160488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0dc1030e0 0x7fe0dc111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1961160488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0dc102790 0x7fe0dc102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1961160488 >> 172.21.15.39:0/1961160488 conn(0x7fe0dc0fe520 msgr2=0x7fe0dc100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1961160488 shutdown_connections 2024-05-04T03:08:29.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.123+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1961160488 wait complete. 2024-05-04T03:08:29.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.124+0000 7fe0e1f51700 1 Processor -- start 2024-05-04T03:08:29.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.124+0000 7fe0e1f51700 1 -- start start 2024-05-04T03:08:29.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0e1f51700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc102790 0x7fe0dc19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:29.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0e1f51700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0dc1030e0 0x7fe0dc19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:29.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0e1f51700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0dc109300 0x7fe0dc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:29.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0e1f51700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc114440 con 0x7fe0dc102790 2024-05-04T03:08:29.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0e1f51700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1142c0 con 0x7fe0dc109300 2024-05-04T03:08:29.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0e1f51700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1145c0 con 0x7fe0dc1030e0 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc102790 0x7fe0dc19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc102790 0x7fe0dc19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58242/0 (socket says 172.21.15.39:58242) 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 -- 172.21.15.39:0/1933160393 learned_addr learned my addr 172.21.15.39:0/1933160393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0daffd700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0dc1030e0 0x7fe0dc19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0dbfff700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0dc109300 0x7fe0dc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 -- 172.21.15.39:0/1933160393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0dc1030e0 msgr2=0x7fe0dc19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0dc1030e0 0x7fe0dc19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 -- 172.21.15.39:0/1933160393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0dc109300 msgr2=0x7fe0dc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0dc109300 0x7fe0dc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 -- 172.21.15.39:0/1933160393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc19fd20 con 0x7fe0dc102790 2024-05-04T03:08:29.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0dbfff700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0dc109300 0x7fe0dc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:08:29.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.125+0000 7fe0db7fe700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc102790 0x7fe0dc19ae50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc006010 tx=0x7fe0cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:29.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.126+0000 7fe0d8ff9700 1 -- 172.21.15.39:0/1933160393 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc027070 con 0x7fe0dc102790 2024-05-04T03:08:29.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.126+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc19ffb0 con 0x7fe0dc102790 2024-05-04T03:08:29.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.126+0000 7fe0d8ff9700 1 -- 172.21.15.39:0/1933160393 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0cc00fbc0 con 0x7fe0dc102790 2024-05-04T03:08:29.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.126+0000 7fe0d8ff9700 1 -- 172.21.15.39:0/1933160393 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc022b30 con 0x7fe0dc102790 2024-05-04T03:08:29.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.126+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1a7930 con 0x7fe0dc102790 2024-05-04T03:08:29.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.127+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc10c8a0 con 0x7fe0dc102790 2024-05-04T03:08:29.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.128+0000 7fe0d8ff9700 1 -- 172.21.15.39:0/1933160393 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe0cc02a070 con 0x7fe0dc102790 2024-05-04T03:08:29.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.129+0000 7fe0d8ff9700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0c40b2910 0x7fe0c40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:29.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.129+0000 7fe0d8ff9700 1 -- 172.21.15.39:0/1933160393 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe0cc0e3c40 con 0x7fe0dc102790 2024-05-04T03:08:29.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.129+0000 7fe0daffd700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0c40b2910 0x7fe0c40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:29.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.130+0000 7fe0daffd700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0c40b2910 0x7fe0c40b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0feac0 tx=0x7fe0d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:29.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.134+0000 7fe0d8ff9700 1 -- 172.21.15.39:0/1933160393 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe0cc013070 con 0x7fe0dc102790 2024-05-04T03:08:29.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.357+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0dc0512c0 con 0x7fe0c40b2910 2024-05-04T03:08:29.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.365+0000 7fe0d8ff9700 1 -- 172.21.15.39:0/1933160393 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258917 (secure 0 0 0) 0x7fe0dc0512c0 con 0x7fe0c40b2910 2024-05-04T03:08:29.369 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0c40b2910 msgr2=0x7fe0c40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0c40b2910 0x7fe0c40b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0feac0 tx=0x7fe0d000b410 comp rx=0 tx=0).stop 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc102790 msgr2=0x7fe0dc19ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc102790 0x7fe0dc19ae50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc006010 tx=0x7fe0cc0047b0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 shutdown_connections 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0c40b2910 0x7fe0c40b4e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0dc102790 0x7fe0dc19ae50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0dc109300 0x7fe0dc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 --2- 172.21.15.39:0/1933160393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0dc1030e0 0x7fe0dc19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 >> 172.21.15.39:0/1933160393 conn(0x7fe0dc0fe520 msgr2=0x7fe0dc107320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.368+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 shutdown_connections 2024-05-04T03:08:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:29.369+0000 7fe0e1f51700 1 -- 172.21.15.39:0/1933160393 wait complete. 2024-05-04T03:08:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-04T03:08:30.382 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-05-04T03:08:29.005329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653505,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":405,"num_read_kb":297,"num_write":368,"num_write_kb":2282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376628,"kb_used_data":10796,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565196,"statfs":{"total":1151940427776,"available":1151554760704,"internally_reserved":0,"allocated":11055104,"data_stored":7063893,"data_compressed":109203,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14006,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":127,"num_read_kb":75,"num_write":18,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.018616"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247282+0000","last_change":"2024-05-04T03:07:37.070347+0000","last_active":"2024-05-04T03:08:16.247282+0000","last_peered":"2024-05-04T03:08:16.247282+0000","last_clean":"2024-05-04T03:08:16.247282+0000","last_became_active":"2024-05-04T03:07:37.069997+0000","last_became_peered":"2024-05-04T03:07:37.069997+0000","last_unstale":"2024-05-04T03:08:16.247282+0000","last_undegraded":"2024-05-04T03:08:16.247282+0000","last_fullsized":"2024-05-04T03:08:16.247282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:24:11.362228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247260+0000","last_change":"2024-05-04T03:07:37.071598+0000","last_active":"2024-05-04T03:08:16.247260+0000","last_peered":"2024-05-04T03:08:16.247260+0000","last_clean":"2024-05-04T03:08:16.247260+0000","last_became_active":"2024-05-04T03:07:37.071059+0000","last_became_peered":"2024-05-04T03:07:37.071059+0000","last_unstale":"2024-05-04T03:08:16.247260+0000","last_undegraded":"2024-05-04T03:08:16.247260+0000","last_fullsized":"2024-05-04T03:08:16.247260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:33:33.929246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246145+0000","last_change":"2024-05-04T03:07:37.069951+0000","last_active":"2024-05-04T03:08:16.246145+0000","last_peered":"2024-05-04T03:08:16.246145+0000","last_clean":"2024-05-04T03:08:16.246145+0000","last_became_active":"2024-05-04T03:07:37.069558+0000","last_became_peered":"2024-05-04T03:07:37.069558+0000","last_unstale":"2024-05-04T03:08:16.246145+0000","last_undegraded":"2024-05-04T03:08:16.246145+0000","last_fullsized":"2024-05-04T03:08:16.246145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:37:52.596455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367260+0000","last_change":"2024-05-04T03:07:37.068820+0000","last_active":"2024-05-04T03:08:16.367260+0000","last_peered":"2024-05-04T03:08:16.367260+0000","last_clean":"2024-05-04T03:08:16.367260+0000","last_became_active":"2024-05-04T03:07:37.068323+0000","last_became_peered":"2024-05-04T03:07:37.068323+0000","last_unstale":"2024-05-04T03:08:16.367260+0000","last_undegraded":"2024-05-04T03:08:16.367260+0000","last_fullsized":"2024-05-04T03:08:16.367260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:52:54.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245207+0000","last_change":"2024-05-04T03:07:37.061982+0000","last_active":"2024-05-04T03:08:16.245207+0000","last_peered":"2024-05-04T03:08:16.245207+0000","last_clean":"2024-05-04T03:08:16.245207+0000","last_became_active":"2024-05-04T03:07:37.061621+0000","last_became_peered":"2024-05-04T03:07:37.061621+0000","last_unstale":"2024-05-04T03:08:16.245207+0000","last_undegraded":"2024-05-04T03:08:16.245207+0000","last_fullsized":"2024-05-04T03:08:16.245207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:17:47.421011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248166+0000","last_change":"2024-05-04T03:07:37.058641+0000","last_active":"2024-05-04T03:08:16.248166+0000","last_peered":"2024-05-04T03:08:16.248166+0000","last_clean":"2024-05-04T03:08:16.248166+0000","last_became_active":"2024-05-04T03:07:37.058235+0000","last_became_peered":"2024-05-04T03:07:37.058235+0000","last_unstale":"2024-05-04T03:08:16.248166+0000","last_undegraded":"2024-05-04T03:08:16.248166+0000","last_fullsized":"2024-05-04T03:08:16.248166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:17:45.441777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244522+0000","last_change":"2024-05-04T03:07:37.065959+0000","last_active":"2024-05-04T03:08:16.244522+0000","last_peered":"2024-05-04T03:08:16.244522+0000","last_clean":"2024-05-04T03:08:16.244522+0000","last_became_active":"2024-05-04T03:07:37.065508+0000","last_became_peered":"2024-05-04T03:07:37.065508+0000","last_unstale":"2024-05-04T03:08:16.244522+0000","last_undegraded":"2024-05-04T03:08:16.244522+0000","last_fullsized":"2024-05-04T03:08:16.244522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:17.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243478+0000","last_change":"2024-05-04T03:07:37.065620+0000","last_active":"2024-05-04T03:08:16.243478+0000","last_peered":"2024-05-04T03:08:16.243478+0000","last_clean":"2024-05-04T03:08:16.243478+0000","last_became_active":"2024-05-04T03:07:37.065261+0000","last_became_peered":"2024-05-04T03:07:37.065261+0000","last_unstale":"2024-05-04T03:08:16.243478+0000","last_undegraded":"2024-05-04T03:08:16.243478+0000","last_fullsized":"2024-05-04T03:08:16.243478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:27.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242930+0000","last_change":"2024-05-04T03:07:37.072204+0000","last_active":"2024-05-04T03:08:16.242930+0000","last_peered":"2024-05-04T03:08:16.242930+0000","last_clean":"2024-05-04T03:08:16.242930+0000","last_became_active":"2024-05-04T03:07:37.070622+0000","last_became_peered":"2024-05-04T03:07:37.070622+0000","last_unstale":"2024-05-04T03:08:16.242930+0000","last_undegraded":"2024-05-04T03:08:16.242930+0000","last_fullsized":"2024-05-04T03:08:16.242930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:55:25.931369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.253125+0000","last_change":"2024-05-04T03:08:16.253125+0000","last_active":"2024-05-04T03:08:16.253125+0000","last_peered":"2024-05-04T03:08:16.253125+0000","last_clean":"2024-05-04T03:08:16.253125+0000","last_became_active":"2024-05-04T03:08:16.252682+0000","last_became_peered":"2024-05-04T03:08:16.252682+0000","last_unstale":"2024-05-04T03:08:16.253125+0000","last_undegraded":"2024-05-04T03:08:16.253125+0000","last_fullsized":"2024-05-04T03:08:16.253125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:05.933491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244798+0000","last_change":"2024-05-04T03:07:37.071614+0000","last_active":"2024-05-04T03:08:16.244798+0000","last_peered":"2024-05-04T03:08:16.244798+0000","last_clean":"2024-05-04T03:08:16.244798+0000","last_became_active":"2024-05-04T03:07:37.070927+0000","last_became_peered":"2024-05-04T03:07:37.070927+0000","last_unstale":"2024-05-04T03:08:16.244798+0000","last_undegraded":"2024-05-04T03:08:16.244798+0000","last_fullsized":"2024-05-04T03:08:16.244798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:30.432959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367227+0000","last_change":"2024-05-04T03:07:37.069305+0000","last_active":"2024-05-04T03:08:16.367227+0000","last_peered":"2024-05-04T03:08:16.367227+0000","last_clean":"2024-05-04T03:08:16.367227+0000","last_became_active":"2024-05-04T03:07:37.068804+0000","last_became_peered":"2024-05-04T03:07:37.068804+0000","last_unstale":"2024-05-04T03:08:16.367227+0000","last_undegraded":"2024-05-04T03:08:16.367227+0000","last_fullsized":"2024-05-04T03:08:16.367227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:43.693161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248895+0000","last_change":"2024-05-04T03:07:37.062848+0000","last_active":"2024-05-04T03:08:16.248895+0000","last_peered":"2024-05-04T03:08:16.248895+0000","last_clean":"2024-05-04T03:08:16.248895+0000","last_became_active":"2024-05-04T03:07:37.062548+0000","last_became_peered":"2024-05-04T03:07:37.062548+0000","last_unstale":"2024-05-04T03:08:16.248895+0000","last_undegraded":"2024-05-04T03:08:16.248895+0000","last_fullsized":"2024-05-04T03:08:16.248895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246059+0000","last_change":"2024-05-04T03:07:37.062598+0000","last_active":"2024-05-04T03:08:16.246059+0000","last_peered":"2024-05-04T03:08:16.246059+0000","last_clean":"2024-05-04T03:08:16.246059+0000","last_became_active":"2024-05-04T03:07:37.061711+0000","last_became_peered":"2024-05-04T03:07:37.061711+0000","last_unstale":"2024-05-04T03:08:16.246059+0000","last_undegraded":"2024-05-04T03:08:16.246059+0000","last_fullsized":"2024-05-04T03:08:16.246059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:29:03.153680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246387+0000","last_change":"2024-05-04T03:07:37.067889+0000","last_active":"2024-05-04T03:08:16.246387+0000","last_peered":"2024-05-04T03:08:16.246387+0000","last_clean":"2024-05-04T03:08:16.246387+0000","last_became_active":"2024-05-04T03:07:37.066768+0000","last_became_peered":"2024-05-04T03:07:37.066768+0000","last_unstale":"2024-05-04T03:08:16.246387+0000","last_undegraded":"2024-05-04T03:08:16.246387+0000","last_fullsized":"2024-05-04T03:08:16.246387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:01:18.136379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242305+0000","last_change":"2024-05-04T03:07:37.070438+0000","last_active":"2024-05-04T03:08:16.242305+0000","last_peered":"2024-05-04T03:08:16.242305+0000","last_clean":"2024-05-04T03:08:16.242305+0000","last_became_active":"2024-05-04T03:07:37.070007+0000","last_became_peered":"2024-05-04T03:07:37.070007+0000","last_unstale":"2024-05-04T03:08:16.242305+0000","last_undegraded":"2024-05-04T03:08:16.242305+0000","last_fullsized":"2024-05-04T03:08:16.242305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:22.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247556+0000","last_change":"2024-05-04T03:07:37.070020+0000","last_active":"2024-05-04T03:08:16.247556+0000","last_peered":"2024-05-04T03:08:16.247556+0000","last_clean":"2024-05-04T03:08:16.247556+0000","last_became_active":"2024-05-04T03:07:37.069629+0000","last_became_peered":"2024-05-04T03:07:37.069629+0000","last_unstale":"2024-05-04T03:08:16.247556+0000","last_undegraded":"2024-05-04T03:08:16.247556+0000","last_fullsized":"2024-05-04T03:08:16.247556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:28.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245734+0000","last_change":"2024-05-04T03:07:37.068651+0000","last_active":"2024-05-04T03:08:16.245734+0000","last_peered":"2024-05-04T03:08:16.245734+0000","last_clean":"2024-05-04T03:08:16.245734+0000","last_became_active":"2024-05-04T03:07:37.067135+0000","last_became_peered":"2024-05-04T03:07:37.067135+0000","last_unstale":"2024-05-04T03:08:16.245734+0000","last_undegraded":"2024-05-04T03:08:16.245734+0000","last_fullsized":"2024-05-04T03:08:16.245734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:00:59.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250832+0000","last_change":"2024-05-04T03:07:37.055419+0000","last_active":"2024-05-04T03:08:16.250832+0000","last_peered":"2024-05-04T03:08:16.250832+0000","last_clean":"2024-05-04T03:08:16.250832+0000","last_became_active":"2024-05-04T03:07:37.055024+0000","last_became_peered":"2024-05-04T03:07:37.055024+0000","last_unstale":"2024-05-04T03:08:16.250832+0000","last_undegraded":"2024-05-04T03:08:16.250832+0000","last_fullsized":"2024-05-04T03:08:16.250832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:55:05.506505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":70,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:27.224689+0000","last_change":"2024-05-04T03:07:34.019975+0000","last_active":"2024-05-04T03:08:27.224689+0000","last_peered":"2024-05-04T03:08:27.224689+0000","last_clean":"2024-05-04T03:08:27.224689+0000","last_became_active":"2024-05-04T03:07:34.019532+0000","last_became_peered":"2024-05-04T03:07:34.019532+0000","last_unstale":"2024-05-04T03:08:27.224689+0000","last_undegraded":"2024-05-04T03:08:27.224689+0000","last_fullsized":"2024-05-04T03:08:27.224689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:11:03.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243957+0000","last_change":"2024-05-04T03:07:37.064725+0000","last_active":"2024-05-04T03:08:16.243957+0000","last_peered":"2024-05-04T03:08:16.243957+0000","last_clean":"2024-05-04T03:08:16.243957+0000","last_became_active":"2024-05-04T03:07:37.064385+0000","last_became_peered":"2024-05-04T03:07:37.064385+0000","last_unstale":"2024-05-04T03:08:16.243957+0000","last_undegraded":"2024-05-04T03:08:16.243957+0000","last_fullsized":"2024-05-04T03:08:16.243957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:22:12.024328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244081+0000","last_change":"2024-05-04T03:07:34.157179+0000","last_active":"2024-05-04T03:08:16.244081+0000","last_peered":"2024-05-04T03:08:16.244081+0000","last_clean":"2024-05-04T03:08:16.244081+0000","last_became_active":"2024-05-04T03:07:34.156790+0000","last_became_peered":"2024-05-04T03:07:34.156790+0000","last_unstale":"2024-05-04T03:08:16.244081+0000","last_undegraded":"2024-05-04T03:08:16.244081+0000","last_fullsized":"2024-05-04T03:08:16.244081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:32.066410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246454+0000","last_change":"2024-05-04T03:07:37.070481+0000","last_active":"2024-05-04T03:08:16.246454+0000","last_peered":"2024-05-04T03:08:16.246454+0000","last_clean":"2024-05-04T03:08:16.246454+0000","last_became_active":"2024-05-04T03:07:37.069903+0000","last_became_peered":"2024-05-04T03:07:37.069903+0000","last_unstale":"2024-05-04T03:08:16.246454+0000","last_undegraded":"2024-05-04T03:08:16.246454+0000","last_fullsized":"2024-05-04T03:08:16.246454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:45:40.764716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245723+0000","last_change":"2024-05-04T03:07:34.023978+0000","last_active":"2024-05-04T03:08:16.245723+0000","last_peered":"2024-05-04T03:08:16.245723+0000","last_clean":"2024-05-04T03:08:16.245723+0000","last_became_active":"2024-05-04T03:07:34.023199+0000","last_became_peered":"2024-05-04T03:07:34.023199+0000","last_unstale":"2024-05-04T03:08:16.245723+0000","last_undegraded":"2024-05-04T03:08:16.245723+0000","last_fullsized":"2024-05-04T03:08:16.245723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:17:43.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242450+0000","last_change":"2024-05-04T03:07:37.072661+0000","last_active":"2024-05-04T03:08:16.242450+0000","last_peered":"2024-05-04T03:08:16.242450+0000","last_clean":"2024-05-04T03:08:16.242450+0000","last_became_active":"2024-05-04T03:07:37.071393+0000","last_became_peered":"2024-05-04T03:07:37.071393+0000","last_unstale":"2024-05-04T03:08:16.242450+0000","last_undegraded":"2024-05-04T03:08:16.242450+0000","last_fullsized":"2024-05-04T03:08:16.242450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:39:39.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245493+0000","last_change":"2024-05-04T03:07:34.018543+0000","last_active":"2024-05-04T03:08:16.245493+0000","last_peered":"2024-05-04T03:08:16.245493+0000","last_clean":"2024-05-04T03:08:16.245493+0000","last_became_active":"2024-05-04T03:07:34.018145+0000","last_became_peered":"2024-05-04T03:07:34.018145+0000","last_unstale":"2024-05-04T03:08:16.245493+0000","last_undegraded":"2024-05-04T03:08:16.245493+0000","last_fullsized":"2024-05-04T03:08:16.245493+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:44.529492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248677+0000","last_change":"2024-05-04T03:07:37.070811+0000","last_active":"2024-05-04T03:08:16.248677+0000","last_peered":"2024-05-04T03:08:16.248677+0000","last_clean":"2024-05-04T03:08:16.248677+0000","last_became_active":"2024-05-04T03:07:37.070441+0000","last_became_peered":"2024-05-04T03:07:37.070441+0000","last_unstale":"2024-05-04T03:08:16.248677+0000","last_undegraded":"2024-05-04T03:08:16.248677+0000","last_fullsized":"2024-05-04T03:08:16.248677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:01:48.146339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":61,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:24.222161+0000","last_change":"2024-05-04T03:07:34.020508+0000","last_active":"2024-05-04T03:08:24.222161+0000","last_peered":"2024-05-04T03:08:24.222161+0000","last_clean":"2024-05-04T03:08:24.222161+0000","last_became_active":"2024-05-04T03:07:34.020076+0000","last_became_peered":"2024-05-04T03:07:34.020076+0000","last_unstale":"2024-05-04T03:08:24.222161+0000","last_undegraded":"2024-05-04T03:08:24.222161+0000","last_fullsized":"2024-05-04T03:08:24.222161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:27.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245429+0000","last_change":"2024-05-04T03:07:37.060004+0000","last_active":"2024-05-04T03:08:16.245429+0000","last_peered":"2024-05-04T03:08:16.245429+0000","last_clean":"2024-05-04T03:08:16.245429+0000","last_became_active":"2024-05-04T03:07:37.059350+0000","last_became_peered":"2024-05-04T03:07:37.059350+0000","last_unstale":"2024-05-04T03:08:16.245429+0000","last_undegraded":"2024-05-04T03:08:16.245429+0000","last_fullsized":"2024-05-04T03:08:16.245429+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:37:07.315935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248068+0000","last_change":"2024-05-04T03:07:34.020174+0000","last_active":"2024-05-04T03:08:16.248068+0000","last_peered":"2024-05-04T03:08:16.248068+0000","last_clean":"2024-05-04T03:08:16.248068+0000","last_became_active":"2024-05-04T03:07:34.019790+0000","last_became_peered":"2024-05-04T03:07:34.019790+0000","last_unstale":"2024-05-04T03:08:16.248068+0000","last_undegraded":"2024-05-04T03:08:16.248068+0000","last_fullsized":"2024-05-04T03:08:16.248068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:37:37.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.252143+0000","last_change":"2024-05-04T03:08:16.251277+0000","last_active":"2024-05-04T03:08:16.252143+0000","last_peered":"2024-05-04T03:08:16.252143+0000","last_clean":"2024-05-04T03:08:16.252143+0000","last_became_active":"2024-05-04T03:08:16.250836+0000","last_became_peered":"2024-05-04T03:08:16.250836+0000","last_unstale":"2024-05-04T03:08:16.252143+0000","last_undegraded":"2024-05-04T03:08:16.252143+0000","last_fullsized":"2024-05-04T03:08:16.252143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:59:47.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248397+0000","last_change":"2024-05-04T03:07:34.013188+0000","last_active":"2024-05-04T03:08:16.248397+0000","last_peered":"2024-05-04T03:08:16.248397+0000","last_clean":"2024-05-04T03:08:16.248397+0000","last_became_active":"2024-05-04T03:07:34.012759+0000","last_became_peered":"2024-05-04T03:07:34.012759+0000","last_unstale":"2024-05-04T03:08:16.248397+0000","last_undegraded":"2024-05-04T03:08:16.248397+0000","last_fullsized":"2024-05-04T03:08:16.248397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:54:29.621250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243425+0000","last_change":"2024-05-04T03:07:37.056265+0000","last_active":"2024-05-04T03:08:16.243425+0000","last_peered":"2024-05-04T03:08:16.243425+0000","last_clean":"2024-05-04T03:08:16.243425+0000","last_became_active":"2024-05-04T03:07:37.055957+0000","last_became_peered":"2024-05-04T03:07:37.055957+0000","last_unstale":"2024-05-04T03:08:16.243425+0000","last_undegraded":"2024-05-04T03:08:16.243425+0000","last_fullsized":"2024-05-04T03:08:16.243425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:41.739034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247855+0000","last_change":"2024-05-04T03:07:34.020722+0000","last_active":"2024-05-04T03:08:16.247855+0000","last_peered":"2024-05-04T03:08:16.247855+0000","last_clean":"2024-05-04T03:08:16.247855+0000","last_became_active":"2024-05-04T03:07:34.020028+0000","last_became_peered":"2024-05-04T03:07:34.020028+0000","last_unstale":"2024-05-04T03:08:16.247855+0000","last_undegraded":"2024-05-04T03:08:16.247855+0000","last_fullsized":"2024-05-04T03:08:16.247855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:23:20.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245364+0000","last_change":"2024-05-04T03:07:37.059680+0000","last_active":"2024-05-04T03:08:16.245364+0000","last_peered":"2024-05-04T03:08:16.245364+0000","last_clean":"2024-05-04T03:08:16.245364+0000","last_became_active":"2024-05-04T03:07:37.059294+0000","last_became_peered":"2024-05-04T03:07:37.059294+0000","last_unstale":"2024-05-04T03:08:16.245364+0000","last_undegraded":"2024-05-04T03:08:16.245364+0000","last_fullsized":"2024-05-04T03:08:16.245364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:50:06.251706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":86,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:25.222476+0000","last_change":"2024-05-04T03:07:34.019055+0000","last_active":"2024-05-04T03:08:25.222476+0000","last_peered":"2024-05-04T03:08:25.222476+0000","last_clean":"2024-05-04T03:08:25.222476+0000","last_became_active":"2024-05-04T03:07:34.018638+0000","last_became_peered":"2024-05-04T03:07:34.018638+0000","last_unstale":"2024-05-04T03:08:25.222476+0000","last_undegraded":"2024-05-04T03:08:25.222476+0000","last_fullsized":"2024-05-04T03:08:25.222476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:58:01.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242844+0000","last_change":"2024-05-04T03:07:37.063037+0000","last_active":"2024-05-04T03:08:16.242844+0000","last_peered":"2024-05-04T03:08:16.242844+0000","last_clean":"2024-05-04T03:08:16.242844+0000","last_became_active":"2024-05-04T03:07:37.062350+0000","last_became_peered":"2024-05-04T03:07:37.062350+0000","last_unstale":"2024-05-04T03:08:16.242844+0000","last_undegraded":"2024-05-04T03:08:16.242844+0000","last_fullsized":"2024-05-04T03:08:16.242844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:41:15.898241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.373765+0000","last_change":"2024-05-04T03:08:16.373765+0000","last_active":"2024-05-04T03:08:16.373765+0000","last_peered":"2024-05-04T03:08:16.373765+0000","last_clean":"2024-05-04T03:08:16.373765+0000","last_became_active":"2024-05-04T03:08:16.368588+0000","last_became_peered":"2024-05-04T03:08:16.368588+0000","last_unstale":"2024-05-04T03:08:16.373765+0000","last_undegraded":"2024-05-04T03:08:16.373765+0000","last_fullsized":"2024-05-04T03:08:16.373765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:55:07.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246604+0000","last_change":"2024-05-04T03:07:37.071330+0000","last_active":"2024-05-04T03:08:16.246604+0000","last_peered":"2024-05-04T03:08:16.246604+0000","last_clean":"2024-05-04T03:08:16.246604+0000","last_became_active":"2024-05-04T03:07:37.070743+0000","last_became_peered":"2024-05-04T03:07:37.070743+0000","last_unstale":"2024-05-04T03:08:16.246604+0000","last_undegraded":"2024-05-04T03:08:16.246604+0000","last_fullsized":"2024-05-04T03:08:16.246604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:10:00.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248817+0000","last_change":"2024-05-04T03:07:34.157432+0000","last_active":"2024-05-04T03:08:16.248817+0000","last_peered":"2024-05-04T03:08:16.248817+0000","last_clean":"2024-05-04T03:08:16.248817+0000","last_became_active":"2024-05-04T03:07:34.156976+0000","last_became_peered":"2024-05-04T03:07:34.156976+0000","last_unstale":"2024-05-04T03:08:16.248817+0000","last_undegraded":"2024-05-04T03:08:16.248817+0000","last_fullsized":"2024-05-04T03:08:16.248817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:56.082287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242517+0000","last_change":"2024-05-04T03:07:37.064301+0000","last_active":"2024-05-04T03:08:16.242517+0000","last_peered":"2024-05-04T03:08:16.242517+0000","last_clean":"2024-05-04T03:08:16.242517+0000","last_became_active":"2024-05-04T03:07:37.063691+0000","last_became_peered":"2024-05-04T03:07:37.063691+0000","last_unstale":"2024-05-04T03:08:16.242517+0000","last_undegraded":"2024-05-04T03:08:16.242517+0000","last_fullsized":"2024-05-04T03:08:16.242517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:22:13.844703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":120,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:25.223886+0000","last_change":"2024-05-04T03:07:34.021595+0000","last_active":"2024-05-04T03:08:25.223886+0000","last_peered":"2024-05-04T03:08:25.223886+0000","last_clean":"2024-05-04T03:08:25.223886+0000","last_became_active":"2024-05-04T03:07:34.021126+0000","last_became_peered":"2024-05-04T03:07:34.021126+0000","last_unstale":"2024-05-04T03:08:25.223886+0000","last_undegraded":"2024-05-04T03:08:25.223886+0000","last_fullsized":"2024-05-04T03:08:25.223886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:55.094800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56090,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":64,"num_read_kb":86,"num_write":32,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245497+0000","last_change":"2024-05-04T03:07:37.066365+0000","last_active":"2024-05-04T03:08:16.245497+0000","last_peered":"2024-05-04T03:08:16.245497+0000","last_clean":"2024-05-04T03:08:16.245497+0000","last_became_active":"2024-05-04T03:07:37.065851+0000","last_became_peered":"2024-05-04T03:07:37.065851+0000","last_unstale":"2024-05-04T03:08:16.245497+0000","last_undegraded":"2024-05-04T03:08:16.245497+0000","last_fullsized":"2024-05-04T03:08:16.245497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:03:43.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.263114+0000","last_change":"2024-05-04T03:08:16.262600+0000","last_active":"2024-05-04T03:08:16.263114+0000","last_peered":"2024-05-04T03:08:16.263114+0000","last_clean":"2024-05-04T03:08:16.263114+0000","last_became_active":"2024-05-04T03:08:16.250619+0000","last_became_peered":"2024-05-04T03:08:16.250619+0000","last_unstale":"2024-05-04T03:08:16.263114+0000","last_undegraded":"2024-05-04T03:08:16.263114+0000","last_fullsized":"2024-05-04T03:08:16.263114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:51:21.596467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248552+0000","last_change":"2024-05-04T03:07:37.055356+0000","last_active":"2024-05-04T03:08:16.248552+0000","last_peered":"2024-05-04T03:08:16.248552+0000","last_clean":"2024-05-04T03:08:16.248552+0000","last_became_active":"2024-05-04T03:07:37.055045+0000","last_became_peered":"2024-05-04T03:07:37.055045+0000","last_unstale":"2024-05-04T03:08:16.248552+0000","last_undegraded":"2024-05-04T03:08:16.248552+0000","last_fullsized":"2024-05-04T03:08:16.248552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:06.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247382+0000","last_change":"2024-05-04T03:07:34.021525+0000","last_active":"2024-05-04T03:08:16.247382+0000","last_peered":"2024-05-04T03:08:16.247382+0000","last_clean":"2024-05-04T03:08:16.247382+0000","last_became_active":"2024-05-04T03:07:34.021042+0000","last_became_peered":"2024-05-04T03:07:34.021042+0000","last_unstale":"2024-05-04T03:08:16.247382+0000","last_undegraded":"2024-05-04T03:08:16.247382+0000","last_fullsized":"2024-05-04T03:08:16.247382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:44.090176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244794+0000","last_change":"2024-05-04T03:07:37.066471+0000","last_active":"2024-05-04T03:08:16.244794+0000","last_peered":"2024-05-04T03:08:16.244794+0000","last_clean":"2024-05-04T03:08:16.244794+0000","last_became_active":"2024-05-04T03:07:37.066059+0000","last_became_peered":"2024-05-04T03:07:37.066059+0000","last_unstale":"2024-05-04T03:08:16.244794+0000","last_undegraded":"2024-05-04T03:08:16.244794+0000","last_fullsized":"2024-05-04T03:08:16.244794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:16.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:26.223577+0000","last_change":"2024-05-04T03:07:34.021531+0000","last_active":"2024-05-04T03:08:26.223577+0000","last_peered":"2024-05-04T03:08:26.223577+0000","last_clean":"2024-05-04T03:08:26.223577+0000","last_became_active":"2024-05-04T03:07:34.021112+0000","last_became_peered":"2024-05-04T03:07:34.021112+0000","last_unstale":"2024-05-04T03:08:26.223577+0000","last_undegraded":"2024-05-04T03:08:26.223577+0000","last_fullsized":"2024-05-04T03:08:26.223577+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:28.826426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244611+0000","last_change":"2024-05-04T03:07:37.067155+0000","last_active":"2024-05-04T03:08:16.244611+0000","last_peered":"2024-05-04T03:08:16.244611+0000","last_clean":"2024-05-04T03:08:16.244611+0000","last_became_active":"2024-05-04T03:07:37.066783+0000","last_became_peered":"2024-05-04T03:07:37.066783+0000","last_unstale":"2024-05-04T03:08:16.244611+0000","last_undegraded":"2024-05-04T03:08:16.244611+0000","last_fullsized":"2024-05-04T03:08:16.244611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:23.217930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245013+0000","last_change":"2024-05-04T03:07:34.158227+0000","last_active":"2024-05-04T03:08:16.245013+0000","last_peered":"2024-05-04T03:08:16.245013+0000","last_clean":"2024-05-04T03:08:16.245013+0000","last_became_active":"2024-05-04T03:07:34.156905+0000","last_became_peered":"2024-05-04T03:07:34.156905+0000","last_unstale":"2024-05-04T03:08:16.245013+0000","last_undegraded":"2024-05-04T03:08:16.245013+0000","last_fullsized":"2024-05-04T03:08:16.245013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:40:41.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247601+0000","last_change":"2024-05-04T03:07:37.075559+0000","last_active":"2024-05-04T03:08:16.247601+0000","last_peered":"2024-05-04T03:08:16.247601+0000","last_clean":"2024-05-04T03:08:16.247601+0000","last_became_active":"2024-05-04T03:07:37.075220+0000","last_became_peered":"2024-05-04T03:07:37.075220+0000","last_unstale":"2024-05-04T03:08:16.247601+0000","last_undegraded":"2024-05-04T03:08:16.247601+0000","last_fullsized":"2024-05-04T03:08:16.247601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:51:23.942514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246548+0000","last_change":"2024-05-04T03:07:34.020326+0000","last_active":"2024-05-04T03:08:16.246548+0000","last_peered":"2024-05-04T03:08:16.246548+0000","last_clean":"2024-05-04T03:08:16.246548+0000","last_became_active":"2024-05-04T03:07:34.019472+0000","last_became_peered":"2024-05-04T03:07:34.019472+0000","last_unstale":"2024-05-04T03:08:16.246548+0000","last_undegraded":"2024-05-04T03:08:16.246548+0000","last_fullsized":"2024-05-04T03:08:16.246548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:33:28.375901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247007+0000","last_change":"2024-05-04T03:07:37.066223+0000","last_active":"2024-05-04T03:08:16.247007+0000","last_peered":"2024-05-04T03:08:16.247007+0000","last_clean":"2024-05-04T03:08:16.247007+0000","last_became_active":"2024-05-04T03:07:37.065859+0000","last_became_peered":"2024-05-04T03:07:37.065859+0000","last_unstale":"2024-05-04T03:08:16.247007+0000","last_undegraded":"2024-05-04T03:08:16.247007+0000","last_fullsized":"2024-05-04T03:08:16.247007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:44.902986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245474+0000","last_change":"2024-05-04T03:07:34.158674+0000","last_active":"2024-05-04T03:08:16.245474+0000","last_peered":"2024-05-04T03:08:16.245474+0000","last_clean":"2024-05-04T03:08:16.245474+0000","last_became_active":"2024-05-04T03:07:34.157952+0000","last_became_peered":"2024-05-04T03:07:34.157952+0000","last_unstale":"2024-05-04T03:08:16.245474+0000","last_undegraded":"2024-05-04T03:08:16.245474+0000","last_fullsized":"2024-05-04T03:08:16.245474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:19:11.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248252+0000","last_change":"2024-05-04T03:07:37.053897+0000","last_active":"2024-05-04T03:08:16.248252+0000","last_peered":"2024-05-04T03:08:16.248252+0000","last_clean":"2024-05-04T03:08:16.248252+0000","last_became_active":"2024-05-04T03:07:37.053466+0000","last_became_peered":"2024-05-04T03:07:37.053466+0000","last_unstale":"2024-05-04T03:08:16.248252+0000","last_undegraded":"2024-05-04T03:08:16.248252+0000","last_fullsized":"2024-05-04T03:08:16.248252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:50.650805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":62,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:26.223904+0000","last_change":"2024-05-04T03:07:34.021494+0000","last_active":"2024-05-04T03:08:26.223904+0000","last_peered":"2024-05-04T03:08:26.223904+0000","last_clean":"2024-05-04T03:08:26.223904+0000","last_became_active":"2024-05-04T03:07:34.021113+0000","last_became_peered":"2024-05-04T03:07:34.021113+0000","last_unstale":"2024-05-04T03:08:26.223904+0000","last_undegraded":"2024-05-04T03:08:26.223904+0000","last_fullsized":"2024-05-04T03:08:26.223904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:02:09.514961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248164+0000","last_change":"2024-05-04T03:07:37.068199+0000","last_active":"2024-05-04T03:08:16.248164+0000","last_peered":"2024-05-04T03:08:16.248164+0000","last_clean":"2024-05-04T03:08:16.248164+0000","last_became_active":"2024-05-04T03:07:37.067828+0000","last_became_peered":"2024-05-04T03:07:37.067828+0000","last_unstale":"2024-05-04T03:08:16.248164+0000","last_undegraded":"2024-05-04T03:08:16.248164+0000","last_fullsized":"2024-05-04T03:08:16.248164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:28.882920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248113+0000","last_change":"2024-05-04T03:07:37.069753+0000","last_active":"2024-05-04T03:08:16.248113+0000","last_peered":"2024-05-04T03:08:16.248113+0000","last_clean":"2024-05-04T03:08:16.248113+0000","last_became_active":"2024-05-04T03:07:37.069338+0000","last_became_peered":"2024-05-04T03:07:37.069338+0000","last_unstale":"2024-05-04T03:08:16.248113+0000","last_undegraded":"2024-05-04T03:08:16.248113+0000","last_fullsized":"2024-05-04T03:08:16.248113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:37:11.355749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":61,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:25.221689+0000","last_change":"2024-05-04T03:07:34.157229+0000","last_active":"2024-05-04T03:08:25.221689+0000","last_peered":"2024-05-04T03:08:25.221689+0000","last_clean":"2024-05-04T03:08:25.221689+0000","last_became_active":"2024-05-04T03:07:34.156776+0000","last_became_peered":"2024-05-04T03:07:34.156776+0000","last_unstale":"2024-05-04T03:08:25.221689+0000","last_undegraded":"2024-05-04T03:08:25.221689+0000","last_fullsized":"2024-05-04T03:08:25.221689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:26.879754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243715+0000","last_change":"2024-05-04T03:07:37.067025+0000","last_active":"2024-05-04T03:08:16.243715+0000","last_peered":"2024-05-04T03:08:16.243715+0000","last_clean":"2024-05-04T03:08:16.243715+0000","last_became_active":"2024-05-04T03:07:37.066359+0000","last_became_peered":"2024-05-04T03:07:37.066359+0000","last_unstale":"2024-05-04T03:08:16.243715+0000","last_undegraded":"2024-05-04T03:08:16.243715+0000","last_fullsized":"2024-05-04T03:08:16.243715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:44:35.604859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367108+0000","last_change":"2024-05-04T03:07:34.018019+0000","last_active":"2024-05-04T03:08:16.367108+0000","last_peered":"2024-05-04T03:08:16.367108+0000","last_clean":"2024-05-04T03:08:16.367108+0000","last_became_active":"2024-05-04T03:07:34.017556+0000","last_became_peered":"2024-05-04T03:07:34.017556+0000","last_unstale":"2024-05-04T03:08:16.367108+0000","last_undegraded":"2024-05-04T03:08:16.367108+0000","last_fullsized":"2024-05-04T03:08:16.367108+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:31:34.158164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244461+0000","last_change":"2024-05-04T03:06:06.704755+0000","last_active":"2024-05-04T03:08:16.244461+0000","last_peered":"2024-05-04T03:08:16.244461+0000","last_clean":"2024-05-04T03:08:16.244461+0000","last_became_active":"2024-05-04T03:06:06.698747+0000","last_became_peered":"2024-05-04T03:06:06.698747+0000","last_unstale":"2024-05-04T03:08:16.244461+0000","last_undegraded":"2024-05-04T03:08:16.244461+0000","last_fullsized":"2024-05-04T03:08:16.244461+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_clean_scrub_stamp":"2024-05-04T03:02:43.385377+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:04:50.012414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248266+0000","last_change":"2024-05-04T03:07:37.057043+0000","last_active":"2024-05-04T03:08:16.248266+0000","last_peered":"2024-05-04T03:08:16.248266+0000","last_clean":"2024-05-04T03:08:16.248266+0000","last_became_active":"2024-05-04T03:07:37.056725+0000","last_became_peered":"2024-05-04T03:07:37.056725+0000","last_unstale":"2024-05-04T03:08:16.248266+0000","last_undegraded":"2024-05-04T03:08:16.248266+0000","last_fullsized":"2024-05-04T03:08:16.248266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:13:28.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.376510+0000","last_change":"2024-05-04T03:08:16.376510+0000","last_active":"2024-05-04T03:08:16.376510+0000","last_peered":"2024-05-04T03:08:16.376510+0000","last_clean":"2024-05-04T03:08:16.376510+0000","last_became_active":"2024-05-04T03:08:16.368625+0000","last_became_peered":"2024-05-04T03:08:16.368625+0000","last_unstale":"2024-05-04T03:08:16.376510+0000","last_undegraded":"2024-05-04T03:08:16.376510+0000","last_fullsized":"2024-05-04T03:08:16.376510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:27:50.504487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245592+0000","last_change":"2024-05-04T03:07:34.157346+0000","last_active":"2024-05-04T03:08:16.245592+0000","last_peered":"2024-05-04T03:08:16.245592+0000","last_clean":"2024-05-04T03:08:16.245592+0000","last_became_active":"2024-05-04T03:07:34.156914+0000","last_became_peered":"2024-05-04T03:07:34.156914+0000","last_unstale":"2024-05-04T03:08:16.245592+0000","last_undegraded":"2024-05-04T03:08:16.245592+0000","last_fullsized":"2024-05-04T03:08:16.245592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:54:17.059644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248699+0000","last_change":"2024-05-04T03:07:37.054373+0000","last_active":"2024-05-04T03:08:16.248699+0000","last_peered":"2024-05-04T03:08:16.248699+0000","last_clean":"2024-05-04T03:08:16.248699+0000","last_became_active":"2024-05-04T03:07:37.053764+0000","last_became_peered":"2024-05-04T03:07:37.053764+0000","last_unstale":"2024-05-04T03:08:16.248699+0000","last_undegraded":"2024-05-04T03:08:16.248699+0000","last_fullsized":"2024-05-04T03:08:16.248699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:13:47.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:21.079361+0000","last_change":"2024-05-04T03:07:34.020190+0000","last_active":"2024-05-04T03:08:21.079361+0000","last_peered":"2024-05-04T03:08:21.079361+0000","last_clean":"2024-05-04T03:08:21.079361+0000","last_became_active":"2024-05-04T03:07:34.019786+0000","last_became_peered":"2024-05-04T03:07:34.019786+0000","last_unstale":"2024-05-04T03:08:21.079361+0000","last_undegraded":"2024-05-04T03:08:21.079361+0000","last_fullsized":"2024-05-04T03:08:21.079361+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:11.243414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244079+0000","last_change":"2024-05-04T03:07:37.055274+0000","last_active":"2024-05-04T03:08:16.244079+0000","last_peered":"2024-05-04T03:08:16.244079+0000","last_clean":"2024-05-04T03:08:16.244079+0000","last_became_active":"2024-05-04T03:07:37.054968+0000","last_became_peered":"2024-05-04T03:07:37.054968+0000","last_unstale":"2024-05-04T03:08:16.244079+0000","last_undegraded":"2024-05-04T03:08:16.244079+0000","last_fullsized":"2024-05-04T03:08:16.244079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.229266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367455+0000","last_change":"2024-05-04T03:07:34.132966+0000","last_active":"2024-05-04T03:08:16.367455+0000","last_peered":"2024-05-04T03:08:16.367455+0000","last_clean":"2024-05-04T03:08:16.367455+0000","last_became_active":"2024-05-04T03:07:34.132529+0000","last_became_peered":"2024-05-04T03:07:34.132529+0000","last_unstale":"2024-05-04T03:08:16.367455+0000","last_undegraded":"2024-05-04T03:08:16.367455+0000","last_fullsized":"2024-05-04T03:08:16.367455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:31:37.122097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.368747+0000","last_change":"2024-05-04T03:08:16.368747+0000","last_active":"2024-05-04T03:08:16.368747+0000","last_peered":"2024-05-04T03:08:16.368747+0000","last_clean":"2024-05-04T03:08:16.368747+0000","last_became_active":"2024-05-04T03:08:16.368376+0000","last_became_peered":"2024-05-04T03:08:16.368376+0000","last_unstale":"2024-05-04T03:08:16.368747+0000","last_undegraded":"2024-05-04T03:08:16.368747+0000","last_fullsized":"2024-05-04T03:08:16.368747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:42:32.085650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242445+0000","last_change":"2024-05-04T03:07:34.157219+0000","last_active":"2024-05-04T03:08:16.242445+0000","last_peered":"2024-05-04T03:08:16.242445+0000","last_clean":"2024-05-04T03:08:16.242445+0000","last_became_active":"2024-05-04T03:07:34.156866+0000","last_became_peered":"2024-05-04T03:07:34.156866+0000","last_unstale":"2024-05-04T03:08:16.242445+0000","last_undegraded":"2024-05-04T03:08:16.242445+0000","last_fullsized":"2024-05-04T03:08:16.242445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:15:34.138825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245287+0000","last_change":"2024-05-04T03:07:37.058733+0000","last_active":"2024-05-04T03:08:16.245287+0000","last_peered":"2024-05-04T03:08:16.245287+0000","last_clean":"2024-05-04T03:08:16.245287+0000","last_became_active":"2024-05-04T03:07:37.058396+0000","last_became_peered":"2024-05-04T03:07:37.058396+0000","last_unstale":"2024-05-04T03:08:16.245287+0000","last_undegraded":"2024-05-04T03:08:16.245287+0000","last_fullsized":"2024-05-04T03:08:16.245287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:35.563728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243514+0000","last_change":"2024-05-04T03:07:34.016458+0000","last_active":"2024-05-04T03:08:16.243514+0000","last_peered":"2024-05-04T03:08:16.243514+0000","last_clean":"2024-05-04T03:08:16.243514+0000","last_became_active":"2024-05-04T03:07:34.016038+0000","last_became_peered":"2024-05-04T03:07:34.016038+0000","last_unstale":"2024-05-04T03:08:16.243514+0000","last_undegraded":"2024-05-04T03:08:16.243514+0000","last_fullsized":"2024-05-04T03:08:16.243514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:39:25.156047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247376+0000","last_change":"2024-05-04T03:07:37.064024+0000","last_active":"2024-05-04T03:08:16.247376+0000","last_peered":"2024-05-04T03:08:16.247376+0000","last_clean":"2024-05-04T03:08:16.247376+0000","last_became_active":"2024-05-04T03:07:37.063642+0000","last_became_peered":"2024-05-04T03:07:37.063642+0000","last_unstale":"2024-05-04T03:08:16.247376+0000","last_undegraded":"2024-05-04T03:08:16.247376+0000","last_fullsized":"2024-05-04T03:08:16.247376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:59:58.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.267243+0000","last_change":"2024-05-04T03:08:16.266763+0000","last_active":"2024-05-04T03:08:16.267243+0000","last_peered":"2024-05-04T03:08:16.267243+0000","last_clean":"2024-05-04T03:08:16.267243+0000","last_became_active":"2024-05-04T03:08:16.251212+0000","last_became_peered":"2024-05-04T03:08:16.251212+0000","last_unstale":"2024-05-04T03:08:16.267243+0000","last_undegraded":"2024-05-04T03:08:16.267243+0000","last_fullsized":"2024-05-04T03:08:16.267243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:31:27.444610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243667+0000","last_change":"2024-05-04T03:07:37.063753+0000","last_active":"2024-05-04T03:08:16.243667+0000","last_peered":"2024-05-04T03:08:16.243667+0000","last_clean":"2024-05-04T03:08:16.243667+0000","last_became_active":"2024-05-04T03:07:37.063318+0000","last_became_peered":"2024-05-04T03:07:37.063318+0000","last_unstale":"2024-05-04T03:08:16.243667+0000","last_undegraded":"2024-05-04T03:08:16.243667+0000","last_fullsized":"2024-05-04T03:08:16.243667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:48.217136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245115+0000","last_change":"2024-05-04T03:07:34.022522+0000","last_active":"2024-05-04T03:08:16.245115+0000","last_peered":"2024-05-04T03:08:16.245115+0000","last_clean":"2024-05-04T03:08:16.245115+0000","last_became_active":"2024-05-04T03:07:34.021018+0000","last_became_peered":"2024-05-04T03:07:34.021018+0000","last_unstale":"2024-05-04T03:08:16.245115+0000","last_undegraded":"2024-05-04T03:08:16.245115+0000","last_fullsized":"2024-05-04T03:08:16.245115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:47:04.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244631+0000","last_change":"2024-05-04T03:07:37.069930+0000","last_active":"2024-05-04T03:08:16.244631+0000","last_peered":"2024-05-04T03:08:16.244631+0000","last_clean":"2024-05-04T03:08:16.244631+0000","last_became_active":"2024-05-04T03:07:37.069580+0000","last_became_peered":"2024-05-04T03:07:37.069580+0000","last_unstale":"2024-05-04T03:08:16.244631+0000","last_undegraded":"2024-05-04T03:08:16.244631+0000","last_fullsized":"2024-05-04T03:08:16.244631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:15:33.815970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":62,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:26.224290+0000","last_change":"2024-05-04T03:07:34.157969+0000","last_active":"2024-05-04T03:08:26.224290+0000","last_peered":"2024-05-04T03:08:26.224290+0000","last_clean":"2024-05-04T03:08:26.224290+0000","last_became_active":"2024-05-04T03:07:34.157563+0000","last_became_peered":"2024-05-04T03:07:34.157563+0000","last_unstale":"2024-05-04T03:08:26.224290+0000","last_undegraded":"2024-05-04T03:08:26.224290+0000","last_fullsized":"2024-05-04T03:08:26.224290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:00:43.562109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245064+0000","last_change":"2024-05-04T03:07:37.071279+0000","last_active":"2024-05-04T03:08:16.245064+0000","last_peered":"2024-05-04T03:08:16.245064+0000","last_clean":"2024-05-04T03:08:16.245064+0000","last_became_active":"2024-05-04T03:07:37.070449+0000","last_became_peered":"2024-05-04T03:07:37.070449+0000","last_unstale":"2024-05-04T03:08:16.245064+0000","last_undegraded":"2024-05-04T03:08:16.245064+0000","last_fullsized":"2024-05-04T03:08:16.245064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:13:22.535788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248538+0000","last_change":"2024-05-04T03:07:34.017010+0000","last_active":"2024-05-04T03:08:16.248538+0000","last_peered":"2024-05-04T03:08:16.248538+0000","last_clean":"2024-05-04T03:08:16.248538+0000","last_became_active":"2024-05-04T03:07:34.016521+0000","last_became_peered":"2024-05-04T03:07:34.016521+0000","last_unstale":"2024-05-04T03:08:16.248538+0000","last_undegraded":"2024-05-04T03:08:16.248538+0000","last_fullsized":"2024-05-04T03:08:16.248538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:36.684656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245167+0000","last_change":"2024-05-04T03:07:37.066526+0000","last_active":"2024-05-04T03:08:16.245167+0000","last_peered":"2024-05-04T03:08:16.245167+0000","last_clean":"2024-05-04T03:08:16.245167+0000","last_became_active":"2024-05-04T03:07:37.065312+0000","last_became_peered":"2024-05-04T03:07:37.065312+0000","last_unstale":"2024-05-04T03:08:16.245167+0000","last_undegraded":"2024-05-04T03:08:16.245167+0000","last_fullsized":"2024-05-04T03:08:16.245167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:10.077295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248927+0000","last_change":"2024-05-04T03:07:34.016898+0000","last_active":"2024-05-04T03:08:16.248927+0000","last_peered":"2024-05-04T03:08:16.248927+0000","last_clean":"2024-05-04T03:08:16.248927+0000","last_became_active":"2024-05-04T03:07:34.016524+0000","last_became_peered":"2024-05-04T03:07:34.016524+0000","last_unstale":"2024-05-04T03:08:16.248927+0000","last_undegraded":"2024-05-04T03:08:16.248927+0000","last_fullsized":"2024-05-04T03:08:16.248927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:01.236207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245122+0000","last_change":"2024-05-04T03:07:37.061501+0000","last_active":"2024-05-04T03:08:16.245122+0000","last_peered":"2024-05-04T03:08:16.245122+0000","last_clean":"2024-05-04T03:08:16.245122+0000","last_became_active":"2024-05-04T03:07:37.061077+0000","last_became_peered":"2024-05-04T03:07:37.061077+0000","last_unstale":"2024-05-04T03:08:16.245122+0000","last_undegraded":"2024-05-04T03:08:16.245122+0000","last_fullsized":"2024-05-04T03:08:16.245122+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:32.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247502+0000","last_change":"2024-05-04T03:07:37.064612+0000","last_active":"2024-05-04T03:08:16.247502+0000","last_peered":"2024-05-04T03:08:16.247502+0000","last_clean":"2024-05-04T03:08:16.247502+0000","last_became_active":"2024-05-04T03:07:37.064303+0000","last_became_peered":"2024-05-04T03:07:37.064303+0000","last_unstale":"2024-05-04T03:08:16.247502+0000","last_undegraded":"2024-05-04T03:08:16.247502+0000","last_fullsized":"2024-05-04T03:08:16.247502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:47:07.811765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250296+0000","last_change":"2024-05-04T03:07:37.061577+0000","last_active":"2024-05-04T03:08:16.250296+0000","last_peered":"2024-05-04T03:08:16.250296+0000","last_clean":"2024-05-04T03:08:16.250296+0000","last_became_active":"2024-05-04T03:07:37.061214+0000","last_became_peered":"2024-05-04T03:07:37.061214+0000","last_unstale":"2024-05-04T03:08:16.250296+0000","last_undegraded":"2024-05-04T03:08:16.250296+0000","last_fullsized":"2024-05-04T03:08:16.250296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:52:11.733257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.251332+0000","last_change":"2024-05-04T03:07:37.062906+0000","last_active":"2024-05-04T03:08:16.251332+0000","last_peered":"2024-05-04T03:08:16.251332+0000","last_clean":"2024-05-04T03:08:16.251332+0000","last_became_active":"2024-05-04T03:07:37.062510+0000","last_became_peered":"2024-05-04T03:07:37.062510+0000","last_unstale":"2024-05-04T03:08:16.251332+0000","last_undegraded":"2024-05-04T03:08:16.251332+0000","last_fullsized":"2024-05-04T03:08:16.251332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:57:58.910490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246507+0000","last_change":"2024-05-04T03:07:37.068367+0000","last_active":"2024-05-04T03:08:16.246507+0000","last_peered":"2024-05-04T03:08:16.246507+0000","last_clean":"2024-05-04T03:08:16.246507+0000","last_became_active":"2024-05-04T03:07:37.068038+0000","last_became_peered":"2024-05-04T03:07:37.068038+0000","last_unstale":"2024-05-04T03:08:16.246507+0000","last_undegraded":"2024-05-04T03:08:16.246507+0000","last_fullsized":"2024-05-04T03:08:16.246507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:11:19.195002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250759+0000","last_change":"2024-05-04T03:08:16.250759+0000","last_active":"2024-05-04T03:08:16.250759+0000","last_peered":"2024-05-04T03:08:16.250759+0000","last_clean":"2024-05-04T03:08:16.250759+0000","last_became_active":"2024-05-04T03:08:16.250379+0000","last_became_peered":"2024-05-04T03:08:16.250379+0000","last_unstale":"2024-05-04T03:08:16.250759+0000","last_undegraded":"2024-05-04T03:08:16.250759+0000","last_fullsized":"2024-05-04T03:08:16.250759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:58:09.419009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246960+0000","last_change":"2024-05-04T03:07:37.064584+0000","last_active":"2024-05-04T03:08:16.246960+0000","last_peered":"2024-05-04T03:08:16.246960+0000","last_clean":"2024-05-04T03:08:16.246960+0000","last_became_active":"2024-05-04T03:07:37.064183+0000","last_became_peered":"2024-05-04T03:07:37.064183+0000","last_unstale":"2024-05-04T03:08:16.246960+0000","last_undegraded":"2024-05-04T03:08:16.246960+0000","last_fullsized":"2024-05-04T03:08:16.246960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:17.121081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250902+0000","last_change":"2024-05-04T03:07:37.066742+0000","last_active":"2024-05-04T03:08:16.250902+0000","last_peered":"2024-05-04T03:08:16.250902+0000","last_clean":"2024-05-04T03:08:16.250902+0000","last_became_active":"2024-05-04T03:07:37.066126+0000","last_became_peered":"2024-05-04T03:07:37.066126+0000","last_unstale":"2024-05-04T03:08:16.250902+0000","last_undegraded":"2024-05-04T03:08:16.250902+0000","last_fullsized":"2024-05-04T03:08:16.250902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:57.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244050+0000","last_change":"2024-05-04T03:07:37.068122+0000","last_active":"2024-05-04T03:08:16.244050+0000","last_peered":"2024-05-04T03:08:16.244050+0000","last_clean":"2024-05-04T03:08:16.244050+0000","last_became_active":"2024-05-04T03:07:37.066321+0000","last_became_peered":"2024-05-04T03:07:37.066321+0000","last_unstale":"2024-05-04T03:08:16.244050+0000","last_undegraded":"2024-05-04T03:08:16.244050+0000","last_fullsized":"2024-05-04T03:08:16.244050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:25:09.321853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247940+0000","last_change":"2024-05-04T03:07:37.060322+0000","last_active":"2024-05-04T03:08:16.247940+0000","last_peered":"2024-05-04T03:08:16.247940+0000","last_clean":"2024-05-04T03:08:16.247940+0000","last_became_active":"2024-05-04T03:07:37.059944+0000","last_became_peered":"2024-05-04T03:07:37.059944+0000","last_unstale":"2024-05-04T03:08:16.247940+0000","last_undegraded":"2024-05-04T03:08:16.247940+0000","last_fullsized":"2024-05-04T03:08:16.247940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.105085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243469+0000","last_change":"2024-05-04T03:07:37.056269+0000","last_active":"2024-05-04T03:08:16.243469+0000","last_peered":"2024-05-04T03:08:16.243469+0000","last_clean":"2024-05-04T03:08:16.243469+0000","last_became_active":"2024-05-04T03:07:37.055886+0000","last_became_peered":"2024-05-04T03:07:37.055886+0000","last_unstale":"2024-05-04T03:08:16.243469+0000","last_undegraded":"2024-05-04T03:08:16.243469+0000","last_fullsized":"2024-05-04T03:08:16.243469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:55:11.292577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242733+0000","last_change":"2024-05-04T03:07:37.071013+0000","last_active":"2024-05-04T03:08:16.242733+0000","last_peered":"2024-05-04T03:08:16.242733+0000","last_clean":"2024-05-04T03:08:16.242733+0000","last_became_active":"2024-05-04T03:07:37.069346+0000","last_became_peered":"2024-05-04T03:07:37.069346+0000","last_unstale":"2024-05-04T03:08:16.242733+0000","last_undegraded":"2024-05-04T03:08:16.242733+0000","last_fullsized":"2024-05-04T03:08:16.242733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:19:11.898766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.249009+0000","last_change":"2024-05-04T03:07:37.064962+0000","last_active":"2024-05-04T03:08:16.249009+0000","last_peered":"2024-05-04T03:08:16.249009+0000","last_clean":"2024-05-04T03:08:16.249009+0000","last_became_active":"2024-05-04T03:07:37.064637+0000","last_became_peered":"2024-05-04T03:07:37.064637+0000","last_unstale":"2024-05-04T03:08:16.249009+0000","last_undegraded":"2024-05-04T03:08:16.249009+0000","last_fullsized":"2024-05-04T03:08:16.249009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:10:10.659621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246565+0000","last_change":"2024-05-04T03:07:37.056721+0000","last_active":"2024-05-04T03:08:16.246565+0000","last_peered":"2024-05-04T03:08:16.246565+0000","last_clean":"2024-05-04T03:08:16.246565+0000","last_became_active":"2024-05-04T03:07:37.056353+0000","last_became_peered":"2024-05-04T03:07:37.056353+0000","last_unstale":"2024-05-04T03:08:16.246565+0000","last_undegraded":"2024-05-04T03:08:16.246565+0000","last_fullsized":"2024-05-04T03:08:16.246565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:30:47.476579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286351,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62903,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":329,"num_read_kb":233,"num_write":243,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":197223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710746,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":880640,"data_stored":425832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1073274,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070501,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":425694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266729,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":426072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1018590,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":425411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":486671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":426750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":486823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":425236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":425663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":1017877,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:08:30.391 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-04T03:08:30.392 INFO:tasks.cephadm:Setting up client nodes... 2024-05-04T03:08:30.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-04T03:08:30.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:30 smithi066 ceph-mon[33264]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:08:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:30 smithi066 ceph-mon[33264]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:08:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:30 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:30 smithi167 ceph-mon[33614]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:08:30.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:30 smithi167 ceph-mon[33614]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:08:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:30 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:30 smithi039 ceph-mon[29715]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:08:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:30 smithi039 ceph-mon[29715]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:08:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:30 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:30.736 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.476+0000 7f7aac218700 1 -- 172.21.15.39:0/1565518775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 msgr2=0x7f7aa410e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.476+0000 7f7aac218700 1 --2- 172.21.15.39:0/1565518775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 0x7f7aa410e8f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00b0a0 tx=0x7f7a9c00b3b0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.476+0000 7f7aac218700 1 -- 172.21.15.39:0/1565518775 shutdown_connections 2024-05-04T03:08:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.476+0000 7f7aac218700 1 --2- 172.21.15.39:0/1565518775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 0x7f7aa410e8f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.476+0000 7f7aac218700 1 --2- 172.21.15.39:0/1565518775 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa4106570 0x7f7aa4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.476+0000 7f7aac218700 1 --2- 172.21.15.39:0/1565518775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa41076a0 0x7f7aa4107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.476+0000 7f7aac218700 1 -- 172.21.15.39:0/1565518775 >> 172.21.15.39:0/1565518775 conn(0x7f7aa4101bd0 msgr2=0x7f7aa4104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:32.478 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:32 smithi039 ceph-mon[29715]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-04T03:08:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.476+0000 7f7aac218700 1 -- 172.21.15.39:0/1565518775 shutdown_connections 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.477+0000 7f7aac218700 1 -- 172.21.15.39:0/1565518775 wait complete. 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.477+0000 7f7aac218700 1 Processor -- start 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.477+0000 7f7aac218700 1 -- start start 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aac218700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa4106570 0x7f7aa419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aac218700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa41076a0 0x7f7aa419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aac218700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 0x7f7aa41a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aac218700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4111ed0 con 0x7f7aa4108040 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aac218700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4111d50 con 0x7f7aa41076a0 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aac218700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4112050 con 0x7f7aa4106570 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 0x7f7aa41a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 0x7f7aa41a3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58270/0 (socket says 172.21.15.39:58270) 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 -- 172.21.15.39:0/3930314929 learned_addr learned my addr 172.21.15.39:0/3930314929 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aa9fb4700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa4106570 0x7f7aa419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aa97b3700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa41076a0 0x7f7aa419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 -- 172.21.15.39:0/3930314929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa4106570 msgr2=0x7f7aa419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa4106570 0x7f7aa419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 -- 172.21.15.39:0/3930314929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa41076a0 msgr2=0x7f7aa419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa41076a0 0x7f7aa419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 -- 172.21.15.39:0/3930314929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa41a4580 con 0x7f7aa4108040 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aa97b3700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa41076a0 0x7f7aa419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.478+0000 7f7aaa7b5700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 0x7f7aa41a3f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0062a0 tx=0x7f7a9c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.479+0000 7f7a9affd700 1 -- 172.21.15.39:0/3930314929 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c021070 con 0x7f7aa4108040 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.479+0000 7f7a9affd700 1 -- 172.21.15.39:0/3930314929 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a9c007b80 con 0x7f7aa4108040 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.479+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa41a4810 con 0x7f7aa4108040 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.479+0000 7f7a9affd700 1 -- 172.21.15.39:0/3930314929 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004730 con 0x7f7aa4108040 2024-05-04T03:08:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.479+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa41a4c60 con 0x7f7aa4108040 2024-05-04T03:08:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.480+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa40512c0 con 0x7f7aa4108040 2024-05-04T03:08:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.481+0000 7f7a9affd700 1 -- 172.21.15.39:0/3930314929 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7a9c003a00 con 0x7f7aa4108040 2024-05-04T03:08:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.482+0000 7f7a9affd700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a900b2910 0x7f7a900b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.482+0000 7f7aa9fb4700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a900b2910 0x7f7a900b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.483+0000 7f7a9affd700 1 -- 172.21.15.39:0/3930314929 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7a9c00f020 con 0x7f7aa4108040 2024-05-04T03:08:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.483+0000 7f7aa9fb4700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a900b2910 0x7f7a900b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7aa41075a0 tx=0x7f7aa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:32.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.488+0000 7f7a9affd700 1 -- 172.21.15.39:0/3930314929 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a9c02a020 con 0x7f7aa4108040 2024-05-04T03:08:32.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:32 smithi066 ceph-mon[33264]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-04T03:08:32.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:32 smithi167 ceph-mon[33614]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-04T03:08:32.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.803+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7aa402e5f0 con 0x7f7aa4108040 2024-05-04T03:08:32.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.808+0000 7f7a9affd700 1 -- 172.21.15.39:0/3930314929 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f7a9c0ab940 con 0x7f7aa4108040 2024-05-04T03:08:32.808 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-05-04T03:08:32.809 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCwpjVmL+78LxAAckApuO/iG97bk1wPxMzH3g== 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a900b2910 msgr2=0x7f7a900b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a900b2910 0x7f7a900b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7aa41075a0 tx=0x7f7aa0009260 comp rx=0 tx=0).stop 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 msgr2=0x7f7aa41a3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 0x7f7aa41a3f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0062a0 tx=0x7f7a9c004c60 comp rx=0 tx=0).stop 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 shutdown_connections 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a900b2910 0x7f7a900b4e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa4108040 0x7f7aa41a3f00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7aa4106570 0x7f7aa419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 --2- 172.21.15.39:0/3930314929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7aa41076a0 0x7f7aa419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:32.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.810+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 >> 172.21.15.39:0/3930314929 conn(0x7f7aa4101bd0 msgr2=0x7f7aa410a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:32.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.811+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 shutdown_connections 2024-05-04T03:08:32.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:32.811+0000 7f7aac218700 1 -- 172.21.15.39:0/3930314929 wait complete. 2024-05-04T03:08:33.526 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:08:33.526 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-04T03:08:33.526 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-04T03:08:33.562 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-04T03:08:33.562 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-04T03:08:33.563 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-04T03:08:33.850 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:33.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:33 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3930314929' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-04T03:08:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:33 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3930314929' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-04T03:08:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:33 smithi167 ceph-mon[33614]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-04T03:08:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:08:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:08:33.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:33 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:33.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:33.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:33.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:33 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3930314929' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-04T03:08:33.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:33 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3930314929' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-04T03:08:33.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:33 smithi039 ceph-mon[29715]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-04T03:08:33.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:08:33.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:08:33.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:33 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:33 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3930314929' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-04T03:08:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:33 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3930314929' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-04T03:08:34.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:33 smithi066 ceph-mon[33264]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-04T03:08:34.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:08:34.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:08:34.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:33 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:08:35.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.619+0000 7efdeabb1700 1 -- 172.21.15.39:0/3952415981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde4109860 msgr2=0x7efde41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.619+0000 7efdeabb1700 1 --2- 172.21.15.39:0/3952415981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde4109860 0x7efde41111c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efdd40099c0 tx=0x7efdd4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.619+0000 7efdeabb1700 1 -- 172.21.15.39:0/3952415981 shutdown_connections 2024-05-04T03:08:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.619+0000 7efdeabb1700 1 --2- 172.21.15.39:0/3952415981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde4109860 0x7efde41111c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.619+0000 7efdeabb1700 1 --2- 172.21.15.39:0/3952415981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efde41045e0 0x7efde41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.619+0000 7efdeabb1700 1 --2- 172.21.15.39:0/3952415981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efde4108f10 0x7efde4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.619+0000 7efdeabb1700 1 -- 172.21.15.39:0/3952415981 >> 172.21.15.39:0/3952415981 conn(0x7efde40ffe70 msgr2=0x7efde41022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.620+0000 7efdeabb1700 1 -- 172.21.15.39:0/3952415981 shutdown_connections 2024-05-04T03:08:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.620+0000 7efdeabb1700 1 -- 172.21.15.39:0/3952415981 wait complete. 2024-05-04T03:08:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.620+0000 7efdeabb1700 1 Processor -- start 2024-05-04T03:08:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.620+0000 7efdeabb1700 1 -- start start 2024-05-04T03:08:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efdeabb1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde41045e0 0x7efde419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efdeabb1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efde4108f10 0x7efde419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efdeabb1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efde4109860 0x7efde419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efdeabb1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efde41143b0 con 0x7efde41045e0 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efdeabb1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efde4114230 con 0x7efde4108f10 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efdeabb1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efde4114530 con 0x7efde4109860 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde41045e0 0x7efde419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde41045e0 0x7efde419ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58296/0 (socket says 172.21.15.39:58296) 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 -- 172.21.15.39:0/1514060720 learned_addr learned my addr 172.21.15.39:0/1514060720 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde914e700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efde4109860 0x7efde419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde3fff700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efde4108f10 0x7efde419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 -- 172.21.15.39:0/1514060720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efde4109860 msgr2=0x7efde419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efde4109860 0x7efde419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 -- 172.21.15.39:0/1514060720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efde4108f10 msgr2=0x7efde419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efde4108f10 0x7efde419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 -- 172.21.15.39:0/1514060720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde419fc30 con 0x7efde41045e0 2024-05-04T03:08:35.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde914e700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efde4109860 0x7efde419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:08:35.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.621+0000 7efde894d700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde41045e0 0x7efde419ad60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efddc00ada0 tx=0x7efddc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:35.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.622+0000 7efde1ffb700 1 -- 172.21.15.39:0/1514060720 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc017020 con 0x7efde41045e0 2024-05-04T03:08:35.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.622+0000 7efde1ffb700 1 -- 172.21.15.39:0/1514060720 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efddc00ce10 con 0x7efde41045e0 2024-05-04T03:08:35.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.622+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde419ff20 con 0x7efde41045e0 2024-05-04T03:08:35.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.622+0000 7efde1ffb700 1 -- 172.21.15.39:0/1514060720 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0054e0 con 0x7efde41045e0 2024-05-04T03:08:35.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.622+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde41a7840 con 0x7efde41045e0 2024-05-04T03:08:35.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.623+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde40512c0 con 0x7efde41045e0 2024-05-04T03:08:35.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.625+0000 7efde1ffb700 1 -- 172.21.15.39:0/1514060720 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7efddc004c40 con 0x7efde41045e0 2024-05-04T03:08:35.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.625+0000 7efde1ffb700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdd00b2990 0x7efdd00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:35.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.625+0000 7efde3fff700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdd00b2990 0x7efdd00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:35.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.625+0000 7efde1ffb700 1 -- 172.21.15.39:0/1514060720 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efddc0e2110 con 0x7efde41045e0 2024-05-04T03:08:35.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.626+0000 7efde3fff700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdd00b2990 0x7efdd00b4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efdcc006fd0 tx=0x7efdcc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:35.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.629+0000 7efde1ffb700 1 -- 172.21.15.39:0/1514060720 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efddc021020 con 0x7efde41045e0 2024-05-04T03:08:35.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.909+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7efde402e5f0 con 0x7efde41045e0 2024-05-04T03:08:35.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.916+0000 7efde1ffb700 1 -- 172.21.15.39:0/1514060720 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288644 (secure 0 0 0) 0x7efddc0a9d70 con 0x7efde41045e0 2024-05-04T03:08:35.921 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:08:35.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdd00b2990 msgr2=0x7efdd00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:35.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdd00b2990 0x7efdd00b4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efdcc006fd0 tx=0x7efdcc008040 comp rx=0 tx=0).stop 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde41045e0 msgr2=0x7efde419ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde41045e0 0x7efde419ad60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efddc00ada0 tx=0x7efddc00c330 comp rx=0 tx=0).stop 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 shutdown_connections 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdd00b2990 0x7efdd00b4ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde41045e0 0x7efde419ad60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efde4109860 0x7efde419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 --2- 172.21.15.39:0/1514060720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efde4108f10 0x7efde419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 >> 172.21.15.39:0/1514060720 conn(0x7efde40ffe70 msgr2=0x7efde41086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 shutdown_connections 2024-05-04T03:08:35.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:35.920+0000 7efdeabb1700 1 -- 172.21.15.39:0/1514060720 wait complete. 2024-05-04T03:08:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:36 smithi039 ceph-mon[29715]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-04T03:08:36.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:36 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1514060720' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-04T03:08:36.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:36 smithi066 ceph-mon[33264]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-04T03:08:36.357 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:36 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1514060720' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-04T03:08:36.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:36 smithi167 ceph-mon[33614]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-04T03:08:36.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:36 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1514060720' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-04T03:08:36.539 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":2627629846},{"type":"v1","addr":"172.21.15.39:6801","nonce":2627629846}]},"active_addr":"172.21.15.39:6801/2627629846","active_change":"2024-05-04T03:00:14.875449+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.39:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":573268839}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":550966512}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":4171673484}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":589610496}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":2913476061}]}]} 2024-05-04T03:08:36.545 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-04T03:08:36.545 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-04T03:08:36.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:08:36.801 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:38.290 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:38 smithi039 ceph-mon[29715]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-04T03:08:38.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:38 smithi066 ceph-mon[33264]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-04T03:08:38.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:38 smithi167 ceph-mon[33614]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-04T03:08:38.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 -- 172.21.15.39:0/1747252967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9481045d0 msgr2=0x7fb9481049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 --2- 172.21.15.39:0/1747252967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9481045d0 0x7fb9481049e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb94000b0d0 tx=0x7fb94000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 -- 172.21.15.39:0/1747252967 shutdown_connections 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 --2- 172.21.15.39:0/1747252967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9481045d0 0x7fb9481049e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 --2- 172.21.15.39:0/1747252967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb948109850 0x7fb9481111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 --2- 172.21.15.39:0/1747252967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb948108f00 0x7fb948109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 -- 172.21.15.39:0/1747252967 >> 172.21.15.39:0/1747252967 conn(0x7fb9480ffe80 msgr2=0x7fb9481022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 -- 172.21.15.39:0/1747252967 shutdown_connections 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.560+0000 7fb9504d4700 1 -- 172.21.15.39:0/1747252967 wait complete. 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.561+0000 7fb9504d4700 1 Processor -- start 2024-05-04T03:08:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.561+0000 7fb9504d4700 1 -- start start 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb9504d4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9481045d0 0x7fb94819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb9504d4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb948108f00 0x7fb94819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb9504d4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb948109850 0x7fb94819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb9504d4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9481143f0 con 0x7fb948109850 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb9504d4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb948114270 con 0x7fb948108f00 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb9504d4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb948114570 con 0x7fb9481045d0 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb948109850 0x7fb94819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb948109850 0x7fb94819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56062/0 (socket says 172.21.15.39:56062) 2024-05-04T03:08:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 -- 172.21.15.39:0/479472110 learned_addr learned my addr 172.21.15.39:0/479472110 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94e270700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9481045d0 0x7fb94819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94da6f700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb948108f00 0x7fb94819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 -- 172.21.15.39:0/479472110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9481045d0 msgr2=0x7fb94819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9481045d0 0x7fb94819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 -- 172.21.15.39:0/479472110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb948108f00 msgr2=0x7fb94819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb948108f00 0x7fb94819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 -- 172.21.15.39:0/479472110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94819fcb0 con 0x7fb948109850 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb94ea71700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb948109850 0x7fb94819f610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb93800d980 tx=0x7fb93800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.562+0000 7fb9477fe700 1 -- 172.21.15.39:0/479472110 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938017070 con 0x7fb948109850 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.563+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94819ffa0 con 0x7fb948109850 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.563+0000 7fb9477fe700 1 -- 172.21.15.39:0/479472110 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb938009dd0 con 0x7fb948109850 2024-05-04T03:08:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.563+0000 7fb9477fe700 1 -- 172.21.15.39:0/479472110 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938005260 con 0x7fb948109850 2024-05-04T03:08:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.563+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9481a78c0 con 0x7fb948109850 2024-05-04T03:08:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.563+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9480512c0 con 0x7fb948109850 2024-05-04T03:08:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.565+0000 7fb9477fe700 1 -- 172.21.15.39:0/479472110 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb938009940 con 0x7fb948109850 2024-05-04T03:08:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.566+0000 7fb9477fe700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9300b2910 0x7fb9300b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.566+0000 7fb94e270700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9300b2910 0x7fb9300b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:38.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.566+0000 7fb9477fe700 1 -- 172.21.15.39:0/479472110 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb9380e27d0 con 0x7fb948109850 2024-05-04T03:08:38.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.566+0000 7fb94e270700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9300b2910 0x7fb9300b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb940009900 tx=0x7fb94000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.569+0000 7fb9477fe700 1 -- 172.21.15.39:0/479472110 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb938020020 con 0x7fb948109850 2024-05-04T03:08:38.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.787+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb94802e5f0 con 0x7fb948109850 2024-05-04T03:08:38.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.789+0000 7fb9477fe700 1 -- 172.21.15.39:0/479472110 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22541 (secure 0 0 0) 0x7fb9380aa3a0 con 0x7fb948109850 2024-05-04T03:08:38.790 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:08:38.790 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:08:16.237923+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9300b2910 msgr2=0x7fb9300b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9300b2910 0x7fb9300b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb940009900 tx=0x7fb94000b040 comp rx=0 tx=0).stop 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb948109850 msgr2=0x7fb94819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb948109850 0x7fb94819f610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb93800d980 tx=0x7fb93800dd40 comp rx=0 tx=0).stop 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 shutdown_connections 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9300b2910 0x7fb9300b4e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb948109850 0x7fb94819f610 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9481045d0 0x7fb94819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 --2- 172.21.15.39:0/479472110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb948108f00 0x7fb94819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 >> 172.21.15.39:0/479472110 conn(0x7fb9480ffe80 msgr2=0x7fb9481022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 shutdown_connections 2024-05-04T03:08:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:38.791+0000 7fb9504d4700 1 -- 172.21.15.39:0/479472110 wait complete. 2024-05-04T03:08:39.317 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:39 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/479472110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:39.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:39 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/479472110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:39.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:39 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/479472110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:39.523 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-04T03:08:39.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:08:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:40.164 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:40 smithi039 ceph-mon[29715]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:08:40.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:40 smithi167 ceph-mon[33614]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:08:40.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:40 smithi066 ceph-mon[33264]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:08:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 -- 172.21.15.39:0/699324051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc1029c0 msgr2=0x7f6cfc102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 --2- 172.21.15.39:0/699324051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc1029c0 0x7f6cfc102dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009990 tx=0x7f6cf0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 -- 172.21.15.39:0/699324051 shutdown_connections 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 --2- 172.21.15.39:0/699324051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cfc103310 0x7f6cfc10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 --2- 172.21.15.39:0/699324051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cfc1089f0 0x7f6cfc108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 --2- 172.21.15.39:0/699324051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc1029c0 0x7f6cfc102dd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 -- 172.21.15.39:0/699324051 >> 172.21.15.39:0/699324051 conn(0x7f6cfc0fe520 msgr2=0x7f6cfc100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 -- 172.21.15.39:0/699324051 shutdown_connections 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.536+0000 7f6d02612700 1 -- 172.21.15.39:0/699324051 wait complete. 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.537+0000 7f6d02612700 1 Processor -- start 2024-05-04T03:08:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.537+0000 7f6d02612700 1 -- start start 2024-05-04T03:08:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6d02612700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cfc1029c0 0x7f6cfc199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6d02612700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc103310 0x7f6cfc19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6d02612700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cfc1089f0 0x7f6cfc19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6d02612700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc112af0 con 0x7f6cfc103310 2024-05-04T03:08:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6d02612700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc112970 con 0x7f6cfc1089f0 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6d02612700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc112c70 con 0x7f6cfc1029c0 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc103310 0x7f6cfc19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc103310 0x7f6cfc19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56070/0 (socket says 172.21.15.39:56070) 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 -- 172.21.15.39:0/2354853770 learned_addr learned my addr 172.21.15.39:0/2354853770 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfbfff700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cfc1029c0 0x7f6cfc199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6d00baf700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cfc1089f0 0x7f6cfc19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 -- 172.21.15.39:0/2354853770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cfc1029c0 msgr2=0x7f6cfc199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cfc1029c0 0x7f6cfc199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 -- 172.21.15.39:0/2354853770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cfc1089f0 msgr2=0x7f6cfc19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cfc1089f0 0x7f6cfc19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 -- 172.21.15.39:0/2354853770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc1a9230 con 0x7f6cfc103310 2024-05-04T03:08:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6d00baf700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cfc1089f0 0x7f6cfc19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:08:41.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cfb7fe700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc103310 0x7f6cfc19a070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:41.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.538+0000 7f6cf97fa700 1 -- 172.21.15.39:0/2354853770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0027070 con 0x7f6cfc103310 2024-05-04T03:08:41.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.539+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc1a94c0 con 0x7f6cfc103310 2024-05-04T03:08:41.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.539+0000 7f6cf97fa700 1 -- 172.21.15.39:0/2354853770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cf000fc00 con 0x7f6cfc103310 2024-05-04T03:08:41.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.539+0000 7f6cf97fa700 1 -- 172.21.15.39:0/2354853770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0022cd0 con 0x7f6cfc103310 2024-05-04T03:08:41.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.539+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc1a9970 con 0x7f6cfc103310 2024-05-04T03:08:41.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.540+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc104100 con 0x7f6cfc103310 2024-05-04T03:08:41.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.541+0000 7f6cf97fa700 1 -- 172.21.15.39:0/2354853770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6cf002a070 con 0x7f6cfc103310 2024-05-04T03:08:41.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.542+0000 7f6cf97fa700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ce40b2910 0x7f6ce40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:41.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.542+0000 7f6cfbfff700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ce40b2910 0x7f6ce40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:41.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.542+0000 7f6cf97fa700 1 -- 172.21.15.39:0/2354853770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6cf00e32c0 con 0x7f6cfc103310 2024-05-04T03:08:41.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.542+0000 7f6cfbfff700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ce40b2910 0x7f6ce40b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc103b00 tx=0x7f6cec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.547+0000 7f6cf97fa700 1 -- 172.21.15.39:0/2354853770 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6cf005fdf0 con 0x7f6cfc103310 2024-05-04T03:08:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.765+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cfc1a9bd0 con 0x7f6cfc103310 2024-05-04T03:08:41.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.767+0000 7f6cf97fa700 1 -- 172.21.15.39:0/2354853770 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22541 (secure 0 0 0) 0x7f6cf0013070 con 0x7f6cfc103310 2024-05-04T03:08:41.768 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:08:41.768 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:08:16.237923+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ce40b2910 msgr2=0x7f6ce40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ce40b2910 0x7f6ce40b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc103b00 tx=0x7f6cec009190 comp rx=0 tx=0).stop 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc103310 msgr2=0x7f6cfc19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc103310 0x7f6cfc19a070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0000c00 tx=0x7f6cf00057c0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 shutdown_connections 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ce40b2910 0x7f6ce40b4e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cfc1089f0 0x7f6cfc19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cfc1029c0 0x7f6cfc199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 --2- 172.21.15.39:0/2354853770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cfc103310 0x7f6cfc19a070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 >> 172.21.15.39:0/2354853770 conn(0x7f6cfc0fe520 msgr2=0x7f6cfc104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:41.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 shutdown_connections 2024-05-04T03:08:41.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:41.769+0000 7f6d02612700 1 -- 172.21.15.39:0/2354853770 wait complete. 2024-05-04T03:08:42.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:42 smithi039 ceph-mon[29715]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:08:42.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:42 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2354853770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:42.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-04T03:08:42.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-04T03:08:42.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-04T03:08:42.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-04T03:08:42.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-04T03:08:42.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-04T03:08:42.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-04T03:08:42.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-04T03:08:42.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-04T03:08:42.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-04T03:08:42.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-04T03:08:42.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-04T03:08:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:42 smithi167 ceph-mon[33614]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:08:42.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:42 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2354853770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:42.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:42 smithi066 ceph-mon[33264]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:08:42.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:42 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2354853770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:08:43.073 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:44.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:44 smithi039 ceph-mon[29715]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:08:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:44 smithi167 ceph-mon[33614]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:08:44.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:44 smithi066 ceph-mon[33264]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.869+0000 7f5afc94c700 1 -- 172.21.15.39:0/1840385129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af81029c0 msgr2=0x7f5af8102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.869+0000 7f5afc94c700 1 --2- 172.21.15.39:0/1840385129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af81029c0 0x7f5af8102dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009990 tx=0x7f5aec009ca0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.870+0000 7f5afc94c700 1 -- 172.21.15.39:0/1840385129 shutdown_connections 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.870+0000 7f5afc94c700 1 --2- 172.21.15.39:0/1840385129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af8103310 0x7f5af810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.870+0000 7f5afc94c700 1 --2- 172.21.15.39:0/1840385129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af81089f0 0x7f5af8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.870+0000 7f5afc94c700 1 --2- 172.21.15.39:0/1840385129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af81029c0 0x7f5af8102dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.870+0000 7f5afc94c700 1 -- 172.21.15.39:0/1840385129 >> 172.21.15.39:0/1840385129 conn(0x7f5af80fe6f0 msgr2=0x7f5af8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.870+0000 7f5afc94c700 1 -- 172.21.15.39:0/1840385129 shutdown_connections 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.870+0000 7f5afc94c700 1 -- 172.21.15.39:0/1840385129 wait complete. 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.870+0000 7f5afc94c700 1 Processor -- start 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5afc94c700 1 -- start start 2024-05-04T03:08:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5afc94c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af81029c0 0x7f5af819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5afc94c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af8103310 0x7f5af819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5afc94c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af81089f0 0x7f5af819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5afc94c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81129a0 con 0x7f5af8103310 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5afc94c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5af8112820 con 0x7f5af81089f0 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5afc94c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5af8112b20 con 0x7f5af81029c0 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af8103310 0x7f5af819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af8103310 0x7f5af819c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54632/0 (socket says 172.21.15.39:54632) 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af5d9b700 1 -- 172.21.15.39:0/2349726847 learned_addr learned my addr 172.21.15.39:0/2349726847 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af659c700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af81029c0 0x7f5af819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af6d9d700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af81089f0 0x7f5af819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af5d9b700 1 -- 172.21.15.39:0/2349726847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af81029c0 msgr2=0x7f5af819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af5d9b700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af81029c0 0x7f5af819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af5d9b700 1 -- 172.21.15.39:0/2349726847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af81089f0 msgr2=0x7f5af819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af5d9b700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5af81089f0 0x7f5af819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.871+0000 7f5af5d9b700 1 -- 172.21.15.39:0/2349726847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af81a6ed0 con 0x7f5af8103310 2024-05-04T03:08:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.872+0000 7f5af5d9b700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af8103310 0x7f5af819c150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5aec000c00 tx=0x7f5aec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.872+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec027070 con 0x7f5af8103310 2024-05-04T03:08:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.872+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5aec00fb80 con 0x7f5af8103310 2024-05-04T03:08:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.872+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af81a7160 con 0x7f5af8103310 2024-05-04T03:08:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.872+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec022d00 con 0x7f5af8103310 2024-05-04T03:08:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.872+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af81a7610 con 0x7f5af8103310 2024-05-04T03:08:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.873+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5af81014f0 con 0x7f5af8103310 2024-05-04T03:08:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.874+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5aec02a070 con 0x7f5af8103310 2024-05-04T03:08:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.875+0000 7f5ae77fe700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5ae00b2990 0x7f5ae00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.875+0000 7f5af659c700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5ae00b2990 0x7f5ae00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.875+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5aec0e3790 con 0x7f5af8103310 2024-05-04T03:08:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.875+0000 7f5ae77fe700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] conn(0x7f5ae00bc520 0x7f5ae00bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:45.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.875+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 --> [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5aec00b0b0 con 0x7f5ae00bc520 2024-05-04T03:08:45.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.875+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f5aec0e3be0 con 0x7f5af8103310 2024-05-04T03:08:45.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.876+0000 7f5af659c700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5ae00b2990 0x7f5ae00b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5af8103b00 tx=0x7f5ae80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:45.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.876+0000 7f5af6d9d700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] conn(0x7f5ae00bc520 0x7f5ae00bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:45.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.876+0000 7f5af6d9d700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] conn(0x7f5ae00bc520 0x7f5ae00bea10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:45.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.877+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 <== osd.4 v2:172.21.15.66:6808/1509400752 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5aec00b0b0 con 0x7f5ae00bc520 2024-05-04T03:08:45.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.894+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 --> [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5af8104100 con 0x7f5ae00bc520 2024-05-04T03:08:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.894+0000 7f5ae77fe700 1 -- 172.21.15.39:0/2349726847 <== osd.4 v2:172.21.15.66:6808/1509400752 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5af8104100 con 0x7f5ae00bc520 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 >> [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] conn(0x7f5ae00bc520 msgr2=0x7f5ae00bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] conn(0x7f5ae00bc520 0x7f5ae00bea10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5ae00b2990 msgr2=0x7f5ae00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5ae00b2990 0x7f5ae00b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5af8103b00 tx=0x7f5ae80073d0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af8103310 msgr2=0x7f5af819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 --2- 172.21.15.39:0/2349726847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af8103310 0x7f5af819c150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5aec000c00 tx=0x7f5aec0047b0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5af6d9d700 1 -- 172.21.15.39:0/2349726847 reap_dead start 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 shutdown_connections 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 >> 172.21.15.39:0/2349726847 conn(0x7f5af80fe6f0 msgr2=0x7f5af8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 shutdown_connections 2024-05-04T03:08:45.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:45.895+0000 7f5afc94c700 1 -- 172.21.15.39:0/2349726847 wait complete. 2024-05-04T03:08:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:46.335 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:46.335 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:46.339 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:46.359 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:46 smithi039 ceph-mon[29715]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:08:46.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:46 smithi167 ceph-mon[33614]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:08:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:46 smithi066 ceph-mon[33264]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:08:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.871+0000 7f309aabf700 1 -- 172.21.15.39:0/3832530773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30941054e0 msgr2=0x7f30941058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:47.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.871+0000 7f309aabf700 1 --2- 172.21.15.39:0/3832530773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30941054e0 0x7f30941058f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3088009990 tx=0x7f3088009ca0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.871+0000 7f309aabf700 1 -- 172.21.15.39:0/3832530773 shutdown_connections 2024-05-04T03:08:47.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.871+0000 7f309aabf700 1 --2- 172.21.15.39:0/3832530773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30941054e0 0x7f30941058f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.871+0000 7f309aabf700 1 --2- 172.21.15.39:0/3832530773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f309410e910 0x7f3094110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.871+0000 7f309aabf700 1 --2- 172.21.15.39:0/3832530773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3094105e30 0x7f309410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.871+0000 7f309aabf700 1 -- 172.21.15.39:0/3832530773 >> 172.21.15.39:0/3832530773 conn(0x7f30940fef30 msgr2=0x7f3094101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.872+0000 7f309aabf700 1 -- 172.21.15.39:0/3832530773 shutdown_connections 2024-05-04T03:08:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.872+0000 7f309aabf700 1 -- 172.21.15.39:0/3832530773 wait complete. 2024-05-04T03:08:47.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.873+0000 7f309aabf700 1 Processor -- start 2024-05-04T03:08:47.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.873+0000 7f309aabf700 1 -- start start 2024-05-04T03:08:47.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.873+0000 7f309aabf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30941054e0 0x7f309419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309aabf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3094105e30 0x7f309419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309aabf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309410e910 0x7f309419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309aabf700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3094114330 con 0x7f309410e910 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309aabf700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30941141b0 con 0x7f3094105e30 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309aabf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30941144b0 con 0x7f30941054e0 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309905c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309410e910 0x7f309419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309905c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309410e910 0x7f309419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54640/0 (socket says 172.21.15.39:54640) 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309905c700 1 -- 172.21.15.39:0/2622002209 learned_addr learned my addr 172.21.15.39:0/2622002209 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309885b700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30941054e0 0x7f309419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f3093fff700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3094105e30 0x7f309419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:47.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309905c700 1 -- 172.21.15.39:0/2622002209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30941054e0 msgr2=0x7f309419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309905c700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30941054e0 0x7f309419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309905c700 1 -- 172.21.15.39:0/2622002209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3094105e30 msgr2=0x7f309419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309905c700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3094105e30 0x7f309419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309905c700 1 -- 172.21.15.39:0/2622002209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309419fc20 con 0x7f309410e910 2024-05-04T03:08:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.874+0000 7f309885b700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30941054e0 0x7f309419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:08:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.875+0000 7f309905c700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309410e910 0x7f309419f580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f308c007ee0 tx=0x7f308c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.875+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00dd20 con 0x7f309410e910 2024-05-04T03:08:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.875+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309419ff10 con 0x7f309410e910 2024-05-04T03:08:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.875+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f308c004020 con 0x7f309410e910 2024-05-04T03:08:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.875+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c015660 con 0x7f309410e910 2024-05-04T03:08:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.875+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3094103100 con 0x7f309410e910 2024-05-04T03:08:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.876+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3094063f50 con 0x7f309410e910 2024-05-04T03:08:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.878+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f308c0041d0 con 0x7f309410e910 2024-05-04T03:08:47.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.878+0000 7f3091ffb700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f30840b2910 0x7f30840b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:47.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.879+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f308c0e2830 con 0x7f309410e910 2024-05-04T03:08:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.879+0000 7f309885b700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f30840b2910 0x7f30840b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.879+0000 7f3091ffb700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] conn(0x7f30840bc4a0 0x7f30840be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.879+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 --> [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30840bf060 con 0x7f30840bc4a0 2024-05-04T03:08:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.879+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f308c0e2c80 con 0x7f309410e910 2024-05-04T03:08:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.879+0000 7f3093fff700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] conn(0x7f30840bc4a0 0x7f30840be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.879+0000 7f309885b700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f30840b2910 0x7f30840b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f308800b2e0 tx=0x7f3088005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.879+0000 7f3093fff700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] conn(0x7f30840bc4a0 0x7f30840be990 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:47.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.880+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 <== osd.6 v2:172.21.15.39:6818/519893434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30840bf060 con 0x7f30840bc4a0 2024-05-04T03:08:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.898+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 --> [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3094064350 con 0x7f30840bc4a0 2024-05-04T03:08:47.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f3091ffb700 1 -- 172.21.15.39:0/2622002209 <== osd.6 v2:172.21.15.39:6818/519893434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3094064350 con 0x7f30840bc4a0 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] conn(0x7f30840bc4a0 msgr2=0x7f30840be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] conn(0x7f30840bc4a0 0x7f30840be990 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f30840b2910 msgr2=0x7f30840b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f30840b2910 0x7f30840b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f308800b2e0 tx=0x7f3088005fb0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309410e910 msgr2=0x7f309419f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 --2- 172.21.15.39:0/2622002209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309410e910 0x7f309419f580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f308c007ee0 tx=0x7f308c00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309905c700 1 -- 172.21.15.39:0/2622002209 reap_dead start 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 shutdown_connections 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 >> 172.21.15.39:0/2622002209 conn(0x7f30940fef30 msgr2=0x7f3094100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 shutdown_connections 2024-05-04T03:08:47.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:47.899+0000 7f309aabf700 1 -- 172.21.15.39:0/2622002209 wait complete. 2024-05-04T03:08:48.358 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:48 smithi039 ceph-mon[29715]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:08:48.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:48 smithi167 ceph-mon[33614]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:08:48.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:48 smithi066 ceph-mon[33264]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:08:48.854 INFO:teuthology.orchestra.run.smithi039.stdout:133143986242 2024-05-04T03:08:48.854 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-04T03:08:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:50.346 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:50.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:50 smithi167 ceph-mon[33614]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-04T03:08:50.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:50 smithi039 ceph-mon[29715]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-04T03:08:50.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:50 smithi066 ceph-mon[33264]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-05-04T03:08:52.045 INFO:teuthology.orchestra.run.smithi039.stdout:180388626489 2024-05-04T03:08:52.046 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-04T03:08:52.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:52 smithi167 ceph-mon[33614]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-04T03:08:52.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:52 smithi039 ceph-mon[29715]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-04T03:08:52.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:52 smithi066 ceph-mon[33264]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-04T03:08:54.238 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:54 smithi039 ceph-mon[29715]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:08:54.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:54 smithi167 ceph-mon[33614]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:08:54.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:54 smithi066 ceph-mon[33264]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:08:54.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.705+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3812520953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec107df0 msgr2=0x7fc6ec108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.705+0000 7fc6f3dd1700 1 --2- 172.21.15.39:0/3812520953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec107df0 0x7fc6ec108200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0099c0 tx=0x7fc6dc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:08:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.706+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3812520953 shutdown_connections 2024-05-04T03:08:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.706+0000 7fc6f3dd1700 1 --2- 172.21.15.39:0/3812520953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6ec108740 0x7fc6ec1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.706+0000 7fc6f3dd1700 1 --2- 172.21.15.39:0/3812520953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec1034c0 0x7fc6ec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.706+0000 7fc6f3dd1700 1 --2- 172.21.15.39:0/3812520953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec107df0 0x7fc6ec108200 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.706+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3812520953 >> 172.21.15.39:0/3812520953 conn(0x7fc6ec0fedc0 msgr2=0x7fc6ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.706+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3812520953 shutdown_connections 2024-05-04T03:08:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.706+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3812520953 wait complete. 2024-05-04T03:08:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f3dd1700 1 Processor -- start 2024-05-04T03:08:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f3dd1700 1 -- start start 2024-05-04T03:08:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f3dd1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec1034c0 0x7fc6ec1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f3dd1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6ec107df0 0x7fc6ec1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f3dd1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108740 0x7fc6ec19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f3dd1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec113200 con 0x7fc6ec1034c0 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f3dd1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec113080 con 0x7fc6ec108740 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f3dd1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec113380 con 0x7fc6ec107df0 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f1b6d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec1034c0 0x7fc6ec1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f1b6d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec1034c0 0x7fc6ec1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54660/0 (socket says 172.21.15.39:54660) 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.707+0000 7fc6f1b6d700 1 -- 172.21.15.39:0/3769309667 learned_addr learned my addr 172.21.15.39:0/3769309667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f136c700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6ec107df0 0x7fc6ec1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f236e700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108740 0x7fc6ec19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f1b6d700 1 -- 172.21.15.39:0/3769309667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6ec107df0 msgr2=0x7fc6ec1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f1b6d700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6ec107df0 0x7fc6ec1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f1b6d700 1 -- 172.21.15.39:0/3769309667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108740 msgr2=0x7fc6ec19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f1b6d700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6ec108740 0x7fc6ec19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:54.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f1b6d700 1 -- 172.21.15.39:0/3769309667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec19b260 con 0x7fc6ec1034c0 2024-05-04T03:08:54.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f136c700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6ec107df0 0x7fc6ec1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:08:54.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f1b6d700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec1034c0 0x7fc6ec1a09a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800c920 tx=0x7fc6e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:54.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8007b00 con 0x7fc6ec1034c0 2024-05-04T03:08:54.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6e8004cc0 con 0x7fc6ec1034c0 2024-05-04T03:08:54.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec19b550 con 0x7fc6ec1034c0 2024-05-04T03:08:54.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8005440 con 0x7fc6ec1034c0 2024-05-04T03:08:54.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.708+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec0ff1d0 con 0x7fc6ec1034c0 2024-05-04T03:08:54.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.709+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6ec063f10 con 0x7fc6ec1034c0 2024-05-04T03:08:54.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.712+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc6e8004750 con 0x7fc6ec1034c0 2024-05-04T03:08:54.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.713+0000 7fc6e2ffd700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc6d80b2910 0x7fc6d80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:54.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.713+0000 7fc6f136c700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc6d80b2910 0x7fc6d80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:54.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.713+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc6e80e24a0 con 0x7fc6ec1034c0 2024-05-04T03:08:54.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.713+0000 7fc6e2ffd700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] conn(0x7fc6d80bc4a0 0x7fc6d80be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:54.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.713+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 --> [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6e8012ec0 con 0x7fc6d80bc4a0 2024-05-04T03:08:54.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.713+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fc6e80e28f0 con 0x7fc6ec1034c0 2024-05-04T03:08:54.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.713+0000 7fc6f236e700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] conn(0x7fc6d80bc4a0 0x7fc6d80be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:54.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.714+0000 7fc6f136c700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc6d80b2910 0x7fc6d80b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec109b30 tx=0x7fc6dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:54.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.714+0000 7fc6f236e700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] conn(0x7fc6d80bc4a0 0x7fc6d80be990 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:54.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.715+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 <== osd.0 v2:172.21.15.39:6802/837850170 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6e8012ec0 con 0x7fc6d80bc4a0 2024-05-04T03:08:54.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.732+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 --> [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6ec064310 con 0x7fc6d80bc4a0 2024-05-04T03:08:54.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6e2ffd700 1 -- 172.21.15.39:0/3769309667 <== osd.0 v2:172.21.15.39:6802/837850170 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6ec064310 con 0x7fc6d80bc4a0 2024-05-04T03:08:54.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] conn(0x7fc6d80bc4a0 msgr2=0x7fc6d80be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:54.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] conn(0x7fc6d80bc4a0 0x7fc6d80be990 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:54.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc6d80b2910 msgr2=0x7fc6d80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:54.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc6d80b2910 0x7fc6d80b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec109b30 tx=0x7fc6dc005fd0 comp rx=0 tx=0).stop 2024-05-04T03:08:54.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec1034c0 msgr2=0x7fc6ec1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:54.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 --2- 172.21.15.39:0/3769309667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6ec1034c0 0x7fc6ec1a09a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800c920 tx=0x7fc6e800cc30 comp rx=0 tx=0).stop 2024-05-04T03:08:54.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f236e700 1 -- 172.21.15.39:0/3769309667 reap_dead start 2024-05-04T03:08:54.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 shutdown_connections 2024-05-04T03:08:54.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 >> 172.21.15.39:0/3769309667 conn(0x7fc6ec0fedc0 msgr2=0x7fc6ec06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:54.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.733+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 shutdown_connections 2024-05-04T03:08:54.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:54.734+0000 7fc6f3dd1700 1 -- 172.21.15.39:0/3769309667 wait complete. 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.127+0000 7fbf0c62a700 1 -- 172.21.15.39:0/960506723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105480 msgr2=0x7fbf04105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.127+0000 7fbf0c62a700 1 --2- 172.21.15.39:0/960506723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105480 0x7fbf04105890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009990 tx=0x7fbf00009ca0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 -- 172.21.15.39:0/960506723 shutdown_connections 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 --2- 172.21.15.39:0/960506723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105480 0x7fbf04105890 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 --2- 172.21.15.39:0/960506723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf0410e940 0x7fbf04110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 --2- 172.21.15.39:0/960506723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf04105e60 0x7fbf0410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 -- 172.21.15.39:0/960506723 >> 172.21.15.39:0/960506723 conn(0x7fbf040fef50 msgr2=0x7fbf04101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 -- 172.21.15.39:0/960506723 shutdown_connections 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 -- 172.21.15.39:0/960506723 wait complete. 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 Processor -- start 2024-05-04T03:08:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.128+0000 7fbf0c62a700 1 -- start start 2024-05-04T03:08:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0c62a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf04105480 0x7fbf04103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0c62a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105e60 0x7fbf041036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0c62a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0410e940 0x7fbf04103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0c62a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041143c0 con 0x7fbf04105e60 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0c62a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04114240 con 0x7fbf0410e940 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0c62a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04114540 con 0x7fbf04105480 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105e60 0x7fbf041036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105e60 0x7fbf041036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60858/0 (socket says 172.21.15.39:60858) 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 -- 172.21.15.39:0/862610147 learned_addr learned my addr 172.21.15.39:0/862610147 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0a3c6700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf04105480 0x7fbf04103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 -- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf04105480 msgr2=0x7fbf04103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf04105480 0x7fbf04103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0abc7700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0410e940 0x7fbf04103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 -- 172.21.15.39:0/862610147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0410e940 msgr2=0x7fbf04103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0410e940 0x7fbf04103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 -- 172.21.15.39:0/862610147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf041a9150 con 0x7fbf04105e60 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0a3c6700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf04105480 0x7fbf04103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf0abc7700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf0410e940 0x7fbf04103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbf09bc5700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105e60 0x7fbf041036b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbef400d980 tx=0x7fbef400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.129+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4017070 con 0x7fbf04105e60 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.130+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbef4010460 con 0x7fbf04105e60 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.130+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf041a9440 con 0x7fbf04105e60 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.130+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef400de60 con 0x7fbf04105e60 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.130+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a9980 con 0x7fbf04105e60 2024-05-04T03:08:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.130+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf04063ed0 con 0x7fbf04105e60 2024-05-04T03:08:56.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.132+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbef4010610 con 0x7fbf04105e60 2024-05-04T03:08:56.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.133+0000 7fbefb7fe700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbef00b2910 0x7fbef00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.133+0000 7fbf0a3c6700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbef00b2910 0x7fbef00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.133+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbef40e2300 con 0x7fbf04105e60 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.133+0000 7fbefb7fe700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] conn(0x7fbef00bc4a0 0x7fbef00be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.133+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 --> [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbef4016ec0 con 0x7fbef00bc4a0 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.133+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbef40e2750 con 0x7fbf04105e60 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.134+0000 7fbf0a3c6700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbef00b2910 0x7fbef00b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000b5c0 tx=0x7fbf00005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.134+0000 7fbf0abc7700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] conn(0x7fbef00bc4a0 0x7fbef00be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.134+0000 7fbf0abc7700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] conn(0x7fbef00bc4a0 0x7fbef00be990 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.135+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 <== osd.8 v2:172.21.15.167:6816/948600298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbef4016ec0 con 0x7fbef00bc4a0 2024-05-04T03:08:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.152+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 --> [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf04064350 con 0x7fbef00bc4a0 2024-05-04T03:08:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbefb7fe700 1 -- 172.21.15.39:0/862610147 <== osd.8 v2:172.21.15.167:6816/948600298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbf04064350 con 0x7fbef00bc4a0 2024-05-04T03:08:56.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] conn(0x7fbef00bc4a0 msgr2=0x7fbef00be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] conn(0x7fbef00bc4a0 0x7fbef00be990 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbef00b2910 msgr2=0x7fbef00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbef00b2910 0x7fbef00b4e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000b5c0 tx=0x7fbf00005960 comp rx=0 tx=0).stop 2024-05-04T03:08:56.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105e60 msgr2=0x7fbf041036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 --2- 172.21.15.39:0/862610147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf04105e60 0x7fbf041036b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbef400d980 tx=0x7fbef400dd40 comp rx=0 tx=0).stop 2024-05-04T03:08:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0abc7700 1 -- 172.21.15.39:0/862610147 reap_dead start 2024-05-04T03:08:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 shutdown_connections 2024-05-04T03:08:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 >> 172.21.15.39:0/862610147 conn(0x7fbf040fef50 msgr2=0x7fbf040ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 shutdown_connections 2024-05-04T03:08:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.153+0000 7fbf0c62a700 1 -- 172.21.15.39:0/862610147 wait complete. 2024-05-04T03:08:56.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:56 smithi039 ceph-mon[29715]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:08:56.369 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:56 smithi066 ceph-mon[33264]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:08:56.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:56 smithi167 ceph-mon[33614]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:08:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.564+0000 7f8dd8702700 1 -- 172.21.15.39:0/457944490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 msgr2=0x7f8dd0103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.564+0000 7f8dd8702700 1 --2- 172.21.15.39:0/457944490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 0x7f8dd0103960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0099c0 tx=0x7f8dcc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 -- 172.21.15.39:0/457944490 shutdown_connections 2024-05-04T03:08:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 --2- 172.21.15.39:0/457944490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 0x7f8dd0103960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 --2- 172.21.15.39:0/457944490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd0108780 0x7f8dd01100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 --2- 172.21.15.39:0/457944490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dd0107e30 0x7f8dd0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 -- 172.21.15.39:0/457944490 >> 172.21.15.39:0/457944490 conn(0x7f8dd0078e00 msgr2=0x7f8dd0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 -- 172.21.15.39:0/457944490 shutdown_connections 2024-05-04T03:08:56.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 -- 172.21.15.39:0/457944490 wait complete. 2024-05-04T03:08:56.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 Processor -- start 2024-05-04T03:08:56.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.565+0000 7f8dd8702700 1 -- start start 2024-05-04T03:08:56.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd8702700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 0x7f8dd019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd8702700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd0107e30 0x7f8dd019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd8702700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dd0108780 0x7f8dd0199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd8702700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0113260 con 0x7f8dd0103550 2024-05-04T03:08:56.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd8702700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01130e0 con 0x7f8dd0107e30 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd8702700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01133e0 con 0x7f8dd0108780 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 0x7f8dd019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 0x7f8dd019f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60882/0 (socket says 172.21.15.39:60882) 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 -- 172.21.15.39:0/1245133223 learned_addr learned my addr 172.21.15.39:0/1245133223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd6c9f700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dd0108780 0x7f8dd0199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd5c9d700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd0107e30 0x7f8dd019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 -- 172.21.15.39:0/1245133223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dd0108780 msgr2=0x7f8dd0199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dd0108780 0x7f8dd0199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 -- 172.21.15.39:0/1245133223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd0107e30 msgr2=0x7f8dd019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd0107e30 0x7f8dd019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 -- 172.21.15.39:0/1245133223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd019a130 con 0x7f8dd0103550 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd5c9d700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8dd0107e30 0x7f8dd019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dd649e700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 0x7f8dd019f860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc000c00 tx=0x7f8dcc00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.566+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00f940 con 0x7f8dd0103550 2024-05-04T03:08:56.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.567+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dcc004830 con 0x7f8dd0103550 2024-05-04T03:08:56.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.567+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd019a3c0 con 0x7f8dd0103550 2024-05-04T03:08:56.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.567+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0054b0 con 0x7f8dd0103550 2024-05-04T03:08:56.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.567+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd019a890 con 0x7f8dd0103550 2024-05-04T03:08:56.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.567+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8dd0063f50 con 0x7f8dd0103550 2024-05-04T03:08:56.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.569+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8dcc029060 con 0x7f8dd0103550 2024-05-04T03:08:56.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.570+0000 7f8dc77fe700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8dbc0b2990 0x7f8dbc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.570+0000 7f8dd5c9d700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8dbc0b2990 0x7f8dbc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.570+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8dcc0e3530 con 0x7f8dd0103550 2024-05-04T03:08:56.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.570+0000 7f8dc77fe700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] conn(0x7f8dbc0bc520 0x7f8dbc0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.570+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 --> [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8dbc0bf0e0 con 0x7f8dbc0bc520 2024-05-04T03:08:56.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.570+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f8dcc0e3980 con 0x7f8dd0103550 2024-05-04T03:08:56.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.570+0000 7f8dd6c9f700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] conn(0x7f8dbc0bc520 0x7f8dbc0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.570+0000 7f8dd5c9d700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8dbc0b2990 0x7f8dbc0b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0005fd0 tx=0x7f8dc0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.571+0000 7f8dd6c9f700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] conn(0x7f8dbc0bc520 0x7f8dbc0bea10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.571+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 <== osd.9 v2:172.21.15.39:6826/1483154283 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8dbc0bf0e0 con 0x7f8dbc0bc520 2024-05-04T03:08:56.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 --> [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8dd0064350 con 0x7f8dbc0bc520 2024-05-04T03:08:56.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dc77fe700 1 -- 172.21.15.39:0/1245133223 <== osd.9 v2:172.21.15.39:6826/1483154283 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8dd0064350 con 0x7f8dbc0bc520 2024-05-04T03:08:56.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] conn(0x7f8dbc0bc520 msgr2=0x7f8dbc0bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd8702700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] conn(0x7f8dbc0bc520 0x7f8dbc0bea10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8dbc0b2990 msgr2=0x7f8dbc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd8702700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8dbc0b2990 0x7f8dbc0b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0005fd0 tx=0x7f8dc0005ee0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 msgr2=0x7f8dd019f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd8702700 1 --2- 172.21.15.39:0/1245133223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd0103550 0x7f8dd019f860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc000c00 tx=0x7f8dcc00f510 comp rx=0 tx=0).stop 2024-05-04T03:08:56.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd6c9f700 1 -- 172.21.15.39:0/1245133223 reap_dead start 2024-05-04T03:08:56.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.589+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 shutdown_connections 2024-05-04T03:08:56.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.590+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 >> 172.21.15.39:0/1245133223 conn(0x7f8dd0078e00 msgr2=0x7f8dd0101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:56.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.590+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 shutdown_connections 2024-05-04T03:08:56.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.590+0000 7f8dd8702700 1 -- 172.21.15.39:0/1245133223 wait complete. 2024-05-04T03:08:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 -- 172.21.15.39:0/3650602138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c1089f0 msgr2=0x7f4e7c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 --2- 172.21.15.39:0/3650602138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c1089f0 0x7f4e7c108e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009990 tx=0x7f4e6c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 -- 172.21.15.39:0/3650602138 shutdown_connections 2024-05-04T03:08:56.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 --2- 172.21.15.39:0/3650602138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c1089f0 0x7f4e7c108e00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 --2- 172.21.15.39:0/3650602138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c103310 0x7f4e7c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 --2- 172.21.15.39:0/3650602138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e7c1029c0 0x7f4e7c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 -- 172.21.15.39:0/3650602138 >> 172.21.15.39:0/3650602138 conn(0x7f4e7c0fe520 msgr2=0x7f4e7c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 -- 172.21.15.39:0/3650602138 shutdown_connections 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.887+0000 7f4e82250700 1 -- 172.21.15.39:0/3650602138 wait complete. 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e82250700 1 Processor -- start 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e82250700 1 -- start start 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e82250700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e7c1029c0 0x7f4e7c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e82250700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c103310 0x7f4e7c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e82250700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c1089f0 0x7f4e7c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e82250700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1129d0 con 0x7f4e7c103310 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e82250700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c112850 con 0x7f4e7c1029c0 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e82250700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c112b50 con 0x7f4e7c1089f0 2024-05-04T03:08:56.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e7affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c103310 0x7f4e7c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e7affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c103310 0x7f4e7c19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60900/0 (socket says 172.21.15.39:60900) 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e7affd700 1 -- 172.21.15.39:0/811123065 learned_addr learned my addr 172.21.15.39:0/811123065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.888+0000 7f4e7b7fe700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e7c1029c0 0x7f4e7c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7bfff700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c1089f0 0x7f4e7c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7affd700 1 -- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c1089f0 msgr2=0x7f4e7c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7affd700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c1089f0 0x7f4e7c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7affd700 1 -- 172.21.15.39:0/811123065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e7c1029c0 msgr2=0x7f4e7c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7affd700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e7c1029c0 0x7f4e7c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7affd700 1 -- 172.21.15.39:0/811123065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c19a130 con 0x7f4e7c103310 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7bfff700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e7c1089f0 0x7f4e7c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7b7fe700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e7c1029c0 0x7f4e7c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e7affd700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c103310 0x7f4e7c19fda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c8f0 tx=0x7f4e7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70007b00 con 0x7f4e7c103310 2024-05-04T03:08:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c19a420 con 0x7f4e7c103310 2024-05-04T03:08:56.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e70007ca0 con 0x7f4e7c103310 2024-05-04T03:08:56.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70018660 con 0x7f4e7c103310 2024-05-04T03:08:56.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.889+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1a66f0 con 0x7f4e7c103310 2024-05-04T03:08:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.891+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e7c107920 con 0x7f4e7c103310 2024-05-04T03:08:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.892+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4e7000f450 con 0x7f4e7c103310 2024-05-04T03:08:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.893+0000 7f4e78ff9700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4e640b2910 0x7f4e640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.893+0000 7f4e7b7fe700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4e640b2910 0x7f4e640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.893+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4e700e2630 con 0x7f4e7c103310 2024-05-04T03:08:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.893+0000 7f4e78ff9700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] conn(0x7f4e640bc4a0 0x7f4e640be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.893+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 --> [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e70015e70 con 0x7f4e640bc4a0 2024-05-04T03:08:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.893+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f4e700e2a80 con 0x7f4e7c103310 2024-05-04T03:08:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.894+0000 7f4e7b7fe700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4e640b2910 0x7f4e640b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0059d0 tx=0x7f4e6c005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.894+0000 7f4e7bfff700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] conn(0x7f4e640bc4a0 0x7f4e640be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.894+0000 7f4e7bfff700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] conn(0x7f4e640bc4a0 0x7f4e640be990 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.895+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 <== osd.5 v2:172.21.15.167:6808/3200709973 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e70015e70 con 0x7f4e640bc4a0 2024-05-04T03:08:56.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.912+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 --> [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e7c104100 con 0x7f4e640bc4a0 2024-05-04T03:08:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e78ff9700 1 -- 172.21.15.39:0/811123065 <== osd.5 v2:172.21.15.167:6808/3200709973 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4e7c104100 con 0x7f4e640bc4a0 2024-05-04T03:08:56.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] conn(0x7f4e640bc4a0 msgr2=0x7f4e640be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e82250700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] conn(0x7f4e640bc4a0 0x7f4e640be990 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4e640b2910 msgr2=0x7f4e640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e82250700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4e640b2910 0x7f4e640b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0059d0 tx=0x7f4e6c005940 comp rx=0 tx=0).stop 2024-05-04T03:08:56.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c103310 msgr2=0x7f4e7c19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:56.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e82250700 1 --2- 172.21.15.39:0/811123065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e7c103310 0x7f4e7c19fda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c8f0 tx=0x7f4e7000ccb0 comp rx=0 tx=0).stop 2024-05-04T03:08:56.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e7bfff700 1 -- 172.21.15.39:0/811123065 reap_dead start 2024-05-04T03:08:56.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 shutdown_connections 2024-05-04T03:08:56.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.913+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 >> 172.21.15.39:0/811123065 conn(0x7f4e7c0fe520 msgr2=0x7f4e7c106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:56.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.914+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 shutdown_connections 2024-05-04T03:08:56.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:56.914+0000 7f4e82250700 1 -- 172.21.15.39:0/811123065 wait complete. 2024-05-04T03:08:58.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:08:58 smithi039 ceph-mon[29715]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-04T03:08:58.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 -- 172.21.15.39:0/1521937807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78109890 msgr2=0x7fee781111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 --2- 172.21.15.39:0/1521937807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78109890 0x7fee781111f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fee7000b0d0 tx=0x7fee7000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 -- 172.21.15.39:0/1521937807 shutdown_connections 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 --2- 172.21.15.39:0/1521937807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78109890 0x7fee781111f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 --2- 172.21.15.39:0/1521937807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee78104610 0x7fee78104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 --2- 172.21.15.39:0/1521937807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee78108f40 0x7fee78109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 -- 172.21.15.39:0/1521937807 >> 172.21.15.39:0/1521937807 conn(0x7fee7807af70 msgr2=0x7fee7807b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 -- 172.21.15.39:0/1521937807 shutdown_connections 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.349+0000 7fee7f650700 1 -- 172.21.15.39:0/1521937807 wait complete. 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7f650700 1 Processor -- start 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7f650700 1 -- start start 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7f650700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee78104610 0x7fee781a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7f650700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78108f40 0x7fee781a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7f650700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee78109890 0x7fee7819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7f650700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee78114370 con 0x7fee78108f40 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7f650700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee781141f0 con 0x7fee78109890 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7f650700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee781144f0 con 0x7fee78104610 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7cbeb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78108f40 0x7fee781a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7cbeb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78108f40 0x7fee781a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60924/0 (socket says 172.21.15.39:60924) 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7cbeb700 1 -- 172.21.15.39:0/2616900804 learned_addr learned my addr 172.21.15.39:0/2616900804 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7d3ec700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee78104610 0x7fee781a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7dbed700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee78109890 0x7fee7819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7cbeb700 1 -- 172.21.15.39:0/2616900804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee78104610 msgr2=0x7fee781a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7cbeb700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee78104610 0x7fee781a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7cbeb700 1 -- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee78109890 msgr2=0x7fee7819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7cbeb700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee78109890 0x7fee7819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7cbeb700 1 -- 172.21.15.39:0/2616900804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7819b280 con 0x7fee78108f40 2024-05-04T03:08:58.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7d3ec700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee78104610 0x7fee781a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.350+0000 7fee7dbed700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee78109890 0x7fee7819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.351+0000 7fee7cbeb700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78108f40 0x7fee781a0f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee6800e970 tx=0x7fee6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.351+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68004a30 con 0x7fee78108f40 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.351+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7819b570 con 0x7fee78108f40 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.351+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee6800cc70 con 0x7fee78108f40 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.351+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee781a7770 con 0x7fee78108f40 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.351+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68005520 con 0x7fee78108f40 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.352+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee5c001030 con 0x7fee78108f40 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.353+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fee68004bd0 con 0x7fee78108f40 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.354+0000 7fee6e7fc700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fee640b2910 0x7fee640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.354+0000 7fee7d3ec700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fee640b2910 0x7fee640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.354+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fee680e2970 con 0x7fee78108f40 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.354+0000 7fee6e7fc700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] conn(0x7fee640bc4a0 0x7fee640be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:58.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.354+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 --> [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee640bf060 con 0x7fee640bc4a0 2024-05-04T03:08:58.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.354+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fee680e2dc0 con 0x7fee78108f40 2024-05-04T03:08:58.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.355+0000 7fee7d3ec700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fee640b2910 0x7fee640b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee74011490 tx=0x7fee74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:58.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.355+0000 7fee7dbed700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] conn(0x7fee640bc4a0 0x7fee640be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:58.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.355+0000 7fee7dbed700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] conn(0x7fee640bc4a0 0x7fee640be990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:58.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.356+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 <== osd.1 v2:172.21.15.66:6800/2695940383 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee640bf060 con 0x7fee640bc4a0 2024-05-04T03:08:58.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.372+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 --> [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee5c002e60 con 0x7fee640bc4a0 2024-05-04T03:08:58.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee6e7fc700 1 -- 172.21.15.39:0/2616900804 <== osd.1 v2:172.21.15.66:6800/2695940383 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fee5c002e60 con 0x7fee640bc4a0 2024-05-04T03:08:58.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] conn(0x7fee640bc4a0 msgr2=0x7fee640be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] conn(0x7fee640bc4a0 0x7fee640be990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fee640b2910 msgr2=0x7fee640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fee640b2910 0x7fee640b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee74011490 tx=0x7fee74009260 comp rx=0 tx=0).stop 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78108f40 msgr2=0x7fee781a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 --2- 172.21.15.39:0/2616900804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee78108f40 0x7fee781a0f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fee6800e970 tx=0x7fee6800ec80 comp rx=0 tx=0).stop 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7dbed700 1 -- 172.21.15.39:0/2616900804 reap_dead start 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 shutdown_connections 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 >> 172.21.15.39:0/2616900804 conn(0x7fee7807af70 msgr2=0x7fee78101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 shutdown_connections 2024-05-04T03:08:58.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:08:58.373+0000 7fee7f650700 1 -- 172.21.15.39:0/2616900804 wait complete. 2024-05-04T03:08:58.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:08:58 smithi167 ceph-mon[33614]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-04T03:08:58.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:08:58 smithi066 ceph-mon[33264]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-04T03:08:59.754 INFO:teuthology.orchestra.run.smithi039.stdout:34359738455 2024-05-04T03:08:59.755 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-04T03:09:00.200 INFO:teuthology.orchestra.run.smithi039.stdout:154618822719 2024-05-04T03:09:00.201 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-04T03:09:00.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:00 smithi167 ceph-mon[33614]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:00.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:00 smithi039 ceph-mon[29715]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:00.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:00 smithi066 ceph-mon[33264]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:00.603 INFO:teuthology.orchestra.run.smithi039.stdout:55834574931 2024-05-04T03:09:00.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-04T03:09:02.310 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:02 smithi039 ceph-mon[29715]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-04T03:09:02.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:02 smithi167 ceph-mon[33614]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-04T03:09:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:02 smithi066 ceph-mon[33264]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-04T03:09:03.048 INFO:teuthology.orchestra.run.smithi039.stdout:253403070507 2024-05-04T03:09:03.048 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.273+0000 7f6ff2a96700 1 -- 172.21.15.39:0/3276674920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec108740 msgr2=0x7f6fec1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.273+0000 7f6ff2a96700 1 --2- 172.21.15.39:0/3276674920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec108740 0x7f6fec1100a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400b0d0 tx=0x7f6fe400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.273+0000 7f6ff2a96700 1 -- 172.21.15.39:0/3276674920 shutdown_connections 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.274+0000 7f6ff2a96700 1 --2- 172.21.15.39:0/3276674920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec108740 0x7f6fec1100a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.274+0000 7f6ff2a96700 1 --2- 172.21.15.39:0/3276674920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fec1034c0 0x7f6fec1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.274+0000 7f6ff2a96700 1 --2- 172.21.15.39:0/3276674920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fec107df0 0x7f6fec108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.274+0000 7f6ff2a96700 1 -- 172.21.15.39:0/3276674920 >> 172.21.15.39:0/3276674920 conn(0x7f6fec0fedc0 msgr2=0x7f6fec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.274+0000 7f6ff2a96700 1 -- 172.21.15.39:0/3276674920 shutdown_connections 2024-05-04T03:09:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.274+0000 7f6ff2a96700 1 -- 172.21.15.39:0/3276674920 wait complete. 2024-05-04T03:09:03.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff2a96700 1 Processor -- start 2024-05-04T03:09:03.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff2a96700 1 -- start start 2024-05-04T03:09:03.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff2a96700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec1034c0 0x7f6fec19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:03.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff2a96700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fec107df0 0x7f6fec19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff2a96700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fec108740 0x7f6fec199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff2a96700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec113270 con 0x7f6fec1034c0 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff2a96700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1130f0 con 0x7f6fec108740 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff2a96700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec1133f0 con 0x7f6fec107df0 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6febfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fec107df0 0x7f6fec19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff0832700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec1034c0 0x7f6fec19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6febfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fec107df0 0x7f6fec19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:45644/0 (socket says 172.21.15.39:45644) 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff0832700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec1034c0 0x7f6fec19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60936/0 (socket says 172.21.15.39:60936) 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6febfff700 1 -- 172.21.15.39:0/2836621664 learned_addr learned my addr 172.21.15.39:0/2836621664 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.275+0000 7f6ff1033700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fec108740 0x7f6fec199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6febfff700 1 -- 172.21.15.39:0/2836621664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fec108740 msgr2=0x7f6fec199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6febfff700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fec108740 0x7f6fec199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6febfff700 1 -- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec1034c0 msgr2=0x7f6fec19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6febfff700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec1034c0 0x7f6fec19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6febfff700 1 -- 172.21.15.39:0/2836621664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec19a100 con 0x7f6fec107df0 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6ff0832700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fec1034c0 0x7f6fec19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6febfff700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fec107df0 0x7f6fec19fd70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800e970 tx=0x7f6fd800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8004a30 con 0x7f6fec107df0 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec19a3f0 con 0x7f6fec107df0 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fd800cc70 con 0x7f6fec107df0 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8005520 con 0x7f6fec107df0 2024-05-04T03:09:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.276+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec06b610 con 0x7f6fec107df0 2024-05-04T03:09:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.277+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6fec06bc20 con 0x7f6fec107df0 2024-05-04T03:09:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.280+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6fd800c7e0 con 0x7f6fec107df0 2024-05-04T03:09:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.280+0000 7f6fe9ffb700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6fdc0b2a10 0x7f6fdc0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.280+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6fd80e2880 con 0x7f6fec107df0 2024-05-04T03:09:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.280+0000 7f6ff0832700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6fdc0b2a10 0x7f6fdc0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.281+0000 7f6fe9ffb700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] conn(0x7f6fdc0bc5a0 0x7f6fdc0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.281+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 --> [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6fd800be50 con 0x7f6fdc0bc5a0 2024-05-04T03:09:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.281+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f6fd80e2c70 con 0x7f6fec107df0 2024-05-04T03:09:03.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.281+0000 7f6ff1033700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] conn(0x7f6fdc0bc5a0 0x7f6fdc0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:03.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.281+0000 7f6ff0832700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6fdc0b2a10 0x7f6fdc0b4f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6fec19ab90 tx=0x7f6fe00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:03.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.281+0000 7f6ff1033700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] conn(0x7f6fdc0bc5a0 0x7f6fdc0bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:03.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.282+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 <== osd.3 v2:172.21.15.39:6810/747919958 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6fd800be50 con 0x7f6fdc0bc5a0 2024-05-04T03:09:03.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.299+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 --> [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6fec06c020 con 0x7f6fdc0bc5a0 2024-05-04T03:09:03.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.299+0000 7f6fe9ffb700 1 -- 172.21.15.39:0/2836621664 <== osd.3 v2:172.21.15.39:6810/747919958 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6fec06c020 con 0x7f6fdc0bc5a0 2024-05-04T03:09:03.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.299+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] conn(0x7f6fdc0bc5a0 msgr2=0x7f6fdc0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:03.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.299+0000 7f6ff2a96700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] conn(0x7f6fdc0bc5a0 0x7f6fdc0bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:03.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.299+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6fdc0b2a10 msgr2=0x7f6fdc0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:03.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.299+0000 7f6ff2a96700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6fdc0b2a10 0x7f6fdc0b4f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6fec19ab90 tx=0x7f6fe00073d0 comp rx=0 tx=0).stop 2024-05-04T03:09:03.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.299+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fec107df0 msgr2=0x7f6fec19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:03.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.299+0000 7f6ff2a96700 1 --2- 172.21.15.39:0/2836621664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fec107df0 0x7f6fec19fd70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800e970 tx=0x7f6fd800ec80 comp rx=0 tx=0).stop 2024-05-04T03:09:03.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.300+0000 7f6ff1033700 1 -- 172.21.15.39:0/2836621664 reap_dead start 2024-05-04T03:09:03.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.300+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 shutdown_connections 2024-05-04T03:09:03.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.300+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 >> 172.21.15.39:0/2836621664 conn(0x7f6fec0fedc0 msgr2=0x7f6fec1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:03.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.300+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 shutdown_connections 2024-05-04T03:09:03.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:03.300+0000 7f6ff2a96700 1 -- 172.21.15.39:0/2836621664 wait complete. 2024-05-04T03:09:03.714 INFO:teuthology.orchestra.run.smithi039.stdout:227633266735 2024-05-04T03:09:03.714 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-04T03:09:04.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:04 smithi167 ceph-mon[33614]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-04T03:09:04.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:04 smithi039 ceph-mon[29715]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-04T03:09:04.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:04 smithi066 ceph-mon[33264]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 -- 172.21.15.39:0/1334994106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3770108f00 msgr2=0x7f3770109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 --2- 172.21.15.39:0/1334994106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3770108f00 0x7f3770109310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f37640099c0 tx=0x7f3764009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 -- 172.21.15.39:0/1334994106 shutdown_connections 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 --2- 172.21.15.39:0/1334994106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3770109850 0x7f37701111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 --2- 172.21.15.39:0/1334994106 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37701045d0 0x7f37701049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 --2- 172.21.15.39:0/1334994106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3770108f00 0x7f3770109310 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 -- 172.21.15.39:0/1334994106 >> 172.21.15.39:0/1334994106 conn(0x7f37700ffea0 msgr2=0x7f37701022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 -- 172.21.15.39:0/1334994106 shutdown_connections 2024-05-04T03:09:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.595+0000 7f37750c2700 1 -- 172.21.15.39:0/1334994106 wait complete. 2024-05-04T03:09:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f37750c2700 1 Processor -- start 2024-05-04T03:09:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f37750c2700 1 -- start start 2024-05-04T03:09:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f37750c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37701045d0 0x7f37701a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f37750c2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3770108f00 0x7f37701a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f37750c2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3770109850 0x7f377019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f37750c2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3770114310 con 0x7f37701045d0 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f37750c2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3770114190 con 0x7f3770109850 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f37750c2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3770114490 con 0x7f3770108f00 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f376ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37701045d0 0x7f37701a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f376ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37701045d0 0x7f37701a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60954/0 (socket says 172.21.15.39:60954) 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.596+0000 7f376ed9d700 1 -- 172.21.15.39:0/30334995 learned_addr learned my addr 172.21.15.39:0/30334995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f376e59c700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3770108f00 0x7f37701a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f376f59e700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3770109850 0x7f377019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f376ed9d700 1 -- 172.21.15.39:0/30334995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3770108f00 msgr2=0x7f37701a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f376ed9d700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3770108f00 0x7f37701a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f376ed9d700 1 -- 172.21.15.39:0/30334995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3770109850 msgr2=0x7f377019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f376ed9d700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3770109850 0x7f377019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f376ed9d700 1 -- 172.21.15.39:0/30334995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377019b280 con 0x7f37701045d0 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f376ed9d700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37701045d0 0x7f37701a09c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f376000d980 tx=0x7f376000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760016070 con 0x7f37701045d0 2024-05-04T03:09:04.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377019b570 con 0x7f37701045d0 2024-05-04T03:09:04.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3760004830 con 0x7f37701045d0 2024-05-04T03:09:04.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37600054c0 con 0x7f37701045d0 2024-05-04T03:09:04.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.597+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37701a7770 con 0x7f37701045d0 2024-05-04T03:09:04.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.598+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3770063f10 con 0x7f37701045d0 2024-05-04T03:09:04.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.600+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3760004220 con 0x7f37701045d0 2024-05-04T03:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.601+0000 7f3757fff700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37580b2910 0x7f37580b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.601+0000 7f376e59c700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37580b2910 0x7f37580b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.601+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f37600e26c0 con 0x7f37701045d0 2024-05-04T03:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.601+0000 7f3757fff700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] conn(0x7f37580bc4a0 0x7f37580be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.601+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 --> [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3760015ec0 con 0x7f37580bc4a0 2024-05-04T03:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.601+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f37600e2b10 con 0x7f37701045d0 2024-05-04T03:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.601+0000 7f376e59c700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37580b2910 0x7f37580b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3770107ab0 tx=0x7f3764005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.601+0000 7f376f59e700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] conn(0x7f37580bc4a0 0x7f37580be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:04.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.602+0000 7f376f59e700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] conn(0x7f37580bc4a0 0x7f37580be990 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:04.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.603+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 <== osd.11 v2:172.21.15.167:6824/3994242029 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3760015ec0 con 0x7f37580bc4a0 2024-05-04T03:09:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.619+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 --> [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3770064340 con 0x7f37580bc4a0 2024-05-04T03:09:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f3757fff700 1 -- 172.21.15.39:0/30334995 <== osd.11 v2:172.21.15.167:6824/3994242029 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3770064340 con 0x7f37580bc4a0 2024-05-04T03:09:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 >> [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] conn(0x7f37580bc4a0 msgr2=0x7f37580be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f37750c2700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] conn(0x7f37580bc4a0 0x7f37580be990 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37580b2910 msgr2=0x7f37580b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:04.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f37750c2700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37580b2910 0x7f37580b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3770107ab0 tx=0x7f3764005fb0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37701045d0 msgr2=0x7f37701a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:04.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f37750c2700 1 --2- 172.21.15.39:0/30334995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37701045d0 0x7f37701a09c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f376000d980 tx=0x7f376000dc90 comp rx=0 tx=0).stop 2024-05-04T03:09:04.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f376f59e700 1 -- 172.21.15.39:0/30334995 reap_dead start 2024-05-04T03:09:04.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 shutdown_connections 2024-05-04T03:09:04.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.620+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 >> 172.21.15.39:0/30334995 conn(0x7f37700ffea0 msgr2=0x7f37701022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:04.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.621+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 shutdown_connections 2024-05-04T03:09:04.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:04.621+0000 7f37750c2700 1 -- 172.21.15.39:0/30334995 wait complete. 2024-05-04T03:09:06.056 INFO:teuthology.orchestra.run.smithi039.stdout:300647710754 2024-05-04T03:09:06.057 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-04T03:09:06.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:06 smithi039 ceph-mon[29715]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:06.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:06 smithi167 ceph-mon[33614]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:06.489 INFO:teuthology.orchestra.run.smithi039.stdout:107374182474 2024-05-04T03:09:06.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-04T03:09:06.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:06 smithi066 ceph-mon[33264]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:07.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.098+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/3277722534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84103a30 msgr2=0x7fcf84110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.098+0000 7fcf8b0ad700 1 --2- 172.21.15.39:0/3277722534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84103a30 0x7fcf84110750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009990 tx=0x7fcf70009ca0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.098+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/3277722534 shutdown_connections 2024-05-04T03:09:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.098+0000 7fcf8b0ad700 1 --2- 172.21.15.39:0/3277722534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf84110c90 0x7fcf841130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.098+0000 7fcf8b0ad700 1 --2- 172.21.15.39:0/3277722534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf841030e0 0x7fcf841034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.098+0000 7fcf8b0ad700 1 --2- 172.21.15.39:0/3277722534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84103a30 0x7fcf84110750 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.098+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/3277722534 >> 172.21.15.39:0/3277722534 conn(0x7fcf840fedc0 msgr2=0x7fcf841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.099+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/3277722534 shutdown_connections 2024-05-04T03:09:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.099+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/3277722534 wait complete. 2024-05-04T03:09:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8b0ad700 1 Processor -- start 2024-05-04T03:09:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8b0ad700 1 -- start start 2024-05-04T03:09:07.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8b0ad700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf841030e0 0x7fcf841053c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8b0ad700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf84103a30 0x7fcf84105900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8b0ad700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84110c90 0x7fcf84105e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8b0ad700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf84116520 con 0x7fcf84110c90 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8b0ad700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841163a0 con 0x7fcf841030e0 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8b0ad700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf841166a0 con 0x7fcf84103a30 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8964a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84110c90 0x7fcf84105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8964a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84110c90 0x7fcf84105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41430/0 (socket says 172.21.15.39:41430) 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf8964a700 1 -- 172.21.15.39:0/247223752 learned_addr learned my addr 172.21.15.39:0/247223752 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf88e49700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf841030e0 0x7fcf841053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.100+0000 7fcf83fff700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf84103a30 0x7fcf84105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf8964a700 1 -- 172.21.15.39:0/247223752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf84103a30 msgr2=0x7fcf84105900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf8964a700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf84103a30 0x7fcf84105900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf8964a700 1 -- 172.21.15.39:0/247223752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf841030e0 msgr2=0x7fcf841053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf8964a700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf841030e0 0x7fcf841053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf8964a700 1 -- 172.21.15.39:0/247223752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf841ad4f0 con 0x7fcf84110c90 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf8964a700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84110c90 0x7fcf84105e40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c007ee0 tx=0x7fcf7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:07.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c004bb0 con 0x7fcf84110c90 2024-05-04T03:09:07.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf841ad780 con 0x7fcf84110c90 2024-05-04T03:09:07.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf7c004d50 con 0x7fcf84110c90 2024-05-04T03:09:07.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.101+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0058b0 con 0x7fcf84110c90 2024-05-04T03:09:07.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.102+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf841adce0 con 0x7fcf84110c90 2024-05-04T03:09:07.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.102+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf84063f50 con 0x7fcf84110c90 2024-05-04T03:09:07.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.104+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcf7c01a020 con 0x7fcf84110c90 2024-05-04T03:09:07.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.105+0000 7fcf81ffb700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf740b2990 0x7fcf740b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.105+0000 7fcf88e49700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf740b2990 0x7fcf740b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.105+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fcf7c0e23a0 con 0x7fcf84110c90 2024-05-04T03:09:07.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.105+0000 7fcf81ffb700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] conn(0x7fcf740bc520 0x7fcf740bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.105+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 --> [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf7c01fe50 con 0x7fcf740bc520 2024-05-04T03:09:07.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.105+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fcf7c0e27f0 con 0x7fcf84110c90 2024-05-04T03:09:07.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.106+0000 7fcf83fff700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] conn(0x7fcf740bc520 0x7fcf740bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.106+0000 7fcf88e49700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf740b2990 0x7fcf740b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcf78005950 tx=0x7fcf7800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:07.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.106+0000 7fcf83fff700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] conn(0x7fcf740bc520 0x7fcf740bea10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:07.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.107+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 <== osd.10 v2:172.21.15.66:6824/3028550986 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf7c01fe50 con 0x7fcf740bc520 2024-05-04T03:09:07.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.133+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 --> [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf84064350 con 0x7fcf740bc520 2024-05-04T03:09:07.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf81ffb700 1 -- 172.21.15.39:0/247223752 <== osd.10 v2:172.21.15.66:6824/3028550986 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf84064350 con 0x7fcf740bc520 2024-05-04T03:09:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 >> [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] conn(0x7fcf740bc520 msgr2=0x7fcf740bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] conn(0x7fcf740bc520 0x7fcf740bea10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf740b2990 msgr2=0x7fcf740b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf740b2990 0x7fcf740b4ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcf78005950 tx=0x7fcf7800a300 comp rx=0 tx=0).stop 2024-05-04T03:09:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84110c90 msgr2=0x7fcf84105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 --2- 172.21.15.39:0/247223752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf84110c90 0x7fcf84105e40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c007ee0 tx=0x7fcf7c00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8964a700 1 -- 172.21.15.39:0/247223752 reap_dead start 2024-05-04T03:09:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 shutdown_connections 2024-05-04T03:09:07.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 >> 172.21.15.39:0/247223752 conn(0x7fcf840fedc0 msgr2=0x7fcf84112870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:07.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 shutdown_connections 2024-05-04T03:09:07.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.134+0000 7fcf8b0ad700 1 -- 172.21.15.39:0/247223752 wait complete. 2024-05-04T03:09:07.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 -- 172.21.15.39:0/4105047372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8108f00 msgr2=0x7fa7b8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 --2- 172.21.15.39:0/4105047372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8108f00 0x7fa7b8109310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0099c0 tx=0x7fa7ac009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 -- 172.21.15.39:0/4105047372 shutdown_connections 2024-05-04T03:09:07.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 --2- 172.21.15.39:0/4105047372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b8109850 0x7fa7b81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 --2- 172.21.15.39:0/4105047372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7b8102b50 0x7fa7b8102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 --2- 172.21.15.39:0/4105047372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8108f00 0x7fa7b8109310 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 -- 172.21.15.39:0/4105047372 >> 172.21.15.39:0/4105047372 conn(0x7fa7b80fe6f0 msgr2=0x7fa7b8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:07.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 -- 172.21.15.39:0/4105047372 shutdown_connections 2024-05-04T03:09:07.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.761+0000 7fa7bce54700 1 -- 172.21.15.39:0/4105047372 wait complete. 2024-05-04T03:09:07.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.763+0000 7fa7bce54700 1 Processor -- start 2024-05-04T03:09:07.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.763+0000 7fa7bce54700 1 -- start start 2024-05-04T03:09:07.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.763+0000 7fa7bce54700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8102b50 0x7fa7b819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.763+0000 7fa7bce54700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b8108f00 0x7fa7b819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7bce54700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7b8109850 0x7fa7b81a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7bce54700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b8116560 con 0x7fa7b8102b50 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7bce54700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81163e0 con 0x7fa7b8109850 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7bce54700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81166e0 con 0x7fa7b8108f00 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8102b50 0x7fa7b819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8102b50 0x7fa7b819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41458/0 (socket says 172.21.15.39:41458) 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 -- 172.21.15.39:0/1887892506 learned_addr learned my addr 172.21.15.39:0/1887892506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b5d9b700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b8108f00 0x7fa7b819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 -- 172.21.15.39:0/1887892506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b8108f00 msgr2=0x7fa7b819d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b8108f00 0x7fa7b819d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 -- 172.21.15.39:0/1887892506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7b8109850 msgr2=0x7fa7b81a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7b8109850 0x7fa7b81a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 -- 172.21.15.39:0/1887892506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b81a1db0 con 0x7fa7b8102b50 2024-05-04T03:09:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.764+0000 7fa7b659c700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8102b50 0x7fa7b819cf90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800c9b0 tx=0x7fa7a800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.765+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8007b00 con 0x7fa7b8102b50 2024-05-04T03:09:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.765+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7a8007ca0 con 0x7fa7b8102b50 2024-05-04T03:09:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.765+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8018660 con 0x7fa7b8102b50 2024-05-04T03:09:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.765+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b81a20a0 con 0x7fa7b8102b50 2024-05-04T03:09:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.765+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b81a2500 con 0x7fa7b8102b50 2024-05-04T03:09:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.766+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7b8063f50 con 0x7fa7b8102b50 2024-05-04T03:09:07.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.768+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa7a800f450 con 0x7fa7b8102b50 2024-05-04T03:09:07.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.769+0000 7fa7a77fe700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7a00b2990 0x7fa7a00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.769+0000 7fa7b5d9b700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7a00b2990 0x7fa7a00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.769+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa7a80e2770 con 0x7fa7b8102b50 2024-05-04T03:09:07.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.770+0000 7fa7a77fe700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] conn(0x7fa7a00bc520 0x7fa7a00bea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.770+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 --> [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7a8015e70 con 0x7fa7a00bc520 2024-05-04T03:09:07.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.770+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fa7a80e2b20 con 0x7fa7b8102b50 2024-05-04T03:09:07.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.770+0000 7fa7b5d9b700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7a00b2990 0x7fa7a00b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:07.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.770+0000 7fa7b6d9d700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] conn(0x7fa7a00bc520 0x7fa7a00bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.771+0000 7fa7b6d9d700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] conn(0x7fa7a00bc520 0x7fa7a00bea10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:07.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.771+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 <== osd.7 v2:172.21.15.66:6816/931306967 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7a8015e70 con 0x7fa7a00bc520 2024-05-04T03:09:07.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.788+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 --> [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa7b8064420 con 0x7fa7a00bc520 2024-05-04T03:09:07.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.789+0000 7fa7a77fe700 1 -- 172.21.15.39:0/1887892506 <== osd.7 v2:172.21.15.66:6816/931306967 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa7b8064420 con 0x7fa7a00bc520 2024-05-04T03:09:07.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.789+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 >> [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] conn(0x7fa7a00bc520 msgr2=0x7fa7a00bea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.789+0000 7fa7bce54700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] conn(0x7fa7a00bc520 0x7fa7a00bea10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.789+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7a00b2990 msgr2=0x7fa7a00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.789+0000 7fa7bce54700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7a00b2990 0x7fa7a00b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac01a040 comp rx=0 tx=0).stop 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.789+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8102b50 msgr2=0x7fa7b819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.789+0000 7fa7bce54700 1 --2- 172.21.15.39:0/1887892506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b8102b50 0x7fa7b819cf90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800c9b0 tx=0x7fa7a800ccc0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.789+0000 7fa7b6d9d700 1 -- 172.21.15.39:0/1887892506 reap_dead start 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.790+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 shutdown_connections 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.790+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 >> 172.21.15.39:0/1887892506 conn(0x7fa7b80fe6f0 msgr2=0x7fa7b810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.790+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 shutdown_connections 2024-05-04T03:09:07.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:07.790+0000 7fa7bce54700 1 -- 172.21.15.39:0/1887892506 wait complete. 2024-05-04T03:09:08.290 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:08 smithi039 ceph-mon[29715]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:08.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:08 smithi167 ceph-mon[33614]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:08 smithi066 ceph-mon[33264]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:09.113 INFO:teuthology.orchestra.run.smithi039.stdout:274877906983 2024-05-04T03:09:09.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-04T03:09:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.303+0000 7f441cd27700 1 -- 172.21.15.39:0/2875169978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418108f10 msgr2=0x7f4418109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.303+0000 7f441cd27700 1 --2- 172.21.15.39:0/2875169978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418108f10 0x7f4418109320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f44080099c0 tx=0x7f4408009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.304+0000 7f441cd27700 1 -- 172.21.15.39:0/2875169978 shutdown_connections 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.304+0000 7f441cd27700 1 --2- 172.21.15.39:0/2875169978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4418109860 0x7f44181111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.304+0000 7f441cd27700 1 --2- 172.21.15.39:0/2875169978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44181045e0 0x7f44181049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.304+0000 7f441cd27700 1 --2- 172.21.15.39:0/2875169978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418108f10 0x7f4418109320 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.304+0000 7f441cd27700 1 -- 172.21.15.39:0/2875169978 >> 172.21.15.39:0/2875169978 conn(0x7f44180ffed0 msgr2=0x7f4418102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.304+0000 7f441cd27700 1 -- 172.21.15.39:0/2875169978 shutdown_connections 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.304+0000 7f441cd27700 1 -- 172.21.15.39:0/2875169978 wait complete. 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.304+0000 7f441cd27700 1 Processor -- start 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f441cd27700 1 -- start start 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f441cd27700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44181045e0 0x7f441819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f441cd27700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4418108f10 0x7f441819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f441cd27700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418109860 0x7f441819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f441cd27700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4418114390 con 0x7f4418109860 2024-05-04T03:09:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f441cd27700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4418114210 con 0x7f44181045e0 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f441cd27700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4418114510 con 0x7f4418108f10 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4416d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418109860 0x7f441819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4416d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418109860 0x7f441819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41480/0 (socket says 172.21.15.39:41480) 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f441659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44181045e0 0x7f441819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4416d9d700 1 -- 172.21.15.39:0/3888504557 learned_addr learned my addr 172.21.15.39:0/3888504557 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4415d9b700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4418108f10 0x7f441819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4416d9d700 1 -- 172.21.15.39:0/3888504557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4418108f10 msgr2=0x7f441819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4416d9d700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4418108f10 0x7f441819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4416d9d700 1 -- 172.21.15.39:0/3888504557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44181045e0 msgr2=0x7f441819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4416d9d700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44181045e0 0x7f441819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.305+0000 7f4416d9d700 1 -- 172.21.15.39:0/3888504557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441819fc90 con 0x7f4418109860 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.306+0000 7f4416d9d700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418109860 0x7f441819f5f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f441000b970 tx=0x7f441000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.306+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000d860 con 0x7f4418109860 2024-05-04T03:09:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.306+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441819ff80 con 0x7f4418109860 2024-05-04T03:09:10.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.306+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f441000e430 con 0x7f4418109860 2024-05-04T03:09:10.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.306+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410017660 con 0x7f4418109860 2024-05-04T03:09:10.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.306+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44181a78a0 con 0x7f4418109860 2024-05-04T03:09:10.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.307+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4418063f50 con 0x7f4418109860 2024-05-04T03:09:10.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.308+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f441000da00 con 0x7f4418109860 2024-05-04T03:09:10.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.309+0000 7f44077fe700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44000b2910 0x7f44000b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:10.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.309+0000 7f441659c700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44000b2910 0x7f44000b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:10.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.309+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f44100e23f0 con 0x7f4418109860 2024-05-04T03:09:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.310+0000 7f44077fe700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] conn(0x7f44000bc4a0 0x7f44000be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.310+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 --> [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4410014e70 con 0x7f44000bc4a0 2024-05-04T03:09:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.310+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f44100e2840 con 0x7f4418109860 2024-05-04T03:09:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.310+0000 7f441659c700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44000b2910 0x7f44000b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f440c005950 tx=0x7f440c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.310+0000 7f4415d9b700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] conn(0x7f44000bc4a0 0x7f44000be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.310+0000 7f4415d9b700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] conn(0x7f44000bc4a0 0x7f44000be990 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.311+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 <== osd.2 v2:172.21.15.167:6800/2785227057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4410014e70 con 0x7f44000bc4a0 2024-05-04T03:09:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.328+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 --> [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4418064350 con 0x7f44000bc4a0 2024-05-04T03:09:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.328+0000 7f44077fe700 1 -- 172.21.15.39:0/3888504557 <== osd.2 v2:172.21.15.167:6800/2785227057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4418064350 con 0x7f44000bc4a0 2024-05-04T03:09:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 >> [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] conn(0x7f44000bc4a0 msgr2=0x7f44000be990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] conn(0x7f44000bc4a0 0x7f44000be990 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44000b2910 msgr2=0x7f44000b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44000b2910 0x7f44000b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f440c005950 tx=0x7f440c00b410 comp rx=0 tx=0).stop 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418109860 msgr2=0x7f441819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 --2- 172.21.15.39:0/3888504557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4418109860 0x7f441819f5f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f441000b970 tx=0x7f441000bc80 comp rx=0 tx=0).stop 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f4416d9d700 1 -- 172.21.15.39:0/3888504557 reap_dead start 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 shutdown_connections 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 >> 172.21.15.39:0/3888504557 conn(0x7f44180ffed0 msgr2=0x7f441810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 shutdown_connections 2024-05-04T03:09:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:10.329+0000 7f441cd27700 1 -- 172.21.15.39:0/3888504557 wait complete. 2024-05-04T03:09:10.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:10 smithi167 ceph-mon[33614]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:10.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:10 smithi039 ceph-mon[29715]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:10.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:10 smithi066 ceph-mon[33264]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:11.015 INFO:teuthology.orchestra.run.smithi039.stdout:201863462967 2024-05-04T03:09:11.015 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-04T03:09:11.040 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:11.040 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:11.041 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:11.041 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:11.509 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:11 smithi039 ceph-mon[29715]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:11.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:11 smithi066 ceph-mon[33264]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:11.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:11 smithi167 ceph-mon[33614]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:12.223 INFO:teuthology.orchestra.run.smithi039.stdout:77309411409 2024-05-04T03:09:12.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-04T03:09:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:14.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:14 smithi039 ceph-mon[29715]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:14.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:14 smithi167 ceph-mon[33614]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:14.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:14 smithi066 ceph-mon[33264]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:16.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:16 smithi039 ceph-mon[29715]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:16.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:16 smithi167 ceph-mon[33614]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:16.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:16 smithi066 ceph-mon[33264]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:17.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:17 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3119462848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-04T03:09:17.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:17 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3119462848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-04T03:09:17.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:17 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3119462848' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-04T03:09:18.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:18 smithi039 ceph-mon[29715]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:18.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:18 smithi167 ceph-mon[33614]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:18 smithi066 ceph-mon[33264]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 -- 172.21.15.39:0/164093418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc424108760 msgr2=0x7fc4241100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 --2- 172.21.15.39:0/164093418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc424108760 0x7fc4241100c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00b0d0 tx=0x7fc41c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 -- 172.21.15.39:0/164093418 shutdown_connections 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 --2- 172.21.15.39:0/164093418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc424108760 0x7fc4241100c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 --2- 172.21.15.39:0/164093418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4241034e0 0x7fc4241038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 --2- 172.21.15.39:0/164093418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc424107e10 0x7fc424108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 -- 172.21.15.39:0/164093418 >> 172.21.15.39:0/164093418 conn(0x7fc4240fef30 msgr2=0x7fc424101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 -- 172.21.15.39:0/164093418 shutdown_connections 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.816+0000 7fc42a02c700 1 -- 172.21.15.39:0/164093418 wait complete. 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.817+0000 7fc42a02c700 1 Processor -- start 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.817+0000 7fc42a02c700 1 -- start start 2024-05-04T03:09:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.817+0000 7fc42a02c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4241034e0 0x7fc4241a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.817+0000 7fc42a02c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc424107e10 0x7fc424198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.817+0000 7fc42a02c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424108760 0x7fc424198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.817+0000 7fc42a02c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc424113240 con 0x7fc4241034e0 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.817+0000 7fc42a02c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241130c0 con 0x7fc424107e10 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.817+0000 7fc42a02c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241133c0 con 0x7fc424108760 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4241034e0 0x7fc4241a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4241034e0 0x7fc4241a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42922/0 (socket says 172.21.15.39:42922) 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 -- 172.21.15.39:0/3119462848 learned_addr learned my addr 172.21.15.39:0/3119462848 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc422ffd700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc424107e10 0x7fc424198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc423fff700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424108760 0x7fc424198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 -- 172.21.15.39:0/3119462848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424108760 msgr2=0x7fc424198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424108760 0x7fc424198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 -- 172.21.15.39:0/3119462848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc424107e10 msgr2=0x7fc424198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc424107e10 0x7fc424198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 -- 172.21.15.39:0/3119462848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4241997f0 con 0x7fc4241034e0 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc422ffd700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc424107e10 0x7fc424198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc4237fe700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4241034e0 0x7fc4241a08e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc41400d980 tx=0x7fc41400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.818+0000 7fc420ff9700 1 -- 172.21.15.39:0/3119462848 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414016070 con 0x7fc4241034e0 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.819+0000 7fc420ff9700 1 -- 172.21.15.39:0/3119462848 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc414004830 con 0x7fc4241034e0 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:227633266739 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.819+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4241b6010 con 0x7fc4241034e0 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.819+0000 7fc420ff9700 1 -- 172.21.15.39:0/3119462848 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4140054c0 con 0x7fc4241034e0 2024-05-04T03:09:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.819+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4241b6570 con 0x7fc4241034e0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.821+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc424104990 con 0x7fc4241034e0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.821+0000 7fc420ff9700 1 -- 172.21.15.39:0/3119462848 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc414004220 con 0x7fc4241034e0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.822+0000 7fc420ff9700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc40c0b2990 0x7fc40c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.822+0000 7fc422ffd700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc40c0b2990 0x7fc40c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.822+0000 7fc420ff9700 1 -- 172.21.15.39:0/3119462848 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc4140e1fa0 con 0x7fc4241034e0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.823+0000 7fc422ffd700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc40c0b2990 0x7fc40c0b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc424109700 tx=0x7fc418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:16.828+0000 7fc420ff9700 1 -- 172.21.15.39:0/3119462848 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc41401f020 con 0x7fc4241034e0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.052+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fc42402e5f0 con 0x7fc4241034e0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.052+0000 7fc420ff9700 1 -- 172.21.15.39:0/3119462848 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc4140a9c10 con 0x7fc4241034e0 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc40c0b2990 msgr2=0x7fc40c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc40c0b2990 0x7fc40c0b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc424109700 tx=0x7fc418009260 comp rx=0 tx=0).stop 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4241034e0 msgr2=0x7fc4241a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4241034e0 0x7fc4241a08e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc41400d980 tx=0x7fc41400dc90 comp rx=0 tx=0).stop 2024-05-04T03:09:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 shutdown_connections 2024-05-04T03:09:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc40c0b2990 0x7fc40c0b4ea0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4241034e0 0x7fc4241a08e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc424108760 0x7fc424198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 --2- 172.21.15.39:0/3119462848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc424107e10 0x7fc424198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 >> 172.21.15.39:0/3119462848 conn(0x7fc4240fef30 msgr2=0x7fc42410ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 shutdown_connections 2024-05-04T03:09:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:17.054+0000 7fc42a02c700 1 -- 172.21.15.39:0/3119462848 wait complete. 2024-05-04T03:09:20.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:20 smithi167 ceph-mon[33614]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:20.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:20 smithi039 ceph-mon[29715]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:20.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:20 smithi066 ceph-mon[33264]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:20.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266735 got 227633266739 for osd.8 2024-05-04T03:09:20.671 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 -- 172.21.15.39:0/3743914526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8103550 msgr2=0x7f81d8103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 --2- 172.21.15.39:0/3743914526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8103550 0x7f81d8103960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f81c80099c0 tx=0x7f81c8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 -- 172.21.15.39:0/3743914526 shutdown_connections 2024-05-04T03:09:21.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 --2- 172.21.15.39:0/3743914526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8103550 0x7f81d8103960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 --2- 172.21.15.39:0/3743914526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81d8108780 0x7f81d81100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 --2- 172.21.15.39:0/3743914526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81d8107e30 0x7f81d8108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 -- 172.21.15.39:0/3743914526 >> 172.21.15.39:0/3743914526 conn(0x7f81d8078e00 msgr2=0x7f81d8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:21.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 -- 172.21.15.39:0/3743914526 shutdown_connections 2024-05-04T03:09:21.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.763+0000 7f81dd839700 1 -- 172.21.15.39:0/3743914526 wait complete. 2024-05-04T03:09:21.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.764+0000 7f81dd839700 1 Processor -- start 2024-05-04T03:09:21.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.764+0000 7f81dd839700 1 -- start start 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81dd839700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81d8103550 0x7f81d819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81dd839700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81d8107e30 0x7f81d819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81dd839700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8108780 0x7f81d819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81dd839700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8113260 con 0x7f81d8108780 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81dd839700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81130e0 con 0x7f81d8103550 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81dd839700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81133e0 con 0x7f81d8107e30 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8108780 0x7f81d819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8108780 0x7f81d819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42986/0 (socket says 172.21.15.39:42986) 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 -- 172.21.15.39:0/285275772 learned_addr learned my addr 172.21.15.39:0/285275772 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d67fc700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81d8107e30 0x7f81d819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d6ffd700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81d8103550 0x7f81d819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 -- 172.21.15.39:0/285275772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81d8107e30 msgr2=0x7f81d819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81d8107e30 0x7f81d819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 -- 172.21.15.39:0/285275772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81d8103550 msgr2=0x7f81d819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81d8103550 0x7f81d819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 -- 172.21.15.39:0/285275772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d819fc00 con 0x7f81d8108780 2024-05-04T03:09:21.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d67fc700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81d8107e30 0x7f81d819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:21.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81d77fe700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8108780 0x7f81d819f610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81d0007d70 tx=0x7f81d000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:21.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.765+0000 7f81dc837700 1 -- 172.21.15.39:0/285275772 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0016020 con 0x7f81d8108780 2024-05-04T03:09:21.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.766+0000 7f81dc837700 1 -- 172.21.15.39:0/285275772 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81d000ad40 con 0x7f81d8108780 2024-05-04T03:09:21.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.766+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d819fef0 con 0x7f81d8108780 2024-05-04T03:09:21.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.766+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d81a03e0 con 0x7f81d8108780 2024-05-04T03:09:21.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.766+0000 7f81dc837700 1 -- 172.21.15.39:0/285275772 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d00115e0 con 0x7f81d8108780 2024-05-04T03:09:21.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.767+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d80512c0 con 0x7f81d8108780 2024-05-04T03:09:21.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.769+0000 7f81dc837700 1 -- 172.21.15.39:0/285275772 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f81d0019020 con 0x7f81d8108780 2024-05-04T03:09:21.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.769+0000 7f81dc837700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f81c00b2910 0x7f81c00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:21.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.769+0000 7f81d6ffd700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f81c00b2910 0x7f81c00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:21.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.769+0000 7f81dc837700 1 -- 172.21.15.39:0/285275772 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f81d00e2660 con 0x7f81d8108780 2024-05-04T03:09:21.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.770+0000 7f81d6ffd700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f81c00b2910 0x7f81c00b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f81c8000c00 tx=0x7f81c8005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:21.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.773+0000 7f81dc837700 1 -- 172.21.15.39:0/285275772 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81d0020020 con 0x7f81d8108780 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 -- 172.21.15.39:0/2068878438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 msgr2=0x7f27bc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 --2- 172.21.15.39:0/2068878438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 0x7f27bc1100a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b0d0 tx=0x7f27b400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 -- 172.21.15.39:0/2068878438 shutdown_connections 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 --2- 172.21.15.39:0/2068878438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 0x7f27bc1100a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 --2- 172.21.15.39:0/2068878438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27bc1034c0 0x7f27bc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 --2- 172.21.15.39:0/2068878438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27bc107df0 0x7f27bc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 -- 172.21.15.39:0/2068878438 >> 172.21.15.39:0/2068878438 conn(0x7f27bc0fedc0 msgr2=0x7f27bc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 -- 172.21.15.39:0/2068878438 shutdown_connections 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.641+0000 7f27c15a9700 1 -- 172.21.15.39:0/2068878438 wait complete. 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.642+0000 7f27c15a9700 1 Processor -- start 2024-05-04T03:09:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.642+0000 7f27c15a9700 1 -- start start 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27c15a9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27bc1034c0 0x7f27bc19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27c15a9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27bc107df0 0x7f27bc19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27c15a9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 0x7f27bc19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27c15a9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc113220 con 0x7f27bc108740 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27c15a9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1130a0 con 0x7f27bc1034c0 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27c15a9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc1133a0 con 0x7f27bc107df0 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 0x7f27bc19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 0x7f27bc19c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42964/0 (socket says 172.21.15.39:42964) 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 -- 172.21.15.39:0/3088317331 learned_addr learned my addr 172.21.15.39:0/3088317331 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27ba7fc700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27bc107df0 0x7f27bc19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27baffd700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27bc1034c0 0x7f27bc19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 -- 172.21.15.39:0/3088317331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27bc107df0 msgr2=0x7f27bc19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27bc107df0 0x7f27bc19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 -- 172.21.15.39:0/3088317331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27bc1034c0 msgr2=0x7f27bc19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27bc1034c0 0x7f27bc19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 -- 172.21.15.39:0/3088317331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc1a6e70 con 0x7f27bc108740 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27ba7fc700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27bc107df0 0x7f27bc19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27baffd700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27bc1034c0 0x7f27bc19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27bb7fe700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 0x7f27bc19c5c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009850 tx=0x7f27b4007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27b3fff700 1 -- 172.21.15.39:0/3088317331 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b401f030 con 0x7f27bc108740 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.643+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc1a7100 con 0x7f27bc108740 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.644+0000 7f27b3fff700 1 -- 172.21.15.39:0/3088317331 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27b401bad0 con 0x7f27bc108740 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.644+0000 7f27b3fff700 1 -- 172.21.15.39:0/3088317331 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4012bb0 con 0x7f27bc108740 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.644+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1a75b0 con 0x7f27bc108740 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.644+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc0512c0 con 0x7f27bc108740 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.647+0000 7f27b3fff700 1 -- 172.21.15.39:0/3088317331 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f27b4022030 con 0x7f27bc108740 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.648+0000 7f27b3fff700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f27a80b2910 0x7f27a80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.648+0000 7f27baffd700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f27a80b2910 0x7f27a80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:21.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.648+0000 7f27b3fff700 1 -- 172.21.15.39:0/3088317331 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f27b400f020 con 0x7f27bc108740 2024-05-04T03:09:21.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.648+0000 7f27baffd700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f27a80b2910 0x7f27a80b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f27a4005fd0 tx=0x7f27a4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:21.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.652+0000 7f27b3fff700 1 -- 172.21.15.39:0/3088317331 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27b4029020 con 0x7f27bc108740 2024-05-04T03:09:21.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.875+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f27bc02e5f0 con 0x7f27bc108740 2024-05-04T03:09:21.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.875+0000 7f27b3fff700 1 -- 172.21.15.39:0/3088317331 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f27b40ab930 con 0x7f27bc108740 2024-05-04T03:09:21.876 INFO:teuthology.orchestra.run.smithi039.stdout:180388626495 2024-05-04T03:09:21.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f27a80b2910 msgr2=0x7f27a80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:21.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f27a80b2910 0x7f27a80b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f27a4005fd0 tx=0x7f27a4005dc0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 msgr2=0x7f27bc19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:21.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 0x7f27bc19c5c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009850 tx=0x7f27b4007e40 comp rx=0 tx=0).stop 2024-05-04T03:09:21.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 shutdown_connections 2024-05-04T03:09:21.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f27a80b2910 0x7f27a80b4e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27bc108740 0x7f27bc19c5c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27bc1034c0 0x7f27bc19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 --2- 172.21.15.39:0/3088317331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27bc107df0 0x7f27bc19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:21.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 >> 172.21.15.39:0/3088317331 conn(0x7f27bc0fedc0 msgr2=0x7f27bc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:21.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 shutdown_connections 2024-05-04T03:09:21.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.879+0000 7f27c15a9700 1 -- 172.21.15.39:0/3088317331 wait complete. 2024-05-04T03:09:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.997+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f81d819c140 con 0x7f81d8108780 2024-05-04T03:09:21.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.998+0000 7f81dc837700 1 -- 172.21.15.39:0/285275772 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f81d00aa340 con 0x7f81d8108780 2024-05-04T03:09:21.999 INFO:teuthology.orchestra.run.smithi039.stdout:133143986249 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f81c00b2910 msgr2=0x7f81c00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f81c00b2910 0x7f81c00b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f81c8000c00 tx=0x7f81c8005940 comp rx=0 tx=0).stop 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8108780 msgr2=0x7f81d819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8108780 0x7f81d819f610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81d0007d70 tx=0x7f81d000a340 comp rx=0 tx=0).stop 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 shutdown_connections 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f81c00b2910 0x7f81c00b4e20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81d8108780 0x7f81d819f610 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81d8103550 0x7f81d819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 --2- 172.21.15.39:0/285275772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81d8107e30 0x7f81d819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 >> 172.21.15.39:0/285275772 conn(0x7f81d8078e00 msgr2=0x7f81d81016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 shutdown_connections 2024-05-04T03:09:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:22.000+0000 7f81dd839700 1 -- 172.21.15.39:0/285275772 wait complete. 2024-05-04T03:09:22.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:22 smithi039 ceph-mon[29715]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-04T03:09:22.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:22 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2600944690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-04T03:09:22.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:22 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3088317331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-04T03:09:22.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:22 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/285275772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-04T03:09:22.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:22 smithi167 ceph-mon[33614]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-04T03:09:22.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:22 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2600944690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-04T03:09:22.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:22 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3088317331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-04T03:09:22.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:22 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/285275772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-04T03:09:22.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:22 smithi066 ceph-mon[33264]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-04T03:09:22.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:22 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2600944690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-04T03:09:22.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:22 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3088317331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-04T03:09:22.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:22 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/285275772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-04T03:09:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.289+0000 7f8e277cb700 1 -- 172.21.15.39:0/181355100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105500 msgr2=0x7f8e20105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.289+0000 7f8e277cb700 1 --2- 172.21.15.39:0/181355100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105500 0x7f8e20105910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009990 tx=0x7f8e1c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.289+0000 7f8e277cb700 1 -- 172.21.15.39:0/181355100 shutdown_connections 2024-05-04T03:09:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.289+0000 7f8e277cb700 1 --2- 172.21.15.39:0/181355100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105500 0x7f8e20105910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.289+0000 7f8e277cb700 1 --2- 172.21.15.39:0/181355100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e2010e930 0x7f8e20110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.289+0000 7f8e277cb700 1 --2- 172.21.15.39:0/181355100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e20105e50 0x7f8e2010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.290+0000 7f8e277cb700 1 -- 172.21.15.39:0/181355100 >> 172.21.15.39:0/181355100 conn(0x7f8e200fef30 msgr2=0x7f8e20101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.290+0000 7f8e277cb700 1 -- 172.21.15.39:0/181355100 shutdown_connections 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.290+0000 7f8e277cb700 1 -- 172.21.15.39:0/181355100 wait complete. 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.290+0000 7f8e277cb700 1 Processor -- start 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.290+0000 7f8e277cb700 1 -- start start 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e277cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e20105500 0x7f8e201031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e277cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105e50 0x7f8e20103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e277cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e2010e930 0x7f8e20103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e277cb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20114300 con 0x7f8e20105e50 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e277cb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20114180 con 0x7f8e20105500 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e277cb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20114480 con 0x7f8e2010e930 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105e50 0x7f8e20103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105e50 0x7f8e20103730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42936/0 (socket says 172.21.15.39:42936) 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 -- 172.21.15.39:0/2600944690 learned_addr learned my addr 172.21.15.39:0/2600944690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e25567700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e20105500 0x7f8e201031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:22.591 INFO:teuthology.orchestra.run.smithi039.stdout:300647710757 2024-05-04T03:09:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e25d68700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e2010e930 0x7f8e20103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 -- 172.21.15.39:0/2600944690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e2010e930 msgr2=0x7f8e20103c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e2010e930 0x7f8e20103c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 -- 172.21.15.39:0/2600944690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e20105500 msgr2=0x7f8e201031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e20105500 0x7f8e201031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 -- 172.21.15.39:0/2600944690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e201a90e0 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e24d66700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105e50 0x7f8e20103730 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000d950 tx=0x7f8e1000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e167fc700 1 -- 172.21.15.39:0/2600944690 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10016070 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e201a93d0 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e167fc700 1 -- 172.21.15.39:0/2600944690 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e10009dd0 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e167fc700 1 -- 172.21.15.39:0/2600944690 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1000f3c0 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.291+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e201a9910 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.292+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e200512c0 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.294+0000 7f8e167fc700 1 -- 172.21.15.39:0/2600944690 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8e10019070 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.295+0000 7f8e167fc700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8e0c0b2910 0x7f8e0c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.295+0000 7f8e25567700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8e0c0b2910 0x7f8e0c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.296+0000 7f8e167fc700 1 -- 172.21.15.39:0/2600944690 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8e100e1fc0 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.296+0000 7f8e25567700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8e0c0b2910 0x7f8e0c0b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00b5c0 tx=0x7f8e1c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.299+0000 7f8e167fc700 1 -- 172.21.15.39:0/2600944690 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e100a9b90 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.522+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f8e2002e5f0 con 0x7f8e20105e50 2024-05-04T03:09:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.523+0000 7f8e167fc700 1 -- 172.21.15.39:0/2600944690 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8e100a9490 con 0x7f8e20105e50 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8e0c0b2910 msgr2=0x7f8e0c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8e0c0b2910 0x7f8e0c0b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00b5c0 tx=0x7f8e1c005960 comp rx=0 tx=0).stop 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105e50 msgr2=0x7f8e20103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105e50 0x7f8e20103730 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000d950 tx=0x7f8e1000dc60 comp rx=0 tx=0).stop 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 shutdown_connections 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8e0c0b2910 0x7f8e0c0b4e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e2010e930 0x7f8e20103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8e20105500 0x7f8e201031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 --2- 172.21.15.39:0/2600944690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e20105e50 0x7f8e20103730 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 >> 172.21.15.39:0/2600944690 conn(0x7f8e200fef30 msgr2=0x7f8e200ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 shutdown_connections 2024-05-04T03:09:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:21.526+0000 7f8e277cb700 1 -- 172.21.15.39:0/2600944690 wait complete. 2024-05-04T03:09:23.005 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:23.005 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:23.703 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710754 got 300647710757 for osd.11 2024-05-04T03:09:23.703 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 -- 172.21.15.39:0/1815751431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc10e8f0 msgr2=0x7f3dcc110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1815751431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc10e8f0 0x7f3dcc110dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400b0a0 tx=0x7f3dc400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 -- 172.21.15.39:0/1815751431 shutdown_connections 2024-05-04T03:09:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1815751431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc10e8f0 0x7f3dcc110dd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1815751431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dcc105480 0x7f3dcc105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1815751431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dcc105dd0 0x7f3dcc10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 -- 172.21.15.39:0/1815751431 >> 172.21.15.39:0/1815751431 conn(0x7f3dcc0fef30 msgr2=0x7f3dcc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:24.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 -- 172.21.15.39:0/1815751431 shutdown_connections 2024-05-04T03:09:24.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.323+0000 7f3dd4023700 1 -- 172.21.15.39:0/1815751431 wait complete. 2024-05-04T03:09:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd4023700 1 Processor -- start 2024-05-04T03:09:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd4023700 1 -- start start 2024-05-04T03:09:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd4023700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc105480 0x7f3dcc1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd4023700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dcc105dd0 0x7f3dcc1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd4023700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dcc10e8f0 0x7f3dcc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd4023700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc114300 con 0x7f3dcc105480 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd4023700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc114180 con 0x7f3dcc10e8f0 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd4023700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc114480 con 0x7f3dcc105dd0 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.324+0000 7f3dd1dbf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc105480 0x7f3dcc1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd1dbf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc105480 0x7f3dcc1a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43002/0 (socket says 172.21.15.39:43002) 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd1dbf700 1 -- 172.21.15.39:0/1102122352 learned_addr learned my addr 172.21.15.39:0/1102122352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd15be700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dcc105dd0 0x7f3dcc1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd25c0700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dcc10e8f0 0x7f3dcc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd1dbf700 1 -- 172.21.15.39:0/1102122352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dcc105dd0 msgr2=0x7f3dcc1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd1dbf700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dcc105dd0 0x7f3dcc1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd1dbf700 1 -- 172.21.15.39:0/1102122352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dcc10e8f0 msgr2=0x7f3dcc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd1dbf700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dcc10e8f0 0x7f3dcc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd1dbf700 1 -- 172.21.15.39:0/1102122352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc19b130 con 0x7f3dcc105480 2024-05-04T03:09:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd1dbf700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc105480 0x7f3dcc1a0860 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c920 tx=0x7f3dbc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dc2ffd700 1 -- 172.21.15.39:0/1102122352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc004a30 con 0x7f3dcc105480 2024-05-04T03:09:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dc2ffd700 1 -- 172.21.15.39:0/1102122352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dbc0054b0 con 0x7f3dcc105480 2024-05-04T03:09:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc19b3c0 con 0x7f3dcc105480 2024-05-04T03:09:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dc2ffd700 1 -- 172.21.15.39:0/1102122352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc007b70 con 0x7f3dcc105480 2024-05-04T03:09:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.325+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc1a76f0 con 0x7f3dcc105480 2024-05-04T03:09:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.326+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc0512c0 con 0x7f3dcc105480 2024-05-04T03:09:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.328+0000 7f3dc2ffd700 1 -- 172.21.15.39:0/1102122352 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3dbc004bd0 con 0x7f3dcc105480 2024-05-04T03:09:24.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:24 smithi039 ceph-mon[29715]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.329+0000 7f3dc2ffd700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3db80b2990 0x7f3db80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.329+0000 7f3dd15be700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3db80b2990 0x7f3db80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.329+0000 7f3dc2ffd700 1 -- 172.21.15.39:0/1102122352 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3dbc0e2220 con 0x7f3dcc105480 2024-05-04T03:09:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.329+0000 7f3dd15be700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3db80b2990 0x7f3db80b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc1023e0 tx=0x7f3dc800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.332+0000 7f3dc2ffd700 1 -- 172.21.15.39:0/1102122352 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3dbc014030 con 0x7f3dcc105480 2024-05-04T03:09:24.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:24 smithi167 ceph-mon[33614]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.551+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3dcc19bfb0 con 0x7f3dcc105480 2024-05-04T03:09:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.551+0000 7f3dc2ffd700 1 -- 172.21.15.39:0/1102122352 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3dbc005660 con 0x7f3dcc105480 2024-05-04T03:09:24.552 INFO:teuthology.orchestra.run.smithi039.stdout:154618822724 2024-05-04T03:09:24.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3db80b2990 msgr2=0x7f3db80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:24.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3db80b2990 0x7f3db80b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc1023e0 tx=0x7f3dc800a300 comp rx=0 tx=0).stop 2024-05-04T03:09:24.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc105480 msgr2=0x7f3dcc1a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:24.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc105480 0x7f3dcc1a0860 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c920 tx=0x7f3dbc00cc30 comp rx=0 tx=0).stop 2024-05-04T03:09:24.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 shutdown_connections 2024-05-04T03:09:24.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3db80b2990 0x7f3db80b4ea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dcc105480 0x7f3dcc1a0860 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dcc10e8f0 0x7f3dcc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 --2- 172.21.15.39:0/1102122352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3dcc105dd0 0x7f3dcc1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.554+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 >> 172.21.15.39:0/1102122352 conn(0x7f3dcc0fef30 msgr2=0x7f3dcc0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.555+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 shutdown_connections 2024-05-04T03:09:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:24.555+0000 7f3dd4023700 1 -- 172.21.15.39:0/1102122352 wait complete. 2024-05-04T03:09:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:24 smithi066 ceph-mon[33264]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:24.990 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626489 got 180388626495 for osd.6 2024-05-04T03:09:24.990 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:25.286 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:25 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1102122352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 -- 172.21.15.39:0/4027199075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f201034c0 msgr2=0x7f7f201038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4027199075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f201034c0 0x7f7f201038d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0099c0 tx=0x7f7f1c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 -- 172.21.15.39:0/4027199075 shutdown_connections 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4027199075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f201034c0 0x7f7f201038d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4027199075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f20108740 0x7f7f201100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4027199075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f20107df0 0x7f7f20108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 -- 172.21.15.39:0/4027199075 >> 172.21.15.39:0/4027199075 conn(0x7f7f200fedf0 msgr2=0x7f7f20101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 -- 172.21.15.39:0/4027199075 shutdown_connections 2024-05-04T03:09:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.303+0000 7f7f273b2700 1 -- 172.21.15.39:0/4027199075 wait complete. 2024-05-04T03:09:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f273b2700 1 Processor -- start 2024-05-04T03:09:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f273b2700 1 -- start start 2024-05-04T03:09:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f273b2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f201034c0 0x7f7f2019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f273b2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f20107df0 0x7f7f2019c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f273b2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f20108740 0x7f7f2019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f273b2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20113290 con 0x7f7f20107df0 2024-05-04T03:09:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f273b2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20113110 con 0x7f7f201034c0 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f273b2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20113410 con 0x7f7f20108740 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f2494d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f20107df0 0x7f7f2019c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f2494d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f20107df0 0x7f7f2019c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43026/0 (socket says 172.21.15.39:43026) 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f2494d700 1 -- 172.21.15.39:0/4224578408 learned_addr learned my addr 172.21.15.39:0/4224578408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.304+0000 7f7f2514e700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f201034c0 0x7f7f2019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f2494d700 1 -- 172.21.15.39:0/4224578408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f20108740 msgr2=0x7f7f2019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f2494d700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f20108740 0x7f7f2019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f2494d700 1 -- 172.21.15.39:0/4224578408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f201034c0 msgr2=0x7f7f2019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f2494d700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f201034c0 0x7f7f2019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f2494d700 1 -- 172.21.15.39:0/4224578408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f201a6f40 con 0x7f7f20107df0 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f2494d700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f20107df0 0x7f7f2019c110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b500 tx=0x7f7f1000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f167fc700 1 -- 172.21.15.39:0/4224578408 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10015020 con 0x7f7f20107df0 2024-05-04T03:09:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f201a7230 con 0x7f7f20107df0 2024-05-04T03:09:25.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f167fc700 1 -- 172.21.15.39:0/4224578408 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f10011cc0 con 0x7f7f20107df0 2024-05-04T03:09:25.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.305+0000 7f7f167fc700 1 -- 172.21.15.39:0/4224578408 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10010c80 con 0x7f7f20107df0 2024-05-04T03:09:25.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.306+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f201a7720 con 0x7f7f20107df0 2024-05-04T03:09:25.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.307+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f200512c0 con 0x7f7f20107df0 2024-05-04T03:09:25.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.308+0000 7f7f167fc700 1 -- 172.21.15.39:0/4224578408 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7f10018020 con 0x7f7f20107df0 2024-05-04T03:09:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.309+0000 7f7f167fc700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f0c0b2910 0x7f7f0c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:25.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.309+0000 7f7f2514e700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f0c0b2910 0x7f7f0c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:25.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.309+0000 7f7f167fc700 1 -- 172.21.15.39:0/4224578408 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7f100e1a40 con 0x7f7f20107df0 2024-05-04T03:09:25.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.309+0000 7f7f2514e700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f0c0b2910 0x7f7f0c0b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c000c00 tx=0x7f7f1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:25.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.314+0000 7f7f167fc700 1 -- 172.21.15.39:0/4224578408 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f1001f020 con 0x7f7f20107df0 2024-05-04T03:09:25.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:25 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1102122352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-04T03:09:25.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.528+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f7f20076030 con 0x7f7f20107df0 2024-05-04T03:09:25.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.529+0000 7f7f167fc700 1 -- 172.21.15.39:0/4224578408 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7f20076030 con 0x7f7f20107df0 2024-05-04T03:09:25.529 INFO:teuthology.orchestra.run.smithi039.stdout:253403070513 2024-05-04T03:09:25.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f0c0b2910 msgr2=0x7f7f0c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:25.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f0c0b2910 0x7f7f0c0b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c000c00 tx=0x7f7f1c005fb0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f20107df0 msgr2=0x7f7f2019c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f20107df0 0x7f7f2019c110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b500 tx=0x7f7f1000b810 comp rx=0 tx=0).stop 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 shutdown_connections 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7f0c0b2910 0x7f7f0c0b4e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f20108740 0x7f7f2019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f201034c0 0x7f7f2019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 --2- 172.21.15.39:0/4224578408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f20107df0 0x7f7f2019c110 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 >> 172.21.15.39:0/4224578408 conn(0x7f7f200fedf0 msgr2=0x7f7f20101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 shutdown_connections 2024-05-04T03:09:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:25.531+0000 7f7f273b2700 1 -- 172.21.15.39:0/4224578408 wait complete. 2024-05-04T03:09:25.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:25 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1102122352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-04T03:09:26.395 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:26 smithi039 ceph-mon[29715]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:26.395 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:26 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4224578408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-04T03:09:26.424 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986242 got 133143986249 for osd.4 2024-05-04T03:09:26.424 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:26 smithi167 ceph-mon[33614]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:26.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:26 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4224578408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-04T03:09:26.469 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:26 smithi066 ceph-mon[33264]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:26.469 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:26 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4224578408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-04T03:09:27.547 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070507 got 253403070513 for osd.9 2024-05-04T03:09:27.547 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:27.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 -- 172.21.15.39:0/3463704683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 msgr2=0x7f91e010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:27.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 --2- 172.21.15.39:0/3463704683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 0x7f91e010e320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009990 tx=0x7f91d4009ca0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 -- 172.21.15.39:0/3463704683 shutdown_connections 2024-05-04T03:09:27.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 --2- 172.21.15.39:0/3463704683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91e010e8f0 0x7f91e0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 --2- 172.21.15.39:0/3463704683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e0105480 0x7f91e0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 --2- 172.21.15.39:0/3463704683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 0x7f91e010e320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 -- 172.21.15.39:0/3463704683 >> 172.21.15.39:0/3463704683 conn(0x7f91e00fef30 msgr2=0x7f91e0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:27.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 -- 172.21.15.39:0/3463704683 shutdown_connections 2024-05-04T03:09:27.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.775+0000 7f91e576c700 1 -- 172.21.15.39:0/3463704683 wait complete. 2024-05-04T03:09:27.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.776+0000 7f91e576c700 1 Processor -- start 2024-05-04T03:09:27.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.776+0000 7f91e576c700 1 -- start start 2024-05-04T03:09:27.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91e576c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e0105480 0x7f91e0103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:27.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91e576c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 0x7f91e01036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:27.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91e576c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91e010e8f0 0x7f91e0103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:27.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91e576c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0114320 con 0x7f91e0105dd0 2024-05-04T03:09:27.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91e576c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91e01141a0 con 0x7f91e010e8f0 2024-05-04T03:09:27.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91e576c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91e01144a0 con 0x7f91e0105480 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91de7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 0x7f91e01036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91de7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 0x7f91e01036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35906/0 (socket says 172.21.15.39:35906) 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91de7fc700 1 -- 172.21.15.39:0/924721949 learned_addr learned my addr 172.21.15.39:0/924721949 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91deffd700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e0105480 0x7f91e0103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.777+0000 7f91df7fe700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91e010e8f0 0x7f91e0103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91de7fc700 1 -- 172.21.15.39:0/924721949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e0105480 msgr2=0x7f91e0103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91de7fc700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e0105480 0x7f91e0103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91de7fc700 1 -- 172.21.15.39:0/924721949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91e010e8f0 msgr2=0x7f91e0103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91de7fc700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91e010e8f0 0x7f91e0103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91de7fc700 1 -- 172.21.15.39:0/924721949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e01a9160 con 0x7f91e0105dd0 2024-05-04T03:09:27.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91deffd700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e0105480 0x7f91e0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:27.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91df7fe700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91e010e8f0 0x7f91e0103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:27.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91de7fc700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 0x7f91e01036b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f91d4000c00 tx=0x7f91d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:27.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91c7fff700 1 -- 172.21.15.39:0/924721949 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4027070 con 0x7f91e0105dd0 2024-05-04T03:09:27.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e01a93f0 con 0x7f91e0105dd0 2024-05-04T03:09:27.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91c7fff700 1 -- 172.21.15.39:0/924721949 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91d400fc70 con 0x7f91e0105dd0 2024-05-04T03:09:27.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91c7fff700 1 -- 172.21.15.39:0/924721949 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4022d40 con 0x7f91e0105dd0 2024-05-04T03:09:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.778+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01a9840 con 0x7f91e0105dd0 2024-05-04T03:09:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.779+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e00512c0 con 0x7f91e0105dd0 2024-05-04T03:09:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.781+0000 7f91c7fff700 1 -- 172.21.15.39:0/924721949 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f91d402a070 con 0x7f91e0105dd0 2024-05-04T03:09:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.782+0000 7f91c7fff700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91c80b2910 0x7f91c80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.782+0000 7f91deffd700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91c80b2910 0x7f91c80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.782+0000 7f91c7fff700 1 -- 172.21.15.39:0/924721949 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f91d40e3130 con 0x7f91e0105dd0 2024-05-04T03:09:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.782+0000 7f91deffd700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91c80b2910 0x7f91c80b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f91d000bcf0 tx=0x7f91d000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:27.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:27.787+0000 7f91c7fff700 1 -- 172.21.15.39:0/924721949 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91d405fdf0 con 0x7f91e0105dd0 2024-05-04T03:09:28.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.009+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f91e002e7b0 con 0x7f91e0105dd0 2024-05-04T03:09:28.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.009+0000 7f91c7fff700 1 -- 172.21.15.39:0/924721949 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f91d4013070 con 0x7f91e0105dd0 2024-05-04T03:09:28.011 INFO:teuthology.orchestra.run.smithi039.stdout:55834574937 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91c80b2910 msgr2=0x7f91c80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91c80b2910 0x7f91c80b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f91d000bcf0 tx=0x7f91d000b2c0 comp rx=0 tx=0).stop 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 msgr2=0x7f91e01036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 0x7f91e01036b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f91d4000c00 tx=0x7f91d4009f90 comp rx=0 tx=0).stop 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 shutdown_connections 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91c80b2910 0x7f91c80b4e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91e010e8f0 0x7f91e0103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91e0105480 0x7f91e0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:28.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 --2- 172.21.15.39:0/924721949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e0105dd0 0x7f91e01036b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:28.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 >> 172.21.15.39:0/924721949 conn(0x7f91e00fef30 msgr2=0x7f91e00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:28.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 shutdown_connections 2024-05-04T03:09:28.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:28.012+0000 7f91e576c700 1 -- 172.21.15.39:0/924721949 wait complete. 2024-05-04T03:09:28.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:28 smithi039 ceph-mon[29715]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:28.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:28 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/924721949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-04T03:09:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:28 smithi167 ceph-mon[33614]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:28.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:28 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/924721949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-04T03:09:28.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:28 smithi066 ceph-mon[33264]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:28.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:28 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/924721949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-04T03:09:29.324 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574931 got 55834574937 for osd.1 2024-05-04T03:09:29.324 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.801+0000 7f1834896700 1 -- 172.21.15.39:0/2798240438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 msgr2=0x7f18301100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.801+0000 7f1834896700 1 --2- 172.21.15.39:0/2798240438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 0x7f18301100a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f182800b0d0 tx=0x7f182800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:09:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.802+0000 7f1834896700 1 -- 172.21.15.39:0/2798240438 shutdown_connections 2024-05-04T03:09:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.802+0000 7f1834896700 1 --2- 172.21.15.39:0/2798240438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 0x7f18301100a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.802+0000 7f1834896700 1 --2- 172.21.15.39:0/2798240438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18301034c0 0x7f18301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.802+0000 7f1834896700 1 --2- 172.21.15.39:0/2798240438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1830107df0 0x7f1830108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.802+0000 7f1834896700 1 -- 172.21.15.39:0/2798240438 >> 172.21.15.39:0/2798240438 conn(0x7f18300fedf0 msgr2=0x7f1830101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.802+0000 7f1834896700 1 -- 172.21.15.39:0/2798240438 shutdown_connections 2024-05-04T03:09:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.802+0000 7f1834896700 1 -- 172.21.15.39:0/2798240438 wait complete. 2024-05-04T03:09:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f1834896700 1 Processor -- start 2024-05-04T03:09:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f1834896700 1 -- start start 2024-05-04T03:09:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f1834896700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18301034c0 0x7f18301a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f1834896700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1830107df0 0x7f18301a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f1834896700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 0x7f183019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f1834896700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1830113220 con 0x7f1830108740 2024-05-04T03:09:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f1834896700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18301130a0 con 0x7f1830107df0 2024-05-04T03:09:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f1834896700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18301133a0 con 0x7f18301034c0 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 0x7f183019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 0x7f183019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35934/0 (socket says 172.21.15.39:35934) 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182ed9d700 1 -- 172.21.15.39:0/382483326 learned_addr learned my addr 172.21.15.39:0/382483326 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182dd9b700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1830107df0 0x7f18301a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182ed9d700 1 -- 172.21.15.39:0/382483326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18301034c0 msgr2=0x7f18301a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182ed9d700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18301034c0 0x7f18301a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182ed9d700 1 -- 172.21.15.39:0/382483326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1830107df0 msgr2=0x7f18301a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182ed9d700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1830107df0 0x7f18301a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.803+0000 7f182ed9d700 1 -- 172.21.15.39:0/382483326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183019b1d0 con 0x7f1830108740 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.804+0000 7f182ed9d700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 0x7f183019aaf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1828007a00 tx=0x7f1828007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.804+0000 7f181f7fe700 1 -- 172.21.15.39:0/382483326 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182801f070 con 0x7f1830108740 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.804+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183019b460 con 0x7f1830108740 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.804+0000 7f181f7fe700 1 -- 172.21.15.39:0/382483326 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1828004c50 con 0x7f1830108740 2024-05-04T03:09:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.804+0000 7f181f7fe700 1 -- 172.21.15.39:0/382483326 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182801b690 con 0x7f1830108740 2024-05-04T03:09:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.804+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18301a7760 con 0x7f1830108740 2024-05-04T03:09:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.805+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1830104970 con 0x7f1830108740 2024-05-04T03:09:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.806+0000 7f181f7fe700 1 -- 172.21.15.39:0/382483326 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1828022070 con 0x7f1830108740 2024-05-04T03:09:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.807+0000 7f181f7fe700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18180b2910 0x7f18180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.807+0000 7f182e59c700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18180b2910 0x7f18180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.807+0000 7f181f7fe700 1 -- 172.21.15.39:0/382483326 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f182800f020 con 0x7f1830108740 2024-05-04T03:09:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.807+0000 7f182e59c700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18180b2910 0x7f18180b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1820005950 tx=0x7f18200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:29.811+0000 7f181f7fe700 1 -- 172.21.15.39:0/382483326 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1828029020 con 0x7f1830108740 2024-05-04T03:09:30.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.029+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f183019bfe0 con 0x7f1830108740 2024-05-04T03:09:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:274877906987 2024-05-04T03:09:30.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.030+0000 7f181f7fe700 1 -- 172.21.15.39:0/382483326 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f18280ab500 con 0x7f1830108740 2024-05-04T03:09:30.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18180b2910 msgr2=0x7f18180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:30.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18180b2910 0x7f18180b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1820005950 tx=0x7f18200058e0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 msgr2=0x7f183019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:30.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 0x7f183019aaf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1828007a00 tx=0x7f1828007a30 comp rx=0 tx=0).stop 2024-05-04T03:09:30.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 shutdown_connections 2024-05-04T03:09:30.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18180b2910 0x7f18180b4e20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1830108740 0x7f183019aaf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18301034c0 0x7f18301a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 --2- 172.21.15.39:0/382483326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1830107df0 0x7f18301a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.032+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 >> 172.21.15.39:0/382483326 conn(0x7f18300fedf0 msgr2=0x7f1830075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:30.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.033+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 shutdown_connections 2024-05-04T03:09:30.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.033+0000 7f1834896700 1 -- 172.21.15.39:0/382483326 wait complete. 2024-05-04T03:09:30.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.061+0000 7fa05672b700 1 -- 172.21.15.39:0/4259875710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0501034c0 msgr2=0x7fa0501038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:30.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.061+0000 7fa05672b700 1 --2- 172.21.15.39:0/4259875710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0501034c0 0x7fa0501038d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa0400099c0 tx=0x7fa040009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.061+0000 7fa05672b700 1 -- 172.21.15.39:0/4259875710 shutdown_connections 2024-05-04T03:09:30.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.061+0000 7fa05672b700 1 --2- 172.21.15.39:0/4259875710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0501034c0 0x7fa0501038d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.061+0000 7fa05672b700 1 --2- 172.21.15.39:0/4259875710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa050108740 0x7fa0501100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.061+0000 7fa05672b700 1 --2- 172.21.15.39:0/4259875710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa050107df0 0x7fa050108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.061+0000 7fa05672b700 1 -- 172.21.15.39:0/4259875710 >> 172.21.15.39:0/4259875710 conn(0x7fa0500fedc0 msgr2=0x7fa0501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:30.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.062+0000 7fa05672b700 1 -- 172.21.15.39:0/4259875710 shutdown_connections 2024-05-04T03:09:30.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.062+0000 7fa05672b700 1 -- 172.21.15.39:0/4259875710 wait complete. 2024-05-04T03:09:30.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.062+0000 7fa05672b700 1 Processor -- start 2024-05-04T03:09:30.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa05672b700 1 -- start start 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa05672b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0501034c0 0x7fa0500ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa05672b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa050107df0 0x7fa0500ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa05672b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa050108740 0x7fa0500ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa05672b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa050113220 con 0x7fa050108740 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa05672b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501130a0 con 0x7fa0501034c0 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa05672b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501133a0 con 0x7fa050107df0 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa04ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0501034c0 0x7fa0500ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa04ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0501034c0 0x7fa0500ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:59468/0 (socket says 172.21.15.39:59468) 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa04ffff700 1 -- 172.21.15.39:0/2359075136 learned_addr learned my addr 172.21.15.39:0/2359075136 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa04f7fe700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa050107df0 0x7fa0500ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.063+0000 7fa054cc8700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa050108740 0x7fa0500ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04ffff700 1 -- 172.21.15.39:0/2359075136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa050107df0 msgr2=0x7fa0500ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:30.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04ffff700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa050107df0 0x7fa0500ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04ffff700 1 -- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa050108740 msgr2=0x7fa0500ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04ffff700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa050108740 0x7fa0500ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04ffff700 1 -- 172.21.15.39:0/2359075136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0501a9060 con 0x7fa0501034c0 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa054cc8700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa050108740 0x7fa0500ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04f7fe700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa050107df0 0x7fa0500ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04ffff700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0501034c0 0x7fa0500ff2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa040000c00 tx=0x7fa04000f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04d7fa700 1 -- 172.21.15.39:0/2359075136 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04000f940 con 0x7fa0501034c0 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0501a92f0 con 0x7fa0501034c0 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04d7fa700 1 -- 172.21.15.39:0/2359075136 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa040004830 con 0x7fa0501034c0 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa04d7fa700 1 -- 172.21.15.39:0/2359075136 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0400054b0 con 0x7fa0501034c0 2024-05-04T03:09:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.064+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0501a9740 con 0x7fa0501034c0 2024-05-04T03:09:30.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.065+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa050104970 con 0x7fa0501034c0 2024-05-04T03:09:30.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.067+0000 7fa04d7fa700 1 -- 172.21.15.39:0/2359075136 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa040029060 con 0x7fa0501034c0 2024-05-04T03:09:30.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.068+0000 7fa04d7fa700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0380b2910 0x7fa0380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:30.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.068+0000 7fa04f7fe700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0380b2910 0x7fa0380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:30.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.068+0000 7fa04d7fa700 1 -- 172.21.15.39:0/2359075136 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa0400e3220 con 0x7fa0501034c0 2024-05-04T03:09:30.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.069+0000 7fa04f7fe700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0380b2910 0x7fa0380b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa044005950 tx=0x7fa0440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:30.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.072+0000 7fa04d7fa700 1 -- 172.21.15.39:0/2359075136 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa040013070 con 0x7fa0501034c0 2024-05-04T03:09:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.289+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa05002e5f0 con 0x7fa0501034c0 2024-05-04T03:09:30.290 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:30 smithi039 ceph-mon[29715]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:30.290 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:30 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/382483326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-04T03:09:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.290+0000 7fa04d7fa700 1 -- 172.21.15.39:0/2359075136 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0400b1e20 con 0x7fa0501034c0 2024-05-04T03:09:30.291 INFO:teuthology.orchestra.run.smithi039.stdout:34359738462 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0380b2910 msgr2=0x7fa0380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0380b2910 0x7fa0380b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa044005950 tx=0x7fa0440058e0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0501034c0 msgr2=0x7fa0500ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0501034c0 0x7fa0500ff2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa040000c00 tx=0x7fa04000f510 comp rx=0 tx=0).stop 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 shutdown_connections 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0380b2910 0x7fa0380b4e20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa050108740 0x7fa0500ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0501034c0 0x7fa0500ff2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 --2- 172.21.15.39:0/2359075136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa050107df0 0x7fa0500ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 >> 172.21.15.39:0/2359075136 conn(0x7fa0500fedc0 msgr2=0x7fa05006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 shutdown_connections 2024-05-04T03:09:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:30.292+0000 7fa05672b700 1 -- 172.21.15.39:0/2359075136 wait complete. 2024-05-04T03:09:30.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:30 smithi167 ceph-mon[33614]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:30.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:30 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/382483326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-04T03:09:30.487 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:30.487 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:30.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:30 smithi066 ceph-mon[33264]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:30.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:30 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/382483326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-04T03:09:31.119 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822719 got 154618822724 for osd.5 2024-05-04T03:09:31.119 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:31.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:31 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2359075136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-04T03:09:31.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:31 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2359075136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-04T03:09:31.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:31 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2359075136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-04T03:09:31.588 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906983 got 274877906987 for osd.10 2024-05-04T03:09:31.588 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:32.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:32 smithi167 ceph-mon[33614]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:32.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:32 smithi039 ceph-mon[29715]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:32.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:32 smithi066 ceph-mon[33264]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:33.353 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738455 got 34359738462 for osd.0 2024-05-04T03:09:33.354 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:34.419 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:34 smithi066 ceph-mon[33264]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:34.419 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:34 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:34.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:34 smithi167 ceph-mon[33614]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:34.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:34 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:34.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:34 smithi039 ceph-mon[29715]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:34.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:34 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1767300547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb84108740 msgr2=0x7fcb841100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1767300547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb84108740 0x7fcb841100a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00b0d0 tx=0x7fcb7c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1767300547 shutdown_connections 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1767300547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb84108740 0x7fcb841100a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1767300547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb841034c0 0x7fcb841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1767300547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84107df0 0x7fcb84108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1767300547 >> 172.21.15.39:0/1767300547 conn(0x7fcb840fedc0 msgr2=0x7fcb841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1767300547 shutdown_connections 2024-05-04T03:09:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.801+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1767300547 wait complete. 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8b2c1700 1 Processor -- start 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8b2c1700 1 -- start start 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8b2c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb841034c0 0x7fcb8419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8b2c1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84107df0 0x7fcb8419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8b2c1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb84108740 0x7fcb841a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8b2c1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84113220 con 0x7fcb841034c0 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8b2c1700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841130a0 con 0x7fcb84108740 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8b2c1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb841133a0 con 0x7fcb84107df0 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8905d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb841034c0 0x7fcb8419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8905d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb841034c0 0x7fcb8419bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41992/0 (socket says 172.21.15.39:41992) 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8905d700 1 -- 172.21.15.39:0/1498431485 learned_addr learned my addr 172.21.15.39:0/1498431485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8985e700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb84108740 0x7fcb841a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.802+0000 7fcb8885c700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84107df0 0x7fcb8419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb8905d700 1 -- 172.21.15.39:0/1498431485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84107df0 msgr2=0x7fcb8419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb8905d700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84107df0 0x7fcb8419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb8905d700 1 -- 172.21.15.39:0/1498431485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb84108740 msgr2=0x7fcb841a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb8905d700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb84108740 0x7fcb841a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb8905d700 1 -- 172.21.15.39:0/1498431485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8419c8b0 con 0x7fcb841034c0 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb8985e700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb84108740 0x7fcb841a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb8905d700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb841034c0 0x7fcb8419bac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000e9a0 tx=0x7fcb8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb7a7fc700 1 -- 172.21.15.39:0/1498431485 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000cbd0 con 0x7fcb841034c0 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb7a7fc700 1 -- 172.21.15.39:0/1498431485 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb8000cd70 con 0x7fcb841034c0 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb841a71a0 con 0x7fcb841034c0 2024-05-04T03:09:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.803+0000 7fcb7a7fc700 1 -- 172.21.15.39:0/1498431485 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80018680 con 0x7fcb841034c0 2024-05-04T03:09:35.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.804+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb841a76b0 con 0x7fcb841034c0 2024-05-04T03:09:35.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.805+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb840512c0 con 0x7fcb841034c0 2024-05-04T03:09:35.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.806+0000 7fcb7a7fc700 1 -- 172.21.15.39:0/1498431485 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcb80010430 con 0x7fcb841034c0 2024-05-04T03:09:35.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.807+0000 7fcb7a7fc700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb700b2910 0x7fcb700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.807+0000 7fcb8885c700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb700b2910 0x7fcb700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.807+0000 7fcb7a7fc700 1 -- 172.21.15.39:0/1498431485 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fcb800e2600 con 0x7fcb841034c0 2024-05-04T03:09:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.807+0000 7fcb8885c700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb700b2910 0x7fcb700b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb84106830 tx=0x7fcb74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:35.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:35.812+0000 7fcb7a7fc700 1 -- 172.21.15.39:0/1498431485 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb800aa1d0 con 0x7fcb841034c0 2024-05-04T03:09:36.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.039+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcb8402e5f0 con 0x7fcb841034c0 2024-05-04T03:09:36.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.039+0000 7fcb7a7fc700 1 -- 172.21.15.39:0/1498431485 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb800b2250 con 0x7fcb841034c0 2024-05-04T03:09:36.040 INFO:teuthology.orchestra.run.smithi039.stdout:77309411414 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb700b2910 msgr2=0x7fcb700b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb700b2910 0x7fcb700b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb84106830 tx=0x7fcb74009260 comp rx=0 tx=0).stop 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb841034c0 msgr2=0x7fcb8419bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb841034c0 0x7fcb8419bac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000e9a0 tx=0x7fcb8000ecb0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 shutdown_connections 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb700b2910 0x7fcb700b4e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb841034c0 0x7fcb8419bac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb84108740 0x7fcb841a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 --2- 172.21.15.39:0/1498431485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb84107df0 0x7fcb8419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 >> 172.21.15.39:0/1498431485 conn(0x7fcb840fedc0 msgr2=0x7fcb841096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:36.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 shutdown_connections 2024-05-04T03:09:36.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.041+0000 7fcb8b2c1700 1 -- 172.21.15.39:0/1498431485 wait complete. 2024-05-04T03:09:36.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:36 smithi039 ceph-mon[29715]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:36.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:36 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:36 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:36 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:36 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:36 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1498431485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 -- 172.21.15.39:0/2786244774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b4103310 msgr2=0x7f30b410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/2786244774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b4103310 0x7f30b410f890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c210 tx=0x7f30ac00c520 comp rx=0 tx=0).stop 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 -- 172.21.15.39:0/2786244774 shutdown_connections 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/2786244774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b4103310 0x7f30b410f890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/2786244774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30b41089f0 0x7f30b4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/2786244774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b41029c0 0x7f30b4102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 -- 172.21.15.39:0/2786244774 >> 172.21.15.39:0/2786244774 conn(0x7f30b40fe6f0 msgr2=0x7f30b4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 -- 172.21.15.39:0/2786244774 shutdown_connections 2024-05-04T03:09:36.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.555+0000 7f30b8fa4700 1 -- 172.21.15.39:0/2786244774 wait complete. 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b8fa4700 1 Processor -- start 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b8fa4700 1 -- start start 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b8fa4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30b41029c0 0x7f30b419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b8fa4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b4103310 0x7f30b419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b8fa4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b41089f0 0x7f30b419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b8fa4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f30b41121a0 con 0x7f30b41089f0 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b8fa4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4112020 con 0x7f30b41029c0 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b8fa4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4112320 con 0x7f30b4103310 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b41089f0 0x7f30b419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:36.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b2d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b41089f0 0x7f30b419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42012/0 (socket says 172.21.15.39:42012) 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b2d9d700 1 -- 172.21.15.39:0/1275654751 learned_addr learned my addr 172.21.15.39:0/1275654751 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b259c700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30b41029c0 0x7f30b419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.556+0000 7f30b1d9b700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b4103310 0x7f30b419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30b2d9d700 1 -- 172.21.15.39:0/1275654751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b4103310 msgr2=0x7f30b419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30b2d9d700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b4103310 0x7f30b419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30b2d9d700 1 -- 172.21.15.39:0/1275654751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30b41029c0 msgr2=0x7f30b419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30b2d9d700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30b41029c0 0x7f30b419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30b2d9d700 1 -- 172.21.15.39:0/1275654751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b419fc00 con 0x7f30b41089f0 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30b2d9d700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b41089f0 0x7f30b419f560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c8f0 tx=0x7f30ac0082f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30a37fe700 1 -- 172.21.15.39:0/1275654751 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac022020 con 0x7f30b41089f0 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30a37fe700 1 -- 172.21.15.39:0/1275654751 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30ac004e40 con 0x7f30b41089f0 2024-05-04T03:09:36.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b419fe90 con 0x7f30b41089f0 2024-05-04T03:09:36.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30a37fe700 1 -- 172.21.15.39:0/1275654751 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac01e470 con 0x7f30b41089f0 2024-05-04T03:09:36.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.557+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b40feb00 con 0x7f30b41089f0 2024-05-04T03:09:36.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.558+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b4104100 con 0x7f30b41089f0 2024-05-04T03:09:36.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.561+0000 7f30a37fe700 1 -- 172.21.15.39:0/1275654751 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f30ac025020 con 0x7f30b41089f0 2024-05-04T03:09:36.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.561+0000 7f30a37fe700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f309c0b2910 0x7f309c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:36.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.561+0000 7f30b259c700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f309c0b2910 0x7f309c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:36.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.561+0000 7f30a37fe700 1 -- 172.21.15.39:0/1275654751 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f30ac010020 con 0x7f30b41089f0 2024-05-04T03:09:36.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.562+0000 7f30b259c700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f309c0b2910 0x7f309c0b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f30b4103b00 tx=0x7f30a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:36.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.565+0000 7f30a37fe700 1 -- 172.21.15.39:0/1275654751 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30ac0abb40 con 0x7f30b41089f0 2024-05-04T03:09:36.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:36 smithi066 ceph-mon[33264]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:36.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:36 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:36 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:36 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:36 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:36 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1498431485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-04T03:09:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:36 smithi167 ceph-mon[33614]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:36 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:36 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:36 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:36 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:36 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1498431485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-04T03:09:36.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.758+0000 7fa3e4509700 1 -- 172.21.15.39:0/1069561000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 msgr2=0x7fa3dc10e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.758+0000 7fa3e4509700 1 --2- 172.21.15.39:0/1069561000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 0x7fa3dc10e300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009990 tx=0x7fa3cc009ca0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.758+0000 7fa3e4509700 1 -- 172.21.15.39:0/1069561000 shutdown_connections 2024-05-04T03:09:36.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.758+0000 7fa3e4509700 1 --2- 172.21.15.39:0/1069561000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3dc10e8d0 0x7fa3dc110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.758+0000 7fa3e4509700 1 --2- 172.21.15.39:0/1069561000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3dc105460 0x7fa3dc105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.758+0000 7fa3e4509700 1 --2- 172.21.15.39:0/1069561000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 0x7fa3dc10e300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.758+0000 7fa3e4509700 1 -- 172.21.15.39:0/1069561000 >> 172.21.15.39:0/1069561000 conn(0x7fa3dc0fedc0 msgr2=0x7fa3dc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:36.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.758+0000 7fa3e4509700 1 -- 172.21.15.39:0/1069561000 shutdown_connections 2024-05-04T03:09:36.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.759+0000 7fa3e4509700 1 -- 172.21.15.39:0/1069561000 wait complete. 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.759+0000 7fa3e4509700 1 Processor -- start 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e4509700 1 -- start start 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e4509700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3dc105460 0x7fa3dc103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e4509700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 0x7fa3dc103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e4509700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3dc10e8d0 0x7fa3dc103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e4509700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1142e0 con 0x7fa3dc105db0 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e4509700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc114160 con 0x7fa3dc10e8d0 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e4509700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc114460 con 0x7fa3dc105460 2024-05-04T03:09:36.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e1aa4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 0x7fa3dc103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e1aa4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 0x7fa3dc103690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42030/0 (socket says 172.21.15.39:42030) 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e1aa4700 1 -- 172.21.15.39:0/3223440159 learned_addr learned my addr 172.21.15.39:0/3223440159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e22a5700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3dc105460 0x7fa3dc103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.760+0000 7fa3e2aa6700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3dc10e8d0 0x7fa3dc103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.761+0000 7fa3e1aa4700 1 -- 172.21.15.39:0/3223440159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3dc105460 msgr2=0x7fa3dc103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.761+0000 7fa3e1aa4700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3dc105460 0x7fa3dc103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.761+0000 7fa3e1aa4700 1 -- 172.21.15.39:0/3223440159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3dc10e8d0 msgr2=0x7fa3dc103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.761+0000 7fa3e1aa4700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3dc10e8d0 0x7fa3dc103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.761+0000 7fa3e1aa4700 1 -- 172.21.15.39:0/3223440159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc1a90b0 con 0x7fa3dc105db0 2024-05-04T03:09:36.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.761+0000 7fa3e1aa4700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 0x7fa3dc103690 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000c00 tx=0x7fa3cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:36.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.764+0000 7fa3d37fe700 1 -- 172.21.15.39:0/3223440159 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc027070 con 0x7fa3dc105db0 2024-05-04T03:09:36.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.764+0000 7fa3d37fe700 1 -- 172.21.15.39:0/3223440159 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3cc00fb00 con 0x7fa3dc105db0 2024-05-04T03:09:36.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.764+0000 7fa3d37fe700 1 -- 172.21.15.39:0/3223440159 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc022c40 con 0x7fa3dc105db0 2024-05-04T03:09:36.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.765+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc19aa70 con 0x7fa3dc105db0 2024-05-04T03:09:36.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.765+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc19af00 con 0x7fa3dc105db0 2024-05-04T03:09:36.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.766+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3dc0512c0 con 0x7fa3dc105db0 2024-05-04T03:09:36.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.770+0000 7fa3d37fe700 1 -- 172.21.15.39:0/3223440159 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa3cc02a070 con 0x7fa3dc105db0 2024-05-04T03:09:36.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.771+0000 7fa3d37fe700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3c80b2a60 0x7fa3c80b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:36.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.771+0000 7fa3d37fe700 1 -- 172.21.15.39:0/3223440159 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa3cc0e35b0 con 0x7fa3dc105db0 2024-05-04T03:09:36.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.772+0000 7fa3e22a5700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3c80b2a60 0x7fa3c80b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:36.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.772+0000 7fa3e22a5700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3c80b2a60 0x7fa3c80b4f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8007be0 tx=0x7fa3d80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:36.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.778+0000 7fa3d37fe700 1 -- 172.21.15.39:0/3223440159 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3cc031080 con 0x7fa3dc105db0 2024-05-04T03:09:36.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.800+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f30b419c090 con 0x7f30b41089f0 2024-05-04T03:09:36.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.800+0000 7f30a37fe700 1 -- 172.21.15.39:0/1275654751 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f30ac0ab440 con 0x7f30b41089f0 2024-05-04T03:09:36.801 INFO:teuthology.orchestra.run.smithi039.stdout:201863462972 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f309c0b2910 msgr2=0x7f309c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f309c0b2910 0x7f309c0b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f30b4103b00 tx=0x7f30a4009260 comp rx=0 tx=0).stop 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b41089f0 msgr2=0x7f30b419f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b41089f0 0x7f30b419f560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c8f0 tx=0x7f30ac0082f0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 shutdown_connections 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f309c0b2910 0x7f309c0b4e20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b41089f0 0x7f30b419f560 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f30b41029c0 0x7f30b419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 --2- 172.21.15.39:0/1275654751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30b4103310 0x7f30b419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:36.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 >> 172.21.15.39:0/1275654751 conn(0x7f30b40fe6f0 msgr2=0x7f30b4105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 shutdown_connections 2024-05-04T03:09:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:36.803+0000 7f30b8fa4700 1 -- 172.21.15.39:0/1275654751 wait complete. 2024-05-04T03:09:37.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.017+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa3dc02e5f0 con 0x7fa3dc105db0 2024-05-04T03:09:37.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.017+0000 7fa3d37fe700 1 -- 172.21.15.39:0/3223440159 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa3cc013070 con 0x7fa3dc105db0 2024-05-04T03:09:37.018 INFO:teuthology.orchestra.run.smithi039.stdout:107374182480 2024-05-04T03:09:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3c80b2a60 msgr2=0x7fa3c80b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3c80b2a60 0x7fa3c80b4f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8007be0 tx=0x7fa3d80073d0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 msgr2=0x7fa3dc103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 0x7fa3dc103690 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000c00 tx=0x7fa3cc0047b0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 shutdown_connections 2024-05-04T03:09:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3c80b2a60 0x7fa3c80b4f70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3dc10e8d0 0x7fa3dc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3dc105460 0x7fa3dc103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 --2- 172.21.15.39:0/3223440159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3dc105db0 0x7fa3dc103690 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.019+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 >> 172.21.15.39:0/3223440159 conn(0x7fa3dc0fedc0 msgr2=0x7fa3dc0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.020+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 shutdown_connections 2024-05-04T03:09:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:37.020+0000 7fa3e4509700 1 -- 172.21.15.39:0/3223440159 wait complete. 2024-05-04T03:09:37.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:37 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1275654751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-04T03:09:37.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:37 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3223440159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-04T03:09:37.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:37 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1275654751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-04T03:09:37.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:37 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3223440159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-04T03:09:37.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:37 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1275654751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-04T03:09:37.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:37 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3223440159' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-04T03:09:38.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462967 got 201863462972 for osd.7 2024-05-04T03:09:38.006 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:38.158 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411409 got 77309411414 for osd.2 2024-05-04T03:09:38.158 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:38.265 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:38 smithi039 ceph-mon[29715]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:38.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:38 smithi066 ceph-mon[33264]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:38.593 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182474 got 107374182480 for osd.3 2024-05-04T03:09:38.593 DEBUG:teuthology.parallel:result is None 2024-05-04T03:09:38.594 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-04T03:09:38.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:09:38.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:38 smithi167 ceph-mon[33614]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:39.320 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:40.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:40 smithi039 ceph-mon[29715]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:40.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:40 smithi066 ceph-mon[33264]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:40.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:40 smithi167 ceph-mon[33614]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 -- 172.21.15.39:0/3627840520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 msgr2=0x7f8254104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 --2- 172.21.15.39:0/3627840520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 0x7f8254104a60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f82440099c0 tx=0x7f8244009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 -- 172.21.15.39:0/3627840520 shutdown_connections 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 --2- 172.21.15.39:0/3627840520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 0x7f8254104a60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 --2- 172.21.15.39:0/3627840520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8254109880 0x7f82541111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 --2- 172.21.15.39:0/3627840520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8254108f30 0x7f8254109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 -- 172.21.15.39:0/3627840520 >> 172.21.15.39:0/3627840520 conn(0x7f8254079f00 msgr2=0x7f825407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 -- 172.21.15.39:0/3627840520 shutdown_connections 2024-05-04T03:09:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.040+0000 7f825988d700 1 -- 172.21.15.39:0/3627840520 wait complete. 2024-05-04T03:09:41.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f825988d700 1 Processor -- start 2024-05-04T03:09:41.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f825988d700 1 -- start start 2024-05-04T03:09:41.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f825988d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 0x7f825419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:41.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f825988d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8254108f30 0x7f825419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f825988d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8254109880 0x7f825419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f825988d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8254114380 con 0x7f8254104650 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f825988d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8254114200 con 0x7f8254109880 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f825988d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8254114500 con 0x7f8254108f30 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f8252ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 0x7f825419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f8252ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 0x7f825419ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42038/0 (socket says 172.21.15.39:42038) 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.041+0000 7f8252ffd700 1 -- 172.21.15.39:0/752026325 learned_addr learned my addr 172.21.15.39:0/752026325 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f82527fc700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8254108f30 0x7f825419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f82537fe700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8254109880 0x7f825419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f8252ffd700 1 -- 172.21.15.39:0/752026325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8254108f30 msgr2=0x7f825419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f8252ffd700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8254108f30 0x7f825419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f8252ffd700 1 -- 172.21.15.39:0/752026325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8254109880 msgr2=0x7f825419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f8252ffd700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8254109880 0x7f825419f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f8252ffd700 1 -- 172.21.15.39:0/752026325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825419fb70 con 0x7f8254104650 2024-05-04T03:09:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f82527fc700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8254108f30 0x7f825419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f82537fe700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8254109880 0x7f825419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f8252ffd700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 0x7f825419ad50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8244000c00 tx=0x7f824400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f825888b700 1 -- 172.21.15.39:0/752026325 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400f940 con 0x7f8254104650 2024-05-04T03:09:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825419fe00 con 0x7f8254104650 2024-05-04T03:09:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f825888b700 1 -- 172.21.15.39:0/752026325 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8244004830 con 0x7f8254104650 2024-05-04T03:09:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82541a02d0 con 0x7f8254104650 2024-05-04T03:09:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.042+0000 7f825888b700 1 -- 172.21.15.39:0/752026325 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82440054b0 con 0x7f8254104650 2024-05-04T03:09:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.044+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82540512c0 con 0x7f8254104650 2024-05-04T03:09:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.045+0000 7f825888b700 1 -- 172.21.15.39:0/752026325 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8244029060 con 0x7f8254104650 2024-05-04T03:09:41.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.046+0000 7f825888b700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f823c0b2910 0x7f823c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:41.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.046+0000 7f82527fc700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f823c0b2910 0x7f823c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:41.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.046+0000 7f825888b700 1 -- 172.21.15.39:0/752026325 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f82440e2f60 con 0x7f8254104650 2024-05-04T03:09:41.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.047+0000 7f82527fc700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f823c0b2910 0x7f823c0b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8248005fd0 tx=0x7f8248005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:41.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.051+0000 7f825888b700 1 -- 172.21.15.39:0/752026325 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8244013070 con 0x7f8254104650 2024-05-04T03:09:41.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.283+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f825419c0b0 con 0x7f823c0b2910 2024-05-04T03:09:41.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.290+0000 7f825888b700 1 -- 172.21.15.39:0/752026325 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258932 (secure 0 0 0) 0x7f825419c0b0 con 0x7f823c0b2910 2024-05-04T03:09:41.296 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f823c0b2910 msgr2=0x7f823c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f823c0b2910 0x7f823c0b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8248005fd0 tx=0x7f8248005e30 comp rx=0 tx=0).stop 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 msgr2=0x7f825419ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 0x7f825419ad50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8244000c00 tx=0x7f824400f510 comp rx=0 tx=0).stop 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 shutdown_connections 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f823c0b2910 0x7f823c0b4e20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8254104650 0x7f825419ad50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8254109880 0x7f825419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 --2- 172.21.15.39:0/752026325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8254108f30 0x7f825419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 >> 172.21.15.39:0/752026325 conn(0x7f8254079f00 msgr2=0x7f8254102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 shutdown_connections 2024-05-04T03:09:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:41.294+0000 7f825988d700 1 -- 172.21.15.39:0/752026325 wait complete. 2024-05-04T03:09:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-04T03:09:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-05-04T03:09:41.027227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657762,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1130,"num_read_kb":667,"num_write":369,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376632,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565192,"statfs":{"total":1151940427776,"available":1151554756608,"internally_reserved":0,"allocated":11059200,"data_stored":7068150,"data_compressed":109203,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003574"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247282+0000","last_change":"2024-05-04T03:07:37.070347+0000","last_active":"2024-05-04T03:08:16.247282+0000","last_peered":"2024-05-04T03:08:16.247282+0000","last_clean":"2024-05-04T03:08:16.247282+0000","last_became_active":"2024-05-04T03:07:37.069997+0000","last_became_peered":"2024-05-04T03:07:37.069997+0000","last_unstale":"2024-05-04T03:08:16.247282+0000","last_undegraded":"2024-05-04T03:08:16.247282+0000","last_fullsized":"2024-05-04T03:08:16.247282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:24:11.362228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247260+0000","last_change":"2024-05-04T03:07:37.071598+0000","last_active":"2024-05-04T03:08:16.247260+0000","last_peered":"2024-05-04T03:08:16.247260+0000","last_clean":"2024-05-04T03:08:16.247260+0000","last_became_active":"2024-05-04T03:07:37.071059+0000","last_became_peered":"2024-05-04T03:07:37.071059+0000","last_unstale":"2024-05-04T03:08:16.247260+0000","last_undegraded":"2024-05-04T03:08:16.247260+0000","last_fullsized":"2024-05-04T03:08:16.247260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:33:33.929246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246145+0000","last_change":"2024-05-04T03:07:37.069951+0000","last_active":"2024-05-04T03:08:16.246145+0000","last_peered":"2024-05-04T03:08:16.246145+0000","last_clean":"2024-05-04T03:08:16.246145+0000","last_became_active":"2024-05-04T03:07:37.069558+0000","last_became_peered":"2024-05-04T03:07:37.069558+0000","last_unstale":"2024-05-04T03:08:16.246145+0000","last_undegraded":"2024-05-04T03:08:16.246145+0000","last_fullsized":"2024-05-04T03:08:16.246145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:37:52.596455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367387+0000","last_change":"2024-05-04T03:07:37.068820+0000","last_active":"2024-05-04T03:08:16.367387+0000","last_peered":"2024-05-04T03:08:16.367387+0000","last_clean":"2024-05-04T03:08:16.367387+0000","last_became_active":"2024-05-04T03:07:37.068323+0000","last_became_peered":"2024-05-04T03:07:37.068323+0000","last_unstale":"2024-05-04T03:08:16.367387+0000","last_undegraded":"2024-05-04T03:08:16.367387+0000","last_fullsized":"2024-05-04T03:08:16.367387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:52:54.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245207+0000","last_change":"2024-05-04T03:07:37.061982+0000","last_active":"2024-05-04T03:08:16.245207+0000","last_peered":"2024-05-04T03:08:16.245207+0000","last_clean":"2024-05-04T03:08:16.245207+0000","last_became_active":"2024-05-04T03:07:37.061621+0000","last_became_peered":"2024-05-04T03:07:37.061621+0000","last_unstale":"2024-05-04T03:08:16.245207+0000","last_undegraded":"2024-05-04T03:08:16.245207+0000","last_fullsized":"2024-05-04T03:08:16.245207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:17:47.421011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248166+0000","last_change":"2024-05-04T03:07:37.058641+0000","last_active":"2024-05-04T03:08:16.248166+0000","last_peered":"2024-05-04T03:08:16.248166+0000","last_clean":"2024-05-04T03:08:16.248166+0000","last_became_active":"2024-05-04T03:07:37.058235+0000","last_became_peered":"2024-05-04T03:07:37.058235+0000","last_unstale":"2024-05-04T03:08:16.248166+0000","last_undegraded":"2024-05-04T03:08:16.248166+0000","last_fullsized":"2024-05-04T03:08:16.248166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:17:45.441777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244522+0000","last_change":"2024-05-04T03:07:37.065959+0000","last_active":"2024-05-04T03:08:16.244522+0000","last_peered":"2024-05-04T03:08:16.244522+0000","last_clean":"2024-05-04T03:08:16.244522+0000","last_became_active":"2024-05-04T03:07:37.065508+0000","last_became_peered":"2024-05-04T03:07:37.065508+0000","last_unstale":"2024-05-04T03:08:16.244522+0000","last_undegraded":"2024-05-04T03:08:16.244522+0000","last_fullsized":"2024-05-04T03:08:16.244522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:17.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243478+0000","last_change":"2024-05-04T03:07:37.065620+0000","last_active":"2024-05-04T03:08:16.243478+0000","last_peered":"2024-05-04T03:08:16.243478+0000","last_clean":"2024-05-04T03:08:16.243478+0000","last_became_active":"2024-05-04T03:07:37.065261+0000","last_became_peered":"2024-05-04T03:07:37.065261+0000","last_unstale":"2024-05-04T03:08:16.243478+0000","last_undegraded":"2024-05-04T03:08:16.243478+0000","last_fullsized":"2024-05-04T03:08:16.243478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:27.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242930+0000","last_change":"2024-05-04T03:07:37.072204+0000","last_active":"2024-05-04T03:08:16.242930+0000","last_peered":"2024-05-04T03:08:16.242930+0000","last_clean":"2024-05-04T03:08:16.242930+0000","last_became_active":"2024-05-04T03:07:37.070622+0000","last_became_peered":"2024-05-04T03:07:37.070622+0000","last_unstale":"2024-05-04T03:08:16.242930+0000","last_undegraded":"2024-05-04T03:08:16.242930+0000","last_fullsized":"2024-05-04T03:08:16.242930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:55:25.931369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.253125+0000","last_change":"2024-05-04T03:08:16.253125+0000","last_active":"2024-05-04T03:08:16.253125+0000","last_peered":"2024-05-04T03:08:16.253125+0000","last_clean":"2024-05-04T03:08:16.253125+0000","last_became_active":"2024-05-04T03:08:16.252682+0000","last_became_peered":"2024-05-04T03:08:16.252682+0000","last_unstale":"2024-05-04T03:08:16.253125+0000","last_undegraded":"2024-05-04T03:08:16.253125+0000","last_fullsized":"2024-05-04T03:08:16.253125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:05.933491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244798+0000","last_change":"2024-05-04T03:07:37.071614+0000","last_active":"2024-05-04T03:08:16.244798+0000","last_peered":"2024-05-04T03:08:16.244798+0000","last_clean":"2024-05-04T03:08:16.244798+0000","last_became_active":"2024-05-04T03:07:37.070927+0000","last_became_peered":"2024-05-04T03:07:37.070927+0000","last_unstale":"2024-05-04T03:08:16.244798+0000","last_undegraded":"2024-05-04T03:08:16.244798+0000","last_fullsized":"2024-05-04T03:08:16.244798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:30.432959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367227+0000","last_change":"2024-05-04T03:07:37.069305+0000","last_active":"2024-05-04T03:08:16.367227+0000","last_peered":"2024-05-04T03:08:16.367227+0000","last_clean":"2024-05-04T03:08:16.367227+0000","last_became_active":"2024-05-04T03:07:37.068804+0000","last_became_peered":"2024-05-04T03:07:37.068804+0000","last_unstale":"2024-05-04T03:08:16.367227+0000","last_undegraded":"2024-05-04T03:08:16.367227+0000","last_fullsized":"2024-05-04T03:08:16.367227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:43.693161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248895+0000","last_change":"2024-05-04T03:07:37.062848+0000","last_active":"2024-05-04T03:08:16.248895+0000","last_peered":"2024-05-04T03:08:16.248895+0000","last_clean":"2024-05-04T03:08:16.248895+0000","last_became_active":"2024-05-04T03:07:37.062548+0000","last_became_peered":"2024-05-04T03:07:37.062548+0000","last_unstale":"2024-05-04T03:08:16.248895+0000","last_undegraded":"2024-05-04T03:08:16.248895+0000","last_fullsized":"2024-05-04T03:08:16.248895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246059+0000","last_change":"2024-05-04T03:07:37.062598+0000","last_active":"2024-05-04T03:08:16.246059+0000","last_peered":"2024-05-04T03:08:16.246059+0000","last_clean":"2024-05-04T03:08:16.246059+0000","last_became_active":"2024-05-04T03:07:37.061711+0000","last_became_peered":"2024-05-04T03:07:37.061711+0000","last_unstale":"2024-05-04T03:08:16.246059+0000","last_undegraded":"2024-05-04T03:08:16.246059+0000","last_fullsized":"2024-05-04T03:08:16.246059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:29:03.153680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246387+0000","last_change":"2024-05-04T03:07:37.067889+0000","last_active":"2024-05-04T03:08:16.246387+0000","last_peered":"2024-05-04T03:08:16.246387+0000","last_clean":"2024-05-04T03:08:16.246387+0000","last_became_active":"2024-05-04T03:07:37.066768+0000","last_became_peered":"2024-05-04T03:07:37.066768+0000","last_unstale":"2024-05-04T03:08:16.246387+0000","last_undegraded":"2024-05-04T03:08:16.246387+0000","last_fullsized":"2024-05-04T03:08:16.246387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:01:18.136379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242305+0000","last_change":"2024-05-04T03:07:37.070438+0000","last_active":"2024-05-04T03:08:16.242305+0000","last_peered":"2024-05-04T03:08:16.242305+0000","last_clean":"2024-05-04T03:08:16.242305+0000","last_became_active":"2024-05-04T03:07:37.070007+0000","last_became_peered":"2024-05-04T03:07:37.070007+0000","last_unstale":"2024-05-04T03:08:16.242305+0000","last_undegraded":"2024-05-04T03:08:16.242305+0000","last_fullsized":"2024-05-04T03:08:16.242305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:22.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247556+0000","last_change":"2024-05-04T03:07:37.070020+0000","last_active":"2024-05-04T03:08:16.247556+0000","last_peered":"2024-05-04T03:08:16.247556+0000","last_clean":"2024-05-04T03:08:16.247556+0000","last_became_active":"2024-05-04T03:07:37.069629+0000","last_became_peered":"2024-05-04T03:07:37.069629+0000","last_unstale":"2024-05-04T03:08:16.247556+0000","last_undegraded":"2024-05-04T03:08:16.247556+0000","last_fullsized":"2024-05-04T03:08:16.247556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:28.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245734+0000","last_change":"2024-05-04T03:07:37.068651+0000","last_active":"2024-05-04T03:08:16.245734+0000","last_peered":"2024-05-04T03:08:16.245734+0000","last_clean":"2024-05-04T03:08:16.245734+0000","last_became_active":"2024-05-04T03:07:37.067135+0000","last_became_peered":"2024-05-04T03:07:37.067135+0000","last_unstale":"2024-05-04T03:08:16.245734+0000","last_undegraded":"2024-05-04T03:08:16.245734+0000","last_fullsized":"2024-05-04T03:08:16.245734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:00:59.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250832+0000","last_change":"2024-05-04T03:07:37.055419+0000","last_active":"2024-05-04T03:08:16.250832+0000","last_peered":"2024-05-04T03:08:16.250832+0000","last_clean":"2024-05-04T03:08:16.250832+0000","last_became_active":"2024-05-04T03:07:37.055024+0000","last_became_peered":"2024-05-04T03:07:37.055024+0000","last_unstale":"2024-05-04T03:08:16.250832+0000","last_undegraded":"2024-05-04T03:08:16.250832+0000","last_fullsized":"2024-05-04T03:08:16.250832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:55:05.506505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:38.348088+0000","last_change":"2024-05-04T03:07:34.019975+0000","last_active":"2024-05-04T03:09:38.348088+0000","last_peered":"2024-05-04T03:09:38.348088+0000","last_clean":"2024-05-04T03:09:38.348088+0000","last_became_active":"2024-05-04T03:07:34.019532+0000","last_became_peered":"2024-05-04T03:07:34.019532+0000","last_unstale":"2024-05-04T03:09:38.348088+0000","last_undegraded":"2024-05-04T03:09:38.348088+0000","last_fullsized":"2024-05-04T03:09:38.348088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:11:03.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244135+0000","last_change":"2024-05-04T03:07:37.064725+0000","last_active":"2024-05-04T03:08:16.244135+0000","last_peered":"2024-05-04T03:08:16.244135+0000","last_clean":"2024-05-04T03:08:16.244135+0000","last_became_active":"2024-05-04T03:07:37.064385+0000","last_became_peered":"2024-05-04T03:07:37.064385+0000","last_unstale":"2024-05-04T03:08:16.244135+0000","last_undegraded":"2024-05-04T03:08:16.244135+0000","last_fullsized":"2024-05-04T03:08:16.244135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:22:12.024328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244201+0000","last_change":"2024-05-04T03:07:34.157179+0000","last_active":"2024-05-04T03:08:16.244201+0000","last_peered":"2024-05-04T03:08:16.244201+0000","last_clean":"2024-05-04T03:08:16.244201+0000","last_became_active":"2024-05-04T03:07:34.156790+0000","last_became_peered":"2024-05-04T03:07:34.156790+0000","last_unstale":"2024-05-04T03:08:16.244201+0000","last_undegraded":"2024-05-04T03:08:16.244201+0000","last_fullsized":"2024-05-04T03:08:16.244201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:32.066410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246454+0000","last_change":"2024-05-04T03:07:37.070481+0000","last_active":"2024-05-04T03:08:16.246454+0000","last_peered":"2024-05-04T03:08:16.246454+0000","last_clean":"2024-05-04T03:08:16.246454+0000","last_became_active":"2024-05-04T03:07:37.069903+0000","last_became_peered":"2024-05-04T03:07:37.069903+0000","last_unstale":"2024-05-04T03:08:16.246454+0000","last_undegraded":"2024-05-04T03:08:16.246454+0000","last_fullsized":"2024-05-04T03:08:16.246454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:45:40.764716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245723+0000","last_change":"2024-05-04T03:07:34.023978+0000","last_active":"2024-05-04T03:08:16.245723+0000","last_peered":"2024-05-04T03:08:16.245723+0000","last_clean":"2024-05-04T03:08:16.245723+0000","last_became_active":"2024-05-04T03:07:34.023199+0000","last_became_peered":"2024-05-04T03:07:34.023199+0000","last_unstale":"2024-05-04T03:08:16.245723+0000","last_undegraded":"2024-05-04T03:08:16.245723+0000","last_fullsized":"2024-05-04T03:08:16.245723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:17:43.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242450+0000","last_change":"2024-05-04T03:07:37.072661+0000","last_active":"2024-05-04T03:08:16.242450+0000","last_peered":"2024-05-04T03:08:16.242450+0000","last_clean":"2024-05-04T03:08:16.242450+0000","last_became_active":"2024-05-04T03:07:37.071393+0000","last_became_peered":"2024-05-04T03:07:37.071393+0000","last_unstale":"2024-05-04T03:08:16.242450+0000","last_undegraded":"2024-05-04T03:08:16.242450+0000","last_fullsized":"2024-05-04T03:08:16.242450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:39:39.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245493+0000","last_change":"2024-05-04T03:07:34.018543+0000","last_active":"2024-05-04T03:08:16.245493+0000","last_peered":"2024-05-04T03:08:16.245493+0000","last_clean":"2024-05-04T03:08:16.245493+0000","last_became_active":"2024-05-04T03:07:34.018145+0000","last_became_peered":"2024-05-04T03:07:34.018145+0000","last_unstale":"2024-05-04T03:08:16.245493+0000","last_undegraded":"2024-05-04T03:08:16.245493+0000","last_fullsized":"2024-05-04T03:08:16.245493+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:44.529492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248677+0000","last_change":"2024-05-04T03:07:37.070811+0000","last_active":"2024-05-04T03:08:16.248677+0000","last_peered":"2024-05-04T03:08:16.248677+0000","last_clean":"2024-05-04T03:08:16.248677+0000","last_became_active":"2024-05-04T03:07:37.070441+0000","last_became_peered":"2024-05-04T03:07:37.070441+0000","last_unstale":"2024-05-04T03:08:16.248677+0000","last_undegraded":"2024-05-04T03:08:16.248677+0000","last_fullsized":"2024-05-04T03:08:16.248677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:01:48.146339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:39.328681+0000","last_change":"2024-05-04T03:07:34.020508+0000","last_active":"2024-05-04T03:09:39.328681+0000","last_peered":"2024-05-04T03:09:39.328681+0000","last_clean":"2024-05-04T03:09:39.328681+0000","last_became_active":"2024-05-04T03:07:34.020076+0000","last_became_peered":"2024-05-04T03:07:34.020076+0000","last_unstale":"2024-05-04T03:09:39.328681+0000","last_undegraded":"2024-05-04T03:09:39.328681+0000","last_fullsized":"2024-05-04T03:09:39.328681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:27.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245429+0000","last_change":"2024-05-04T03:07:37.060004+0000","last_active":"2024-05-04T03:08:16.245429+0000","last_peered":"2024-05-04T03:08:16.245429+0000","last_clean":"2024-05-04T03:08:16.245429+0000","last_became_active":"2024-05-04T03:07:37.059350+0000","last_became_peered":"2024-05-04T03:07:37.059350+0000","last_unstale":"2024-05-04T03:08:16.245429+0000","last_undegraded":"2024-05-04T03:08:16.245429+0000","last_fullsized":"2024-05-04T03:08:16.245429+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:37:07.315935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248068+0000","last_change":"2024-05-04T03:07:34.020174+0000","last_active":"2024-05-04T03:08:16.248068+0000","last_peered":"2024-05-04T03:08:16.248068+0000","last_clean":"2024-05-04T03:08:16.248068+0000","last_became_active":"2024-05-04T03:07:34.019790+0000","last_became_peered":"2024-05-04T03:07:34.019790+0000","last_unstale":"2024-05-04T03:08:16.248068+0000","last_undegraded":"2024-05-04T03:08:16.248068+0000","last_fullsized":"2024-05-04T03:08:16.248068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:37:37.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.252143+0000","last_change":"2024-05-04T03:08:16.251277+0000","last_active":"2024-05-04T03:08:16.252143+0000","last_peered":"2024-05-04T03:08:16.252143+0000","last_clean":"2024-05-04T03:08:16.252143+0000","last_became_active":"2024-05-04T03:08:16.250836+0000","last_became_peered":"2024-05-04T03:08:16.250836+0000","last_unstale":"2024-05-04T03:08:16.252143+0000","last_undegraded":"2024-05-04T03:08:16.252143+0000","last_fullsized":"2024-05-04T03:08:16.252143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:59:47.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248397+0000","last_change":"2024-05-04T03:07:34.013188+0000","last_active":"2024-05-04T03:08:16.248397+0000","last_peered":"2024-05-04T03:08:16.248397+0000","last_clean":"2024-05-04T03:08:16.248397+0000","last_became_active":"2024-05-04T03:07:34.012759+0000","last_became_peered":"2024-05-04T03:07:34.012759+0000","last_unstale":"2024-05-04T03:08:16.248397+0000","last_undegraded":"2024-05-04T03:08:16.248397+0000","last_fullsized":"2024-05-04T03:08:16.248397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:54:29.621250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243425+0000","last_change":"2024-05-04T03:07:37.056265+0000","last_active":"2024-05-04T03:08:16.243425+0000","last_peered":"2024-05-04T03:08:16.243425+0000","last_clean":"2024-05-04T03:08:16.243425+0000","last_became_active":"2024-05-04T03:07:37.055957+0000","last_became_peered":"2024-05-04T03:07:37.055957+0000","last_unstale":"2024-05-04T03:08:16.243425+0000","last_undegraded":"2024-05-04T03:08:16.243425+0000","last_fullsized":"2024-05-04T03:08:16.243425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:41.739034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247855+0000","last_change":"2024-05-04T03:07:34.020722+0000","last_active":"2024-05-04T03:08:16.247855+0000","last_peered":"2024-05-04T03:08:16.247855+0000","last_clean":"2024-05-04T03:08:16.247855+0000","last_became_active":"2024-05-04T03:07:34.020028+0000","last_became_peered":"2024-05-04T03:07:34.020028+0000","last_unstale":"2024-05-04T03:08:16.247855+0000","last_undegraded":"2024-05-04T03:08:16.247855+0000","last_fullsized":"2024-05-04T03:08:16.247855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:23:20.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245364+0000","last_change":"2024-05-04T03:07:37.059680+0000","last_active":"2024-05-04T03:08:16.245364+0000","last_peered":"2024-05-04T03:08:16.245364+0000","last_clean":"2024-05-04T03:08:16.245364+0000","last_became_active":"2024-05-04T03:07:37.059294+0000","last_became_peered":"2024-05-04T03:07:37.059294+0000","last_unstale":"2024-05-04T03:08:16.245364+0000","last_undegraded":"2024-05-04T03:08:16.245364+0000","last_fullsized":"2024-05-04T03:08:16.245364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:50:06.251706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":236,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:40.329512+0000","last_change":"2024-05-04T03:07:34.019055+0000","last_active":"2024-05-04T03:09:40.329512+0000","last_peered":"2024-05-04T03:09:40.329512+0000","last_clean":"2024-05-04T03:09:40.329512+0000","last_became_active":"2024-05-04T03:07:34.018638+0000","last_became_peered":"2024-05-04T03:07:34.018638+0000","last_unstale":"2024-05-04T03:09:40.329512+0000","last_undegraded":"2024-05-04T03:09:40.329512+0000","last_fullsized":"2024-05-04T03:09:40.329512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:58:01.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242844+0000","last_change":"2024-05-04T03:07:37.063037+0000","last_active":"2024-05-04T03:08:16.242844+0000","last_peered":"2024-05-04T03:08:16.242844+0000","last_clean":"2024-05-04T03:08:16.242844+0000","last_became_active":"2024-05-04T03:07:37.062350+0000","last_became_peered":"2024-05-04T03:07:37.062350+0000","last_unstale":"2024-05-04T03:08:16.242844+0000","last_undegraded":"2024-05-04T03:08:16.242844+0000","last_fullsized":"2024-05-04T03:08:16.242844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:41:15.898241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.373765+0000","last_change":"2024-05-04T03:08:16.373765+0000","last_active":"2024-05-04T03:08:16.373765+0000","last_peered":"2024-05-04T03:08:16.373765+0000","last_clean":"2024-05-04T03:08:16.373765+0000","last_became_active":"2024-05-04T03:08:16.368588+0000","last_became_peered":"2024-05-04T03:08:16.368588+0000","last_unstale":"2024-05-04T03:08:16.373765+0000","last_undegraded":"2024-05-04T03:08:16.373765+0000","last_fullsized":"2024-05-04T03:08:16.373765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:55:07.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246604+0000","last_change":"2024-05-04T03:07:37.071330+0000","last_active":"2024-05-04T03:08:16.246604+0000","last_peered":"2024-05-04T03:08:16.246604+0000","last_clean":"2024-05-04T03:08:16.246604+0000","last_became_active":"2024-05-04T03:07:37.070743+0000","last_became_peered":"2024-05-04T03:07:37.070743+0000","last_unstale":"2024-05-04T03:08:16.246604+0000","last_undegraded":"2024-05-04T03:08:16.246604+0000","last_fullsized":"2024-05-04T03:08:16.246604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:10:00.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248817+0000","last_change":"2024-05-04T03:07:34.157432+0000","last_active":"2024-05-04T03:08:16.248817+0000","last_peered":"2024-05-04T03:08:16.248817+0000","last_clean":"2024-05-04T03:08:16.248817+0000","last_became_active":"2024-05-04T03:07:34.156976+0000","last_became_peered":"2024-05-04T03:07:34.156976+0000","last_unstale":"2024-05-04T03:08:16.248817+0000","last_undegraded":"2024-05-04T03:08:16.248817+0000","last_fullsized":"2024-05-04T03:08:16.248817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:56.082287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242517+0000","last_change":"2024-05-04T03:07:37.064301+0000","last_active":"2024-05-04T03:08:16.242517+0000","last_peered":"2024-05-04T03:08:16.242517+0000","last_clean":"2024-05-04T03:08:16.242517+0000","last_became_active":"2024-05-04T03:07:37.063691+0000","last_became_peered":"2024-05-04T03:07:37.063691+0000","last_unstale":"2024-05-04T03:08:16.242517+0000","last_undegraded":"2024-05-04T03:08:16.242517+0000","last_fullsized":"2024-05-04T03:08:16.242517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:22:13.844703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":272,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:40.329521+0000","last_change":"2024-05-04T03:07:34.021595+0000","last_active":"2024-05-04T03:09:40.329521+0000","last_peered":"2024-05-04T03:09:40.329521+0000","last_clean":"2024-05-04T03:09:40.329521+0000","last_became_active":"2024-05-04T03:07:34.021126+0000","last_became_peered":"2024-05-04T03:07:34.021126+0000","last_unstale":"2024-05-04T03:09:40.329521+0000","last_undegraded":"2024-05-04T03:09:40.329521+0000","last_fullsized":"2024-05-04T03:09:40.329521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:55.094800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":215,"num_read_kb":166,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245497+0000","last_change":"2024-05-04T03:07:37.066365+0000","last_active":"2024-05-04T03:08:16.245497+0000","last_peered":"2024-05-04T03:08:16.245497+0000","last_clean":"2024-05-04T03:08:16.245497+0000","last_became_active":"2024-05-04T03:07:37.065851+0000","last_became_peered":"2024-05-04T03:07:37.065851+0000","last_unstale":"2024-05-04T03:08:16.245497+0000","last_undegraded":"2024-05-04T03:08:16.245497+0000","last_fullsized":"2024-05-04T03:08:16.245497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:03:43.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.263114+0000","last_change":"2024-05-04T03:08:16.262600+0000","last_active":"2024-05-04T03:08:16.263114+0000","last_peered":"2024-05-04T03:08:16.263114+0000","last_clean":"2024-05-04T03:08:16.263114+0000","last_became_active":"2024-05-04T03:08:16.250619+0000","last_became_peered":"2024-05-04T03:08:16.250619+0000","last_unstale":"2024-05-04T03:08:16.263114+0000","last_undegraded":"2024-05-04T03:08:16.263114+0000","last_fullsized":"2024-05-04T03:08:16.263114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:51:21.596467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248552+0000","last_change":"2024-05-04T03:07:37.055356+0000","last_active":"2024-05-04T03:08:16.248552+0000","last_peered":"2024-05-04T03:08:16.248552+0000","last_clean":"2024-05-04T03:08:16.248552+0000","last_became_active":"2024-05-04T03:07:37.055045+0000","last_became_peered":"2024-05-04T03:07:37.055045+0000","last_unstale":"2024-05-04T03:08:16.248552+0000","last_undegraded":"2024-05-04T03:08:16.248552+0000","last_fullsized":"2024-05-04T03:08:16.248552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:06.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247382+0000","last_change":"2024-05-04T03:07:34.021525+0000","last_active":"2024-05-04T03:08:16.247382+0000","last_peered":"2024-05-04T03:08:16.247382+0000","last_clean":"2024-05-04T03:08:16.247382+0000","last_became_active":"2024-05-04T03:07:34.021042+0000","last_became_peered":"2024-05-04T03:07:34.021042+0000","last_unstale":"2024-05-04T03:08:16.247382+0000","last_undegraded":"2024-05-04T03:08:16.247382+0000","last_fullsized":"2024-05-04T03:08:16.247382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:44.090176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244794+0000","last_change":"2024-05-04T03:07:37.066471+0000","last_active":"2024-05-04T03:08:16.244794+0000","last_peered":"2024-05-04T03:08:16.244794+0000","last_clean":"2024-05-04T03:08:16.244794+0000","last_became_active":"2024-05-04T03:07:37.066059+0000","last_became_peered":"2024-05-04T03:07:37.066059+0000","last_unstale":"2024-05-04T03:08:16.244794+0000","last_undegraded":"2024-05-04T03:08:16.244794+0000","last_fullsized":"2024-05-04T03:08:16.244794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:16.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":134,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:36.323547+0000","last_change":"2024-05-04T03:07:34.021531+0000","last_active":"2024-05-04T03:09:36.323547+0000","last_peered":"2024-05-04T03:09:36.323547+0000","last_clean":"2024-05-04T03:09:36.323547+0000","last_became_active":"2024-05-04T03:07:34.021112+0000","last_became_peered":"2024-05-04T03:07:34.021112+0000","last_unstale":"2024-05-04T03:09:36.323547+0000","last_undegraded":"2024-05-04T03:09:36.323547+0000","last_fullsized":"2024-05-04T03:09:36.323547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:28.826426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244611+0000","last_change":"2024-05-04T03:07:37.067155+0000","last_active":"2024-05-04T03:08:16.244611+0000","last_peered":"2024-05-04T03:08:16.244611+0000","last_clean":"2024-05-04T03:08:16.244611+0000","last_became_active":"2024-05-04T03:07:37.066783+0000","last_became_peered":"2024-05-04T03:07:37.066783+0000","last_unstale":"2024-05-04T03:08:16.244611+0000","last_undegraded":"2024-05-04T03:08:16.244611+0000","last_fullsized":"2024-05-04T03:08:16.244611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:23.217930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245013+0000","last_change":"2024-05-04T03:07:34.158227+0000","last_active":"2024-05-04T03:08:16.245013+0000","last_peered":"2024-05-04T03:08:16.245013+0000","last_clean":"2024-05-04T03:08:16.245013+0000","last_became_active":"2024-05-04T03:07:34.156905+0000","last_became_peered":"2024-05-04T03:07:34.156905+0000","last_unstale":"2024-05-04T03:08:16.245013+0000","last_undegraded":"2024-05-04T03:08:16.245013+0000","last_fullsized":"2024-05-04T03:08:16.245013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:40:41.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247601+0000","last_change":"2024-05-04T03:07:37.075559+0000","last_active":"2024-05-04T03:08:16.247601+0000","last_peered":"2024-05-04T03:08:16.247601+0000","last_clean":"2024-05-04T03:08:16.247601+0000","last_became_active":"2024-05-04T03:07:37.075220+0000","last_became_peered":"2024-05-04T03:07:37.075220+0000","last_unstale":"2024-05-04T03:08:16.247601+0000","last_undegraded":"2024-05-04T03:08:16.247601+0000","last_fullsized":"2024-05-04T03:08:16.247601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:51:23.942514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246548+0000","last_change":"2024-05-04T03:07:34.020326+0000","last_active":"2024-05-04T03:08:16.246548+0000","last_peered":"2024-05-04T03:08:16.246548+0000","last_clean":"2024-05-04T03:08:16.246548+0000","last_became_active":"2024-05-04T03:07:34.019472+0000","last_became_peered":"2024-05-04T03:07:34.019472+0000","last_unstale":"2024-05-04T03:08:16.246548+0000","last_undegraded":"2024-05-04T03:08:16.246548+0000","last_fullsized":"2024-05-04T03:08:16.246548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:33:28.375901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247007+0000","last_change":"2024-05-04T03:07:37.066223+0000","last_active":"2024-05-04T03:08:16.247007+0000","last_peered":"2024-05-04T03:08:16.247007+0000","last_clean":"2024-05-04T03:08:16.247007+0000","last_became_active":"2024-05-04T03:07:37.065859+0000","last_became_peered":"2024-05-04T03:07:37.065859+0000","last_unstale":"2024-05-04T03:08:16.247007+0000","last_undegraded":"2024-05-04T03:08:16.247007+0000","last_fullsized":"2024-05-04T03:08:16.247007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:44.902986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245539+0000","last_change":"2024-05-04T03:07:34.158674+0000","last_active":"2024-05-04T03:08:16.245539+0000","last_peered":"2024-05-04T03:08:16.245539+0000","last_clean":"2024-05-04T03:08:16.245539+0000","last_became_active":"2024-05-04T03:07:34.157952+0000","last_became_peered":"2024-05-04T03:07:34.157952+0000","last_unstale":"2024-05-04T03:08:16.245539+0000","last_undegraded":"2024-05-04T03:08:16.245539+0000","last_fullsized":"2024-05-04T03:08:16.245539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:19:11.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248252+0000","last_change":"2024-05-04T03:07:37.053897+0000","last_active":"2024-05-04T03:08:16.248252+0000","last_peered":"2024-05-04T03:08:16.248252+0000","last_clean":"2024-05-04T03:08:16.248252+0000","last_became_active":"2024-05-04T03:07:37.053466+0000","last_became_peered":"2024-05-04T03:07:37.053466+0000","last_unstale":"2024-05-04T03:08:16.248252+0000","last_undegraded":"2024-05-04T03:08:16.248252+0000","last_fullsized":"2024-05-04T03:08:16.248252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:50.650805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":131,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:35.343984+0000","last_change":"2024-05-04T03:07:34.021494+0000","last_active":"2024-05-04T03:09:35.343984+0000","last_peered":"2024-05-04T03:09:35.343984+0000","last_clean":"2024-05-04T03:09:35.343984+0000","last_became_active":"2024-05-04T03:07:34.021113+0000","last_became_peered":"2024-05-04T03:07:34.021113+0000","last_unstale":"2024-05-04T03:09:35.343984+0000","last_undegraded":"2024-05-04T03:09:35.343984+0000","last_fullsized":"2024-05-04T03:09:35.343984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:02:09.514961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248164+0000","last_change":"2024-05-04T03:07:37.068199+0000","last_active":"2024-05-04T03:08:16.248164+0000","last_peered":"2024-05-04T03:08:16.248164+0000","last_clean":"2024-05-04T03:08:16.248164+0000","last_became_active":"2024-05-04T03:07:37.067828+0000","last_became_peered":"2024-05-04T03:07:37.067828+0000","last_unstale":"2024-05-04T03:08:16.248164+0000","last_undegraded":"2024-05-04T03:08:16.248164+0000","last_fullsized":"2024-05-04T03:08:16.248164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:28.882920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248172+0000","last_change":"2024-05-04T03:07:37.069753+0000","last_active":"2024-05-04T03:08:16.248172+0000","last_peered":"2024-05-04T03:08:16.248172+0000","last_clean":"2024-05-04T03:08:16.248172+0000","last_became_active":"2024-05-04T03:07:37.069338+0000","last_became_peered":"2024-05-04T03:07:37.069338+0000","last_unstale":"2024-05-04T03:08:16.248172+0000","last_undegraded":"2024-05-04T03:08:16.248172+0000","last_fullsized":"2024-05-04T03:08:16.248172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:37:11.355749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":131,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:35.321763+0000","last_change":"2024-05-04T03:07:34.157229+0000","last_active":"2024-05-04T03:09:35.321763+0000","last_peered":"2024-05-04T03:09:35.321763+0000","last_clean":"2024-05-04T03:09:35.321763+0000","last_became_active":"2024-05-04T03:07:34.156776+0000","last_became_peered":"2024-05-04T03:07:34.156776+0000","last_unstale":"2024-05-04T03:09:35.321763+0000","last_undegraded":"2024-05-04T03:09:35.321763+0000","last_fullsized":"2024-05-04T03:09:35.321763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:26.879754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243715+0000","last_change":"2024-05-04T03:07:37.067025+0000","last_active":"2024-05-04T03:08:16.243715+0000","last_peered":"2024-05-04T03:08:16.243715+0000","last_clean":"2024-05-04T03:08:16.243715+0000","last_became_active":"2024-05-04T03:07:37.066359+0000","last_became_peered":"2024-05-04T03:07:37.066359+0000","last_unstale":"2024-05-04T03:08:16.243715+0000","last_undegraded":"2024-05-04T03:08:16.243715+0000","last_fullsized":"2024-05-04T03:08:16.243715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:44:35.604859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367108+0000","last_change":"2024-05-04T03:07:34.018019+0000","last_active":"2024-05-04T03:08:16.367108+0000","last_peered":"2024-05-04T03:08:16.367108+0000","last_clean":"2024-05-04T03:08:16.367108+0000","last_became_active":"2024-05-04T03:07:34.017556+0000","last_became_peered":"2024-05-04T03:07:34.017556+0000","last_unstale":"2024-05-04T03:08:16.367108+0000","last_undegraded":"2024-05-04T03:08:16.367108+0000","last_fullsized":"2024-05-04T03:08:16.367108+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:31:34.158164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244533+0000","last_change":"2024-05-04T03:06:06.704755+0000","last_active":"2024-05-04T03:08:16.244533+0000","last_peered":"2024-05-04T03:08:16.244533+0000","last_clean":"2024-05-04T03:08:16.244533+0000","last_became_active":"2024-05-04T03:06:06.698747+0000","last_became_peered":"2024-05-04T03:06:06.698747+0000","last_unstale":"2024-05-04T03:08:16.244533+0000","last_undegraded":"2024-05-04T03:08:16.244533+0000","last_fullsized":"2024-05-04T03:08:16.244533+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_clean_scrub_stamp":"2024-05-04T03:02:43.385377+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:04:50.012414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248266+0000","last_change":"2024-05-04T03:07:37.057043+0000","last_active":"2024-05-04T03:08:16.248266+0000","last_peered":"2024-05-04T03:08:16.248266+0000","last_clean":"2024-05-04T03:08:16.248266+0000","last_became_active":"2024-05-04T03:07:37.056725+0000","last_became_peered":"2024-05-04T03:07:37.056725+0000","last_unstale":"2024-05-04T03:08:16.248266+0000","last_undegraded":"2024-05-04T03:08:16.248266+0000","last_fullsized":"2024-05-04T03:08:16.248266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:13:28.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.377000+0000","last_change":"2024-05-04T03:08:16.376510+0000","last_active":"2024-05-04T03:08:16.377000+0000","last_peered":"2024-05-04T03:08:16.377000+0000","last_clean":"2024-05-04T03:08:16.377000+0000","last_became_active":"2024-05-04T03:08:16.368625+0000","last_became_peered":"2024-05-04T03:08:16.368625+0000","last_unstale":"2024-05-04T03:08:16.377000+0000","last_undegraded":"2024-05-04T03:08:16.377000+0000","last_fullsized":"2024-05-04T03:08:16.377000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:27:50.504487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245592+0000","last_change":"2024-05-04T03:07:34.157346+0000","last_active":"2024-05-04T03:08:16.245592+0000","last_peered":"2024-05-04T03:08:16.245592+0000","last_clean":"2024-05-04T03:08:16.245592+0000","last_became_active":"2024-05-04T03:07:34.156914+0000","last_became_peered":"2024-05-04T03:07:34.156914+0000","last_unstale":"2024-05-04T03:08:16.245592+0000","last_undegraded":"2024-05-04T03:08:16.245592+0000","last_fullsized":"2024-05-04T03:08:16.245592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:54:17.059644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248699+0000","last_change":"2024-05-04T03:07:37.054373+0000","last_active":"2024-05-04T03:08:16.248699+0000","last_peered":"2024-05-04T03:08:16.248699+0000","last_clean":"2024-05-04T03:08:16.248699+0000","last_became_active":"2024-05-04T03:07:37.053764+0000","last_became_peered":"2024-05-04T03:07:37.053764+0000","last_unstale":"2024-05-04T03:08:16.248699+0000","last_undegraded":"2024-05-04T03:08:16.248699+0000","last_fullsized":"2024-05-04T03:08:16.248699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:13:47.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:21.079361+0000","last_change":"2024-05-04T03:07:34.020190+0000","last_active":"2024-05-04T03:08:21.079361+0000","last_peered":"2024-05-04T03:08:21.079361+0000","last_clean":"2024-05-04T03:08:21.079361+0000","last_became_active":"2024-05-04T03:07:34.019786+0000","last_became_peered":"2024-05-04T03:07:34.019786+0000","last_unstale":"2024-05-04T03:08:21.079361+0000","last_undegraded":"2024-05-04T03:08:21.079361+0000","last_fullsized":"2024-05-04T03:08:21.079361+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:11.243414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244079+0000","last_change":"2024-05-04T03:07:37.055274+0000","last_active":"2024-05-04T03:08:16.244079+0000","last_peered":"2024-05-04T03:08:16.244079+0000","last_clean":"2024-05-04T03:08:16.244079+0000","last_became_active":"2024-05-04T03:07:37.054968+0000","last_became_peered":"2024-05-04T03:07:37.054968+0000","last_unstale":"2024-05-04T03:08:16.244079+0000","last_undegraded":"2024-05-04T03:08:16.244079+0000","last_fullsized":"2024-05-04T03:08:16.244079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.229266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367455+0000","last_change":"2024-05-04T03:07:34.132966+0000","last_active":"2024-05-04T03:08:16.367455+0000","last_peered":"2024-05-04T03:08:16.367455+0000","last_clean":"2024-05-04T03:08:16.367455+0000","last_became_active":"2024-05-04T03:07:34.132529+0000","last_became_peered":"2024-05-04T03:07:34.132529+0000","last_unstale":"2024-05-04T03:08:16.367455+0000","last_undegraded":"2024-05-04T03:08:16.367455+0000","last_fullsized":"2024-05-04T03:08:16.367455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:31:37.122097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.368747+0000","last_change":"2024-05-04T03:08:16.368747+0000","last_active":"2024-05-04T03:08:16.368747+0000","last_peered":"2024-05-04T03:08:16.368747+0000","last_clean":"2024-05-04T03:08:16.368747+0000","last_became_active":"2024-05-04T03:08:16.368376+0000","last_became_peered":"2024-05-04T03:08:16.368376+0000","last_unstale":"2024-05-04T03:08:16.368747+0000","last_undegraded":"2024-05-04T03:08:16.368747+0000","last_fullsized":"2024-05-04T03:08:16.368747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:42:32.085650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242445+0000","last_change":"2024-05-04T03:07:34.157219+0000","last_active":"2024-05-04T03:08:16.242445+0000","last_peered":"2024-05-04T03:08:16.242445+0000","last_clean":"2024-05-04T03:08:16.242445+0000","last_became_active":"2024-05-04T03:07:34.156866+0000","last_became_peered":"2024-05-04T03:07:34.156866+0000","last_unstale":"2024-05-04T03:08:16.242445+0000","last_undegraded":"2024-05-04T03:08:16.242445+0000","last_fullsized":"2024-05-04T03:08:16.242445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:15:34.138825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245287+0000","last_change":"2024-05-04T03:07:37.058733+0000","last_active":"2024-05-04T03:08:16.245287+0000","last_peered":"2024-05-04T03:08:16.245287+0000","last_clean":"2024-05-04T03:08:16.245287+0000","last_became_active":"2024-05-04T03:07:37.058396+0000","last_became_peered":"2024-05-04T03:07:37.058396+0000","last_unstale":"2024-05-04T03:08:16.245287+0000","last_undegraded":"2024-05-04T03:08:16.245287+0000","last_fullsized":"2024-05-04T03:08:16.245287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:35.563728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243514+0000","last_change":"2024-05-04T03:07:34.016458+0000","last_active":"2024-05-04T03:08:16.243514+0000","last_peered":"2024-05-04T03:08:16.243514+0000","last_clean":"2024-05-04T03:08:16.243514+0000","last_became_active":"2024-05-04T03:07:34.016038+0000","last_became_peered":"2024-05-04T03:07:34.016038+0000","last_unstale":"2024-05-04T03:08:16.243514+0000","last_undegraded":"2024-05-04T03:08:16.243514+0000","last_fullsized":"2024-05-04T03:08:16.243514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:39:25.156047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247376+0000","last_change":"2024-05-04T03:07:37.064024+0000","last_active":"2024-05-04T03:08:16.247376+0000","last_peered":"2024-05-04T03:08:16.247376+0000","last_clean":"2024-05-04T03:08:16.247376+0000","last_became_active":"2024-05-04T03:07:37.063642+0000","last_became_peered":"2024-05-04T03:07:37.063642+0000","last_unstale":"2024-05-04T03:08:16.247376+0000","last_undegraded":"2024-05-04T03:08:16.247376+0000","last_fullsized":"2024-05-04T03:08:16.247376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:59:58.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.267243+0000","last_change":"2024-05-04T03:08:16.266763+0000","last_active":"2024-05-04T03:08:16.267243+0000","last_peered":"2024-05-04T03:08:16.267243+0000","last_clean":"2024-05-04T03:08:16.267243+0000","last_became_active":"2024-05-04T03:08:16.251212+0000","last_became_peered":"2024-05-04T03:08:16.251212+0000","last_unstale":"2024-05-04T03:08:16.267243+0000","last_undegraded":"2024-05-04T03:08:16.267243+0000","last_fullsized":"2024-05-04T03:08:16.267243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:31:27.444610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243667+0000","last_change":"2024-05-04T03:07:37.063753+0000","last_active":"2024-05-04T03:08:16.243667+0000","last_peered":"2024-05-04T03:08:16.243667+0000","last_clean":"2024-05-04T03:08:16.243667+0000","last_became_active":"2024-05-04T03:07:37.063318+0000","last_became_peered":"2024-05-04T03:07:37.063318+0000","last_unstale":"2024-05-04T03:08:16.243667+0000","last_undegraded":"2024-05-04T03:08:16.243667+0000","last_fullsized":"2024-05-04T03:08:16.243667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:48.217136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245115+0000","last_change":"2024-05-04T03:07:34.022522+0000","last_active":"2024-05-04T03:08:16.245115+0000","last_peered":"2024-05-04T03:08:16.245115+0000","last_clean":"2024-05-04T03:08:16.245115+0000","last_became_active":"2024-05-04T03:07:34.021018+0000","last_became_peered":"2024-05-04T03:07:34.021018+0000","last_unstale":"2024-05-04T03:08:16.245115+0000","last_undegraded":"2024-05-04T03:08:16.245115+0000","last_fullsized":"2024-05-04T03:08:16.245115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:47:04.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244631+0000","last_change":"2024-05-04T03:07:37.069930+0000","last_active":"2024-05-04T03:08:16.244631+0000","last_peered":"2024-05-04T03:08:16.244631+0000","last_clean":"2024-05-04T03:08:16.244631+0000","last_became_active":"2024-05-04T03:07:37.069580+0000","last_became_peered":"2024-05-04T03:07:37.069580+0000","last_unstale":"2024-05-04T03:08:16.244631+0000","last_undegraded":"2024-05-04T03:08:16.244631+0000","last_fullsized":"2024-05-04T03:08:16.244631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:15:33.815970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":132,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:36.389635+0000","last_change":"2024-05-04T03:07:34.157969+0000","last_active":"2024-05-04T03:09:36.389635+0000","last_peered":"2024-05-04T03:09:36.389635+0000","last_clean":"2024-05-04T03:09:36.389635+0000","last_became_active":"2024-05-04T03:07:34.157563+0000","last_became_peered":"2024-05-04T03:07:34.157563+0000","last_unstale":"2024-05-04T03:09:36.389635+0000","last_undegraded":"2024-05-04T03:09:36.389635+0000","last_fullsized":"2024-05-04T03:09:36.389635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:00:43.562109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245064+0000","last_change":"2024-05-04T03:07:37.071279+0000","last_active":"2024-05-04T03:08:16.245064+0000","last_peered":"2024-05-04T03:08:16.245064+0000","last_clean":"2024-05-04T03:08:16.245064+0000","last_became_active":"2024-05-04T03:07:37.070449+0000","last_became_peered":"2024-05-04T03:07:37.070449+0000","last_unstale":"2024-05-04T03:08:16.245064+0000","last_undegraded":"2024-05-04T03:08:16.245064+0000","last_fullsized":"2024-05-04T03:08:16.245064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:13:22.535788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248538+0000","last_change":"2024-05-04T03:07:34.017010+0000","last_active":"2024-05-04T03:08:16.248538+0000","last_peered":"2024-05-04T03:08:16.248538+0000","last_clean":"2024-05-04T03:08:16.248538+0000","last_became_active":"2024-05-04T03:07:34.016521+0000","last_became_peered":"2024-05-04T03:07:34.016521+0000","last_unstale":"2024-05-04T03:08:16.248538+0000","last_undegraded":"2024-05-04T03:08:16.248538+0000","last_fullsized":"2024-05-04T03:08:16.248538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:36.684656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245167+0000","last_change":"2024-05-04T03:07:37.066526+0000","last_active":"2024-05-04T03:08:16.245167+0000","last_peered":"2024-05-04T03:08:16.245167+0000","last_clean":"2024-05-04T03:08:16.245167+0000","last_became_active":"2024-05-04T03:07:37.065312+0000","last_became_peered":"2024-05-04T03:07:37.065312+0000","last_unstale":"2024-05-04T03:08:16.245167+0000","last_undegraded":"2024-05-04T03:08:16.245167+0000","last_fullsized":"2024-05-04T03:08:16.245167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:10.077295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248927+0000","last_change":"2024-05-04T03:07:34.016898+0000","last_active":"2024-05-04T03:08:16.248927+0000","last_peered":"2024-05-04T03:08:16.248927+0000","last_clean":"2024-05-04T03:08:16.248927+0000","last_became_active":"2024-05-04T03:07:34.016524+0000","last_became_peered":"2024-05-04T03:07:34.016524+0000","last_unstale":"2024-05-04T03:08:16.248927+0000","last_undegraded":"2024-05-04T03:08:16.248927+0000","last_fullsized":"2024-05-04T03:08:16.248927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:01.236207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245122+0000","last_change":"2024-05-04T03:07:37.061501+0000","last_active":"2024-05-04T03:08:16.245122+0000","last_peered":"2024-05-04T03:08:16.245122+0000","last_clean":"2024-05-04T03:08:16.245122+0000","last_became_active":"2024-05-04T03:07:37.061077+0000","last_became_peered":"2024-05-04T03:07:37.061077+0000","last_unstale":"2024-05-04T03:08:16.245122+0000","last_undegraded":"2024-05-04T03:08:16.245122+0000","last_fullsized":"2024-05-04T03:08:16.245122+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:32.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247502+0000","last_change":"2024-05-04T03:07:37.064612+0000","last_active":"2024-05-04T03:08:16.247502+0000","last_peered":"2024-05-04T03:08:16.247502+0000","last_clean":"2024-05-04T03:08:16.247502+0000","last_became_active":"2024-05-04T03:07:37.064303+0000","last_became_peered":"2024-05-04T03:07:37.064303+0000","last_unstale":"2024-05-04T03:08:16.247502+0000","last_undegraded":"2024-05-04T03:08:16.247502+0000","last_fullsized":"2024-05-04T03:08:16.247502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:47:07.811765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250296+0000","last_change":"2024-05-04T03:07:37.061577+0000","last_active":"2024-05-04T03:08:16.250296+0000","last_peered":"2024-05-04T03:08:16.250296+0000","last_clean":"2024-05-04T03:08:16.250296+0000","last_became_active":"2024-05-04T03:07:37.061214+0000","last_became_peered":"2024-05-04T03:07:37.061214+0000","last_unstale":"2024-05-04T03:08:16.250296+0000","last_undegraded":"2024-05-04T03:08:16.250296+0000","last_fullsized":"2024-05-04T03:08:16.250296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:52:11.733257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.251332+0000","last_change":"2024-05-04T03:07:37.062906+0000","last_active":"2024-05-04T03:08:16.251332+0000","last_peered":"2024-05-04T03:08:16.251332+0000","last_clean":"2024-05-04T03:08:16.251332+0000","last_became_active":"2024-05-04T03:07:37.062510+0000","last_became_peered":"2024-05-04T03:07:37.062510+0000","last_unstale":"2024-05-04T03:08:16.251332+0000","last_undegraded":"2024-05-04T03:08:16.251332+0000","last_fullsized":"2024-05-04T03:08:16.251332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:57:58.910490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246507+0000","last_change":"2024-05-04T03:07:37.068367+0000","last_active":"2024-05-04T03:08:16.246507+0000","last_peered":"2024-05-04T03:08:16.246507+0000","last_clean":"2024-05-04T03:08:16.246507+0000","last_became_active":"2024-05-04T03:07:37.068038+0000","last_became_peered":"2024-05-04T03:07:37.068038+0000","last_unstale":"2024-05-04T03:08:16.246507+0000","last_undegraded":"2024-05-04T03:08:16.246507+0000","last_fullsized":"2024-05-04T03:08:16.246507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:11:19.195002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250759+0000","last_change":"2024-05-04T03:08:16.250759+0000","last_active":"2024-05-04T03:08:16.250759+0000","last_peered":"2024-05-04T03:08:16.250759+0000","last_clean":"2024-05-04T03:08:16.250759+0000","last_became_active":"2024-05-04T03:08:16.250379+0000","last_became_peered":"2024-05-04T03:08:16.250379+0000","last_unstale":"2024-05-04T03:08:16.250759+0000","last_undegraded":"2024-05-04T03:08:16.250759+0000","last_fullsized":"2024-05-04T03:08:16.250759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:58:09.419009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246960+0000","last_change":"2024-05-04T03:07:37.064584+0000","last_active":"2024-05-04T03:08:16.246960+0000","last_peered":"2024-05-04T03:08:16.246960+0000","last_clean":"2024-05-04T03:08:16.246960+0000","last_became_active":"2024-05-04T03:07:37.064183+0000","last_became_peered":"2024-05-04T03:07:37.064183+0000","last_unstale":"2024-05-04T03:08:16.246960+0000","last_undegraded":"2024-05-04T03:08:16.246960+0000","last_fullsized":"2024-05-04T03:08:16.246960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:17.121081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250902+0000","last_change":"2024-05-04T03:07:37.066742+0000","last_active":"2024-05-04T03:08:16.250902+0000","last_peered":"2024-05-04T03:08:16.250902+0000","last_clean":"2024-05-04T03:08:16.250902+0000","last_became_active":"2024-05-04T03:07:37.066126+0000","last_became_peered":"2024-05-04T03:07:37.066126+0000","last_unstale":"2024-05-04T03:08:16.250902+0000","last_undegraded":"2024-05-04T03:08:16.250902+0000","last_fullsized":"2024-05-04T03:08:16.250902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:57.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244050+0000","last_change":"2024-05-04T03:07:37.068122+0000","last_active":"2024-05-04T03:08:16.244050+0000","last_peered":"2024-05-04T03:08:16.244050+0000","last_clean":"2024-05-04T03:08:16.244050+0000","last_became_active":"2024-05-04T03:07:37.066321+0000","last_became_peered":"2024-05-04T03:07:37.066321+0000","last_unstale":"2024-05-04T03:08:16.244050+0000","last_undegraded":"2024-05-04T03:08:16.244050+0000","last_fullsized":"2024-05-04T03:08:16.244050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:25:09.321853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247940+0000","last_change":"2024-05-04T03:07:37.060322+0000","last_active":"2024-05-04T03:08:16.247940+0000","last_peered":"2024-05-04T03:08:16.247940+0000","last_clean":"2024-05-04T03:08:16.247940+0000","last_became_active":"2024-05-04T03:07:37.059944+0000","last_became_peered":"2024-05-04T03:07:37.059944+0000","last_unstale":"2024-05-04T03:08:16.247940+0000","last_undegraded":"2024-05-04T03:08:16.247940+0000","last_fullsized":"2024-05-04T03:08:16.247940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.105085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243469+0000","last_change":"2024-05-04T03:07:37.056269+0000","last_active":"2024-05-04T03:08:16.243469+0000","last_peered":"2024-05-04T03:08:16.243469+0000","last_clean":"2024-05-04T03:08:16.243469+0000","last_became_active":"2024-05-04T03:07:37.055886+0000","last_became_peered":"2024-05-04T03:07:37.055886+0000","last_unstale":"2024-05-04T03:08:16.243469+0000","last_undegraded":"2024-05-04T03:08:16.243469+0000","last_fullsized":"2024-05-04T03:08:16.243469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:55:11.292577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242733+0000","last_change":"2024-05-04T03:07:37.071013+0000","last_active":"2024-05-04T03:08:16.242733+0000","last_peered":"2024-05-04T03:08:16.242733+0000","last_clean":"2024-05-04T03:08:16.242733+0000","last_became_active":"2024-05-04T03:07:37.069346+0000","last_became_peered":"2024-05-04T03:07:37.069346+0000","last_unstale":"2024-05-04T03:08:16.242733+0000","last_undegraded":"2024-05-04T03:08:16.242733+0000","last_fullsized":"2024-05-04T03:08:16.242733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:19:11.898766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.249009+0000","last_change":"2024-05-04T03:07:37.064962+0000","last_active":"2024-05-04T03:08:16.249009+0000","last_peered":"2024-05-04T03:08:16.249009+0000","last_clean":"2024-05-04T03:08:16.249009+0000","last_became_active":"2024-05-04T03:07:37.064637+0000","last_became_peered":"2024-05-04T03:07:37.064637+0000","last_unstale":"2024-05-04T03:08:16.249009+0000","last_undegraded":"2024-05-04T03:08:16.249009+0000","last_fullsized":"2024-05-04T03:08:16.249009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:10:10.659621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246565+0000","last_change":"2024-05-04T03:07:37.056721+0000","last_active":"2024-05-04T03:08:16.246565+0000","last_peered":"2024-05-04T03:08:16.246565+0000","last_clean":"2024-05-04T03:08:16.246565+0000","last_became_active":"2024-05-04T03:07:37.056353+0000","last_became_peered":"2024-05-04T03:07:37.056353+0000","last_unstale":"2024-05-04T03:08:16.246565+0000","last_undegraded":"2024-05-04T03:08:16.246565+0000","last_fullsized":"2024-05-04T03:08:16.246565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:30:47.476579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286351,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67160,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1054,"num_read_kb":603,"num_write":244,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710761,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":880640,"data_stored":425832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1077531,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070516,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":425694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266744,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":426072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1018590,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":425411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":486671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":426750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":486823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":425236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":425663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822728,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":1017877,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:09:42.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:09:42.289 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:42.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:42 smithi066 ceph-mon[33264]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:42 smithi066 ceph-mon[33264]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:09:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:42.573 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:42 smithi039 ceph-mon[29715]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:42.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:42.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:42 smithi039 ceph-mon[29715]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:09:42.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:42.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:42.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:42.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:42.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:42 smithi167 ceph-mon[33614]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:09:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:42 smithi167 ceph-mon[33614]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:09:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:09:44.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/298108781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e81034c0 msgr2=0x7fb2e81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:44.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/298108781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e81034c0 0x7fb2e81038d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80099c0 tx=0x7fb2d8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/298108781 shutdown_connections 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/298108781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e81034c0 0x7fb2e81038d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/298108781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e8108740 0x7fb2e81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/298108781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2e8107df0 0x7fb2e8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/298108781 >> 172.21.15.39:0/298108781 conn(0x7fb2e80fedf0 msgr2=0x7fb2e8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/298108781 shutdown_connections 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.013+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/298108781 wait complete. 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ef2f2700 1 Processor -- start 2024-05-04T03:09:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ef2f2700 1 -- start start 2024-05-04T03:09:44.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ef2f2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e81034c0 0x7fb2e819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:44.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ef2f2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e8107df0 0x7fb2e819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:44.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ef2f2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2e8108740 0x7fb2e819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:44.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ef2f2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8113220 con 0x7fb2e8107df0 2024-05-04T03:09:44.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ef2f2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81130a0 con 0x7fb2e8108740 2024-05-04T03:09:44.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ef2f2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81133a0 con 0x7fb2e81034c0 2024-05-04T03:09:44.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ec88d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e8107df0 0x7fb2e819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:44.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ec88d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e8107df0 0x7fb2e819c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42056/0 (socket says 172.21.15.39:42056) 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ec88d700 1 -- 172.21.15.39:0/3781865343 learned_addr learned my addr 172.21.15.39:0/3781865343 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ed08e700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e81034c0 0x7fb2e819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.014+0000 7fb2ed88f700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2e8108740 0x7fb2e819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2ec88d700 1 -- 172.21.15.39:0/3781865343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e81034c0 msgr2=0x7fb2e819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2ec88d700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e81034c0 0x7fb2e819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2ec88d700 1 -- 172.21.15.39:0/3781865343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2e8108740 msgr2=0x7fb2e819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2ec88d700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2e8108740 0x7fb2e819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2ec88d700 1 -- 172.21.15.39:0/3781865343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e81a6f10 con 0x7fb2e8107df0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2ec88d700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e8107df0 0x7fb2e819c100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400d980 tx=0x7fb2e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2de7fc700 1 -- 172.21.15.39:0/3781865343 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4017070 con 0x7fb2e8107df0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2de7fc700 1 -- 172.21.15.39:0/3781865343 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2e400fd20 con 0x7fb2e8107df0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e81a7200 con 0x7fb2e8107df0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2de7fc700 1 -- 172.21.15.39:0/3781865343 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400e3b0 con 0x7fb2e8107df0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.015+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e81a7790 con 0x7fb2e8107df0 2024-05-04T03:09:44.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.016+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e80512c0 con 0x7fb2e8107df0 2024-05-04T03:09:44.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.018+0000 7fb2de7fc700 1 -- 172.21.15.39:0/3781865343 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb2e400f890 con 0x7fb2e8107df0 2024-05-04T03:09:44.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.019+0000 7fb2de7fc700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb2d40b2990 0x7fb2d40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:44.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.019+0000 7fb2ed08e700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb2d40b2990 0x7fb2d40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:44.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.019+0000 7fb2de7fc700 1 -- 172.21.15.39:0/3781865343 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb2e40e1db0 con 0x7fb2e8107df0 2024-05-04T03:09:44.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.019+0000 7fb2ed08e700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb2d40b2990 0x7fb2d40b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8000c00 tx=0x7fb2d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:44.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.022+0000 7fb2de7fc700 1 -- 172.21.15.39:0/3781865343 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2e40a9a00 con 0x7fb2e8107df0 2024-05-04T03:09:44.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.231+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2e8075fe0 con 0x7fb2d40b2990 2024-05-04T03:09:44.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.239+0000 7fb2de7fc700 1 -- 172.21.15.39:0/3781865343 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258932 (secure 0 0 0) 0x7fb2d8067390 con 0x7fb2d40b2990 2024-05-04T03:09:44.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb2d40b2990 msgr2=0x7fb2d40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb2d40b2990 0x7fb2d40b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8000c00 tx=0x7fb2d8005fb0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e8107df0 msgr2=0x7fb2e819c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e8107df0 0x7fb2e819c100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400d980 tx=0x7fb2e400dc90 comp rx=0 tx=0).stop 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 shutdown_connections 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb2d40b2990 0x7fb2d40b4ea0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2e8108740 0x7fb2e819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e81034c0 0x7fb2e819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 --2- 172.21.15.39:0/3781865343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e8107df0 0x7fb2e819c100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 >> 172.21.15.39:0/3781865343 conn(0x7fb2e80fedf0 msgr2=0x7fb2e8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:44.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 shutdown_connections 2024-05-04T03:09:44.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:44.243+0000 7fb2ef2f2700 1 -- 172.21.15.39:0/3781865343 wait complete. 2024-05-04T03:09:44.246 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-04T03:09:44.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:44 smithi066 ceph-mon[33264]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:44.628 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:44 smithi039 ceph-mon[29715]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:44.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:44 smithi167 ceph-mon[33614]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:09:44.923 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-05-04T03:09:43.027798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657762,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1150,"num_read_kb":672,"num_write":369,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376632,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565192,"statfs":{"total":1151940427776,"available":1151554756608,"internally_reserved":0,"allocated":11059200,"data_stored":7068150,"data_compressed":109203,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003394"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247331+0000","last_change":"2024-05-04T03:07:37.070347+0000","last_active":"2024-05-04T03:08:16.247331+0000","last_peered":"2024-05-04T03:08:16.247331+0000","last_clean":"2024-05-04T03:08:16.247331+0000","last_became_active":"2024-05-04T03:07:37.069997+0000","last_became_peered":"2024-05-04T03:07:37.069997+0000","last_unstale":"2024-05-04T03:08:16.247331+0000","last_undegraded":"2024-05-04T03:08:16.247331+0000","last_fullsized":"2024-05-04T03:08:16.247331+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:24:11.362228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247260+0000","last_change":"2024-05-04T03:07:37.071598+0000","last_active":"2024-05-04T03:08:16.247260+0000","last_peered":"2024-05-04T03:08:16.247260+0000","last_clean":"2024-05-04T03:08:16.247260+0000","last_became_active":"2024-05-04T03:07:37.071059+0000","last_became_peered":"2024-05-04T03:07:37.071059+0000","last_unstale":"2024-05-04T03:08:16.247260+0000","last_undegraded":"2024-05-04T03:08:16.247260+0000","last_fullsized":"2024-05-04T03:08:16.247260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:33:33.929246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246145+0000","last_change":"2024-05-04T03:07:37.069951+0000","last_active":"2024-05-04T03:08:16.246145+0000","last_peered":"2024-05-04T03:08:16.246145+0000","last_clean":"2024-05-04T03:08:16.246145+0000","last_became_active":"2024-05-04T03:07:37.069558+0000","last_became_peered":"2024-05-04T03:07:37.069558+0000","last_unstale":"2024-05-04T03:08:16.246145+0000","last_undegraded":"2024-05-04T03:08:16.246145+0000","last_fullsized":"2024-05-04T03:08:16.246145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:37:52.596455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367387+0000","last_change":"2024-05-04T03:07:37.068820+0000","last_active":"2024-05-04T03:08:16.367387+0000","last_peered":"2024-05-04T03:08:16.367387+0000","last_clean":"2024-05-04T03:08:16.367387+0000","last_became_active":"2024-05-04T03:07:37.068323+0000","last_became_peered":"2024-05-04T03:07:37.068323+0000","last_unstale":"2024-05-04T03:08:16.367387+0000","last_undegraded":"2024-05-04T03:08:16.367387+0000","last_fullsized":"2024-05-04T03:08:16.367387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:52:54.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245207+0000","last_change":"2024-05-04T03:07:37.061982+0000","last_active":"2024-05-04T03:08:16.245207+0000","last_peered":"2024-05-04T03:08:16.245207+0000","last_clean":"2024-05-04T03:08:16.245207+0000","last_became_active":"2024-05-04T03:07:37.061621+0000","last_became_peered":"2024-05-04T03:07:37.061621+0000","last_unstale":"2024-05-04T03:08:16.245207+0000","last_undegraded":"2024-05-04T03:08:16.245207+0000","last_fullsized":"2024-05-04T03:08:16.245207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:17:47.421011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248166+0000","last_change":"2024-05-04T03:07:37.058641+0000","last_active":"2024-05-04T03:08:16.248166+0000","last_peered":"2024-05-04T03:08:16.248166+0000","last_clean":"2024-05-04T03:08:16.248166+0000","last_became_active":"2024-05-04T03:07:37.058235+0000","last_became_peered":"2024-05-04T03:07:37.058235+0000","last_unstale":"2024-05-04T03:08:16.248166+0000","last_undegraded":"2024-05-04T03:08:16.248166+0000","last_fullsized":"2024-05-04T03:08:16.248166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:17:45.441777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244522+0000","last_change":"2024-05-04T03:07:37.065959+0000","last_active":"2024-05-04T03:08:16.244522+0000","last_peered":"2024-05-04T03:08:16.244522+0000","last_clean":"2024-05-04T03:08:16.244522+0000","last_became_active":"2024-05-04T03:07:37.065508+0000","last_became_peered":"2024-05-04T03:07:37.065508+0000","last_unstale":"2024-05-04T03:08:16.244522+0000","last_undegraded":"2024-05-04T03:08:16.244522+0000","last_fullsized":"2024-05-04T03:08:16.244522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:17.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243478+0000","last_change":"2024-05-04T03:07:37.065620+0000","last_active":"2024-05-04T03:08:16.243478+0000","last_peered":"2024-05-04T03:08:16.243478+0000","last_clean":"2024-05-04T03:08:16.243478+0000","last_became_active":"2024-05-04T03:07:37.065261+0000","last_became_peered":"2024-05-04T03:07:37.065261+0000","last_unstale":"2024-05-04T03:08:16.243478+0000","last_undegraded":"2024-05-04T03:08:16.243478+0000","last_fullsized":"2024-05-04T03:08:16.243478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:27.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242930+0000","last_change":"2024-05-04T03:07:37.072204+0000","last_active":"2024-05-04T03:08:16.242930+0000","last_peered":"2024-05-04T03:08:16.242930+0000","last_clean":"2024-05-04T03:08:16.242930+0000","last_became_active":"2024-05-04T03:07:37.070622+0000","last_became_peered":"2024-05-04T03:07:37.070622+0000","last_unstale":"2024-05-04T03:08:16.242930+0000","last_undegraded":"2024-05-04T03:08:16.242930+0000","last_fullsized":"2024-05-04T03:08:16.242930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:55:25.931369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.253125+0000","last_change":"2024-05-04T03:08:16.253125+0000","last_active":"2024-05-04T03:08:16.253125+0000","last_peered":"2024-05-04T03:08:16.253125+0000","last_clean":"2024-05-04T03:08:16.253125+0000","last_became_active":"2024-05-04T03:08:16.252682+0000","last_became_peered":"2024-05-04T03:08:16.252682+0000","last_unstale":"2024-05-04T03:08:16.253125+0000","last_undegraded":"2024-05-04T03:08:16.253125+0000","last_fullsized":"2024-05-04T03:08:16.253125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:05.933491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244798+0000","last_change":"2024-05-04T03:07:37.071614+0000","last_active":"2024-05-04T03:08:16.244798+0000","last_peered":"2024-05-04T03:08:16.244798+0000","last_clean":"2024-05-04T03:08:16.244798+0000","last_became_active":"2024-05-04T03:07:37.070927+0000","last_became_peered":"2024-05-04T03:07:37.070927+0000","last_unstale":"2024-05-04T03:08:16.244798+0000","last_undegraded":"2024-05-04T03:08:16.244798+0000","last_fullsized":"2024-05-04T03:08:16.244798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:30.432959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367227+0000","last_change":"2024-05-04T03:07:37.069305+0000","last_active":"2024-05-04T03:08:16.367227+0000","last_peered":"2024-05-04T03:08:16.367227+0000","last_clean":"2024-05-04T03:08:16.367227+0000","last_became_active":"2024-05-04T03:07:37.068804+0000","last_became_peered":"2024-05-04T03:07:37.068804+0000","last_unstale":"2024-05-04T03:08:16.367227+0000","last_undegraded":"2024-05-04T03:08:16.367227+0000","last_fullsized":"2024-05-04T03:08:16.367227+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:43.693161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248895+0000","last_change":"2024-05-04T03:07:37.062848+0000","last_active":"2024-05-04T03:08:16.248895+0000","last_peered":"2024-05-04T03:08:16.248895+0000","last_clean":"2024-05-04T03:08:16.248895+0000","last_became_active":"2024-05-04T03:07:37.062548+0000","last_became_peered":"2024-05-04T03:07:37.062548+0000","last_unstale":"2024-05-04T03:08:16.248895+0000","last_undegraded":"2024-05-04T03:08:16.248895+0000","last_fullsized":"2024-05-04T03:08:16.248895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246059+0000","last_change":"2024-05-04T03:07:37.062598+0000","last_active":"2024-05-04T03:08:16.246059+0000","last_peered":"2024-05-04T03:08:16.246059+0000","last_clean":"2024-05-04T03:08:16.246059+0000","last_became_active":"2024-05-04T03:07:37.061711+0000","last_became_peered":"2024-05-04T03:07:37.061711+0000","last_unstale":"2024-05-04T03:08:16.246059+0000","last_undegraded":"2024-05-04T03:08:16.246059+0000","last_fullsized":"2024-05-04T03:08:16.246059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:29:03.153680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246387+0000","last_change":"2024-05-04T03:07:37.067889+0000","last_active":"2024-05-04T03:08:16.246387+0000","last_peered":"2024-05-04T03:08:16.246387+0000","last_clean":"2024-05-04T03:08:16.246387+0000","last_became_active":"2024-05-04T03:07:37.066768+0000","last_became_peered":"2024-05-04T03:07:37.066768+0000","last_unstale":"2024-05-04T03:08:16.246387+0000","last_undegraded":"2024-05-04T03:08:16.246387+0000","last_fullsized":"2024-05-04T03:08:16.246387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:01:18.136379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242305+0000","last_change":"2024-05-04T03:07:37.070438+0000","last_active":"2024-05-04T03:08:16.242305+0000","last_peered":"2024-05-04T03:08:16.242305+0000","last_clean":"2024-05-04T03:08:16.242305+0000","last_became_active":"2024-05-04T03:07:37.070007+0000","last_became_peered":"2024-05-04T03:07:37.070007+0000","last_unstale":"2024-05-04T03:08:16.242305+0000","last_undegraded":"2024-05-04T03:08:16.242305+0000","last_fullsized":"2024-05-04T03:08:16.242305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:22.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247556+0000","last_change":"2024-05-04T03:07:37.070020+0000","last_active":"2024-05-04T03:08:16.247556+0000","last_peered":"2024-05-04T03:08:16.247556+0000","last_clean":"2024-05-04T03:08:16.247556+0000","last_became_active":"2024-05-04T03:07:37.069629+0000","last_became_peered":"2024-05-04T03:07:37.069629+0000","last_unstale":"2024-05-04T03:08:16.247556+0000","last_undegraded":"2024-05-04T03:08:16.247556+0000","last_fullsized":"2024-05-04T03:08:16.247556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:28.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245734+0000","last_change":"2024-05-04T03:07:37.068651+0000","last_active":"2024-05-04T03:08:16.245734+0000","last_peered":"2024-05-04T03:08:16.245734+0000","last_clean":"2024-05-04T03:08:16.245734+0000","last_became_active":"2024-05-04T03:07:37.067135+0000","last_became_peered":"2024-05-04T03:07:37.067135+0000","last_unstale":"2024-05-04T03:08:16.245734+0000","last_undegraded":"2024-05-04T03:08:16.245734+0000","last_fullsized":"2024-05-04T03:08:16.245734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:00:59.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250832+0000","last_change":"2024-05-04T03:07:37.055419+0000","last_active":"2024-05-04T03:08:16.250832+0000","last_peered":"2024-05-04T03:08:16.250832+0000","last_clean":"2024-05-04T03:08:16.250832+0000","last_became_active":"2024-05-04T03:07:37.055024+0000","last_became_peered":"2024-05-04T03:07:37.055024+0000","last_unstale":"2024-05-04T03:08:16.250832+0000","last_undegraded":"2024-05-04T03:08:16.250832+0000","last_fullsized":"2024-05-04T03:08:16.250832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:55:05.506505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:38.348088+0000","last_change":"2024-05-04T03:07:34.019975+0000","last_active":"2024-05-04T03:09:38.348088+0000","last_peered":"2024-05-04T03:09:38.348088+0000","last_clean":"2024-05-04T03:09:38.348088+0000","last_became_active":"2024-05-04T03:07:34.019532+0000","last_became_peered":"2024-05-04T03:07:34.019532+0000","last_unstale":"2024-05-04T03:09:38.348088+0000","last_undegraded":"2024-05-04T03:09:38.348088+0000","last_fullsized":"2024-05-04T03:09:38.348088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:11:03.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244135+0000","last_change":"2024-05-04T03:07:37.064725+0000","last_active":"2024-05-04T03:08:16.244135+0000","last_peered":"2024-05-04T03:08:16.244135+0000","last_clean":"2024-05-04T03:08:16.244135+0000","last_became_active":"2024-05-04T03:07:37.064385+0000","last_became_peered":"2024-05-04T03:07:37.064385+0000","last_unstale":"2024-05-04T03:08:16.244135+0000","last_undegraded":"2024-05-04T03:08:16.244135+0000","last_fullsized":"2024-05-04T03:08:16.244135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:22:12.024328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244201+0000","last_change":"2024-05-04T03:07:34.157179+0000","last_active":"2024-05-04T03:08:16.244201+0000","last_peered":"2024-05-04T03:08:16.244201+0000","last_clean":"2024-05-04T03:08:16.244201+0000","last_became_active":"2024-05-04T03:07:34.156790+0000","last_became_peered":"2024-05-04T03:07:34.156790+0000","last_unstale":"2024-05-04T03:08:16.244201+0000","last_undegraded":"2024-05-04T03:08:16.244201+0000","last_fullsized":"2024-05-04T03:08:16.244201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:32.066410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246454+0000","last_change":"2024-05-04T03:07:37.070481+0000","last_active":"2024-05-04T03:08:16.246454+0000","last_peered":"2024-05-04T03:08:16.246454+0000","last_clean":"2024-05-04T03:08:16.246454+0000","last_became_active":"2024-05-04T03:07:37.069903+0000","last_became_peered":"2024-05-04T03:07:37.069903+0000","last_unstale":"2024-05-04T03:08:16.246454+0000","last_undegraded":"2024-05-04T03:08:16.246454+0000","last_fullsized":"2024-05-04T03:08:16.246454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:45:40.764716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245723+0000","last_change":"2024-05-04T03:07:34.023978+0000","last_active":"2024-05-04T03:08:16.245723+0000","last_peered":"2024-05-04T03:08:16.245723+0000","last_clean":"2024-05-04T03:08:16.245723+0000","last_became_active":"2024-05-04T03:07:34.023199+0000","last_became_peered":"2024-05-04T03:07:34.023199+0000","last_unstale":"2024-05-04T03:08:16.245723+0000","last_undegraded":"2024-05-04T03:08:16.245723+0000","last_fullsized":"2024-05-04T03:08:16.245723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:17:43.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242450+0000","last_change":"2024-05-04T03:07:37.072661+0000","last_active":"2024-05-04T03:08:16.242450+0000","last_peered":"2024-05-04T03:08:16.242450+0000","last_clean":"2024-05-04T03:08:16.242450+0000","last_became_active":"2024-05-04T03:07:37.071393+0000","last_became_peered":"2024-05-04T03:07:37.071393+0000","last_unstale":"2024-05-04T03:08:16.242450+0000","last_undegraded":"2024-05-04T03:08:16.242450+0000","last_fullsized":"2024-05-04T03:08:16.242450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:39:39.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245493+0000","last_change":"2024-05-04T03:07:34.018543+0000","last_active":"2024-05-04T03:08:16.245493+0000","last_peered":"2024-05-04T03:08:16.245493+0000","last_clean":"2024-05-04T03:08:16.245493+0000","last_became_active":"2024-05-04T03:07:34.018145+0000","last_became_peered":"2024-05-04T03:07:34.018145+0000","last_unstale":"2024-05-04T03:08:16.245493+0000","last_undegraded":"2024-05-04T03:08:16.245493+0000","last_fullsized":"2024-05-04T03:08:16.245493+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:44.529492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248677+0000","last_change":"2024-05-04T03:07:37.070811+0000","last_active":"2024-05-04T03:08:16.248677+0000","last_peered":"2024-05-04T03:08:16.248677+0000","last_clean":"2024-05-04T03:08:16.248677+0000","last_became_active":"2024-05-04T03:07:37.070441+0000","last_became_peered":"2024-05-04T03:07:37.070441+0000","last_unstale":"2024-05-04T03:08:16.248677+0000","last_undegraded":"2024-05-04T03:08:16.248677+0000","last_fullsized":"2024-05-04T03:08:16.248677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:01:48.146339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:39.328681+0000","last_change":"2024-05-04T03:07:34.020508+0000","last_active":"2024-05-04T03:09:39.328681+0000","last_peered":"2024-05-04T03:09:39.328681+0000","last_clean":"2024-05-04T03:09:39.328681+0000","last_became_active":"2024-05-04T03:07:34.020076+0000","last_became_peered":"2024-05-04T03:07:34.020076+0000","last_unstale":"2024-05-04T03:09:39.328681+0000","last_undegraded":"2024-05-04T03:09:39.328681+0000","last_fullsized":"2024-05-04T03:09:39.328681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:27.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245429+0000","last_change":"2024-05-04T03:07:37.060004+0000","last_active":"2024-05-04T03:08:16.245429+0000","last_peered":"2024-05-04T03:08:16.245429+0000","last_clean":"2024-05-04T03:08:16.245429+0000","last_became_active":"2024-05-04T03:07:37.059350+0000","last_became_peered":"2024-05-04T03:07:37.059350+0000","last_unstale":"2024-05-04T03:08:16.245429+0000","last_undegraded":"2024-05-04T03:08:16.245429+0000","last_fullsized":"2024-05-04T03:08:16.245429+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:37:07.315935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248068+0000","last_change":"2024-05-04T03:07:34.020174+0000","last_active":"2024-05-04T03:08:16.248068+0000","last_peered":"2024-05-04T03:08:16.248068+0000","last_clean":"2024-05-04T03:08:16.248068+0000","last_became_active":"2024-05-04T03:07:34.019790+0000","last_became_peered":"2024-05-04T03:07:34.019790+0000","last_unstale":"2024-05-04T03:08:16.248068+0000","last_undegraded":"2024-05-04T03:08:16.248068+0000","last_fullsized":"2024-05-04T03:08:16.248068+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:37:37.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.252143+0000","last_change":"2024-05-04T03:08:16.251277+0000","last_active":"2024-05-04T03:08:16.252143+0000","last_peered":"2024-05-04T03:08:16.252143+0000","last_clean":"2024-05-04T03:08:16.252143+0000","last_became_active":"2024-05-04T03:08:16.250836+0000","last_became_peered":"2024-05-04T03:08:16.250836+0000","last_unstale":"2024-05-04T03:08:16.252143+0000","last_undegraded":"2024-05-04T03:08:16.252143+0000","last_fullsized":"2024-05-04T03:08:16.252143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:59:47.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248397+0000","last_change":"2024-05-04T03:07:34.013188+0000","last_active":"2024-05-04T03:08:16.248397+0000","last_peered":"2024-05-04T03:08:16.248397+0000","last_clean":"2024-05-04T03:08:16.248397+0000","last_became_active":"2024-05-04T03:07:34.012759+0000","last_became_peered":"2024-05-04T03:07:34.012759+0000","last_unstale":"2024-05-04T03:08:16.248397+0000","last_undegraded":"2024-05-04T03:08:16.248397+0000","last_fullsized":"2024-05-04T03:08:16.248397+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:54:29.621250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243425+0000","last_change":"2024-05-04T03:07:37.056265+0000","last_active":"2024-05-04T03:08:16.243425+0000","last_peered":"2024-05-04T03:08:16.243425+0000","last_clean":"2024-05-04T03:08:16.243425+0000","last_became_active":"2024-05-04T03:07:37.055957+0000","last_became_peered":"2024-05-04T03:07:37.055957+0000","last_unstale":"2024-05-04T03:08:16.243425+0000","last_undegraded":"2024-05-04T03:08:16.243425+0000","last_fullsized":"2024-05-04T03:08:16.243425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:41.739034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247855+0000","last_change":"2024-05-04T03:07:34.020722+0000","last_active":"2024-05-04T03:08:16.247855+0000","last_peered":"2024-05-04T03:08:16.247855+0000","last_clean":"2024-05-04T03:08:16.247855+0000","last_became_active":"2024-05-04T03:07:34.020028+0000","last_became_peered":"2024-05-04T03:07:34.020028+0000","last_unstale":"2024-05-04T03:08:16.247855+0000","last_undegraded":"2024-05-04T03:08:16.247855+0000","last_fullsized":"2024-05-04T03:08:16.247855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:23:20.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245364+0000","last_change":"2024-05-04T03:07:37.059680+0000","last_active":"2024-05-04T03:08:16.245364+0000","last_peered":"2024-05-04T03:08:16.245364+0000","last_clean":"2024-05-04T03:08:16.245364+0000","last_became_active":"2024-05-04T03:07:37.059294+0000","last_became_peered":"2024-05-04T03:07:37.059294+0000","last_unstale":"2024-05-04T03:08:16.245364+0000","last_undegraded":"2024-05-04T03:08:16.245364+0000","last_fullsized":"2024-05-04T03:08:16.245364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:50:06.251706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":236,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:40.329512+0000","last_change":"2024-05-04T03:07:34.019055+0000","last_active":"2024-05-04T03:09:40.329512+0000","last_peered":"2024-05-04T03:09:40.329512+0000","last_clean":"2024-05-04T03:09:40.329512+0000","last_became_active":"2024-05-04T03:07:34.018638+0000","last_became_peered":"2024-05-04T03:07:34.018638+0000","last_unstale":"2024-05-04T03:09:40.329512+0000","last_undegraded":"2024-05-04T03:09:40.329512+0000","last_fullsized":"2024-05-04T03:09:40.329512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:58:01.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242844+0000","last_change":"2024-05-04T03:07:37.063037+0000","last_active":"2024-05-04T03:08:16.242844+0000","last_peered":"2024-05-04T03:08:16.242844+0000","last_clean":"2024-05-04T03:08:16.242844+0000","last_became_active":"2024-05-04T03:07:37.062350+0000","last_became_peered":"2024-05-04T03:07:37.062350+0000","last_unstale":"2024-05-04T03:08:16.242844+0000","last_undegraded":"2024-05-04T03:08:16.242844+0000","last_fullsized":"2024-05-04T03:08:16.242844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:41:15.898241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.373765+0000","last_change":"2024-05-04T03:08:16.373765+0000","last_active":"2024-05-04T03:08:16.373765+0000","last_peered":"2024-05-04T03:08:16.373765+0000","last_clean":"2024-05-04T03:08:16.373765+0000","last_became_active":"2024-05-04T03:08:16.368588+0000","last_became_peered":"2024-05-04T03:08:16.368588+0000","last_unstale":"2024-05-04T03:08:16.373765+0000","last_undegraded":"2024-05-04T03:08:16.373765+0000","last_fullsized":"2024-05-04T03:08:16.373765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:55:07.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246604+0000","last_change":"2024-05-04T03:07:37.071330+0000","last_active":"2024-05-04T03:08:16.246604+0000","last_peered":"2024-05-04T03:08:16.246604+0000","last_clean":"2024-05-04T03:08:16.246604+0000","last_became_active":"2024-05-04T03:07:37.070743+0000","last_became_peered":"2024-05-04T03:07:37.070743+0000","last_unstale":"2024-05-04T03:08:16.246604+0000","last_undegraded":"2024-05-04T03:08:16.246604+0000","last_fullsized":"2024-05-04T03:08:16.246604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:10:00.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248817+0000","last_change":"2024-05-04T03:07:34.157432+0000","last_active":"2024-05-04T03:08:16.248817+0000","last_peered":"2024-05-04T03:08:16.248817+0000","last_clean":"2024-05-04T03:08:16.248817+0000","last_became_active":"2024-05-04T03:07:34.156976+0000","last_became_peered":"2024-05-04T03:07:34.156976+0000","last_unstale":"2024-05-04T03:08:16.248817+0000","last_undegraded":"2024-05-04T03:08:16.248817+0000","last_fullsized":"2024-05-04T03:08:16.248817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:56.082287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242517+0000","last_change":"2024-05-04T03:07:37.064301+0000","last_active":"2024-05-04T03:08:16.242517+0000","last_peered":"2024-05-04T03:08:16.242517+0000","last_clean":"2024-05-04T03:08:16.242517+0000","last_became_active":"2024-05-04T03:07:37.063691+0000","last_became_peered":"2024-05-04T03:07:37.063691+0000","last_unstale":"2024-05-04T03:08:16.242517+0000","last_undegraded":"2024-05-04T03:08:16.242517+0000","last_fullsized":"2024-05-04T03:08:16.242517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:22:13.844703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":272,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:40.329521+0000","last_change":"2024-05-04T03:07:34.021595+0000","last_active":"2024-05-04T03:09:40.329521+0000","last_peered":"2024-05-04T03:09:40.329521+0000","last_clean":"2024-05-04T03:09:40.329521+0000","last_became_active":"2024-05-04T03:07:34.021126+0000","last_became_peered":"2024-05-04T03:07:34.021126+0000","last_unstale":"2024-05-04T03:09:40.329521+0000","last_undegraded":"2024-05-04T03:09:40.329521+0000","last_fullsized":"2024-05-04T03:09:40.329521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:55.094800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":215,"num_read_kb":166,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245497+0000","last_change":"2024-05-04T03:07:37.066365+0000","last_active":"2024-05-04T03:08:16.245497+0000","last_peered":"2024-05-04T03:08:16.245497+0000","last_clean":"2024-05-04T03:08:16.245497+0000","last_became_active":"2024-05-04T03:07:37.065851+0000","last_became_peered":"2024-05-04T03:07:37.065851+0000","last_unstale":"2024-05-04T03:08:16.245497+0000","last_undegraded":"2024-05-04T03:08:16.245497+0000","last_fullsized":"2024-05-04T03:08:16.245497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:03:43.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.263114+0000","last_change":"2024-05-04T03:08:16.262600+0000","last_active":"2024-05-04T03:08:16.263114+0000","last_peered":"2024-05-04T03:08:16.263114+0000","last_clean":"2024-05-04T03:08:16.263114+0000","last_became_active":"2024-05-04T03:08:16.250619+0000","last_became_peered":"2024-05-04T03:08:16.250619+0000","last_unstale":"2024-05-04T03:08:16.263114+0000","last_undegraded":"2024-05-04T03:08:16.263114+0000","last_fullsized":"2024-05-04T03:08:16.263114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:51:21.596467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248552+0000","last_change":"2024-05-04T03:07:37.055356+0000","last_active":"2024-05-04T03:08:16.248552+0000","last_peered":"2024-05-04T03:08:16.248552+0000","last_clean":"2024-05-04T03:08:16.248552+0000","last_became_active":"2024-05-04T03:07:37.055045+0000","last_became_peered":"2024-05-04T03:07:37.055045+0000","last_unstale":"2024-05-04T03:08:16.248552+0000","last_undegraded":"2024-05-04T03:08:16.248552+0000","last_fullsized":"2024-05-04T03:08:16.248552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:06.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247382+0000","last_change":"2024-05-04T03:07:34.021525+0000","last_active":"2024-05-04T03:08:16.247382+0000","last_peered":"2024-05-04T03:08:16.247382+0000","last_clean":"2024-05-04T03:08:16.247382+0000","last_became_active":"2024-05-04T03:07:34.021042+0000","last_became_peered":"2024-05-04T03:07:34.021042+0000","last_unstale":"2024-05-04T03:08:16.247382+0000","last_undegraded":"2024-05-04T03:08:16.247382+0000","last_fullsized":"2024-05-04T03:08:16.247382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:44.090176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244794+0000","last_change":"2024-05-04T03:07:37.066471+0000","last_active":"2024-05-04T03:08:16.244794+0000","last_peered":"2024-05-04T03:08:16.244794+0000","last_clean":"2024-05-04T03:08:16.244794+0000","last_became_active":"2024-05-04T03:07:37.066059+0000","last_became_peered":"2024-05-04T03:07:37.066059+0000","last_unstale":"2024-05-04T03:08:16.244794+0000","last_undegraded":"2024-05-04T03:08:16.244794+0000","last_fullsized":"2024-05-04T03:08:16.244794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:16.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":139,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:41.330938+0000","last_change":"2024-05-04T03:07:34.021531+0000","last_active":"2024-05-04T03:09:41.330938+0000","last_peered":"2024-05-04T03:09:41.330938+0000","last_clean":"2024-05-04T03:09:41.330938+0000","last_became_active":"2024-05-04T03:07:34.021112+0000","last_became_peered":"2024-05-04T03:07:34.021112+0000","last_unstale":"2024-05-04T03:09:41.330938+0000","last_undegraded":"2024-05-04T03:09:41.330938+0000","last_fullsized":"2024-05-04T03:09:41.330938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:28.826426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244611+0000","last_change":"2024-05-04T03:07:37.067155+0000","last_active":"2024-05-04T03:08:16.244611+0000","last_peered":"2024-05-04T03:08:16.244611+0000","last_clean":"2024-05-04T03:08:16.244611+0000","last_became_active":"2024-05-04T03:07:37.066783+0000","last_became_peered":"2024-05-04T03:07:37.066783+0000","last_unstale":"2024-05-04T03:08:16.244611+0000","last_undegraded":"2024-05-04T03:08:16.244611+0000","last_fullsized":"2024-05-04T03:08:16.244611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:23.217930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245013+0000","last_change":"2024-05-04T03:07:34.158227+0000","last_active":"2024-05-04T03:08:16.245013+0000","last_peered":"2024-05-04T03:08:16.245013+0000","last_clean":"2024-05-04T03:08:16.245013+0000","last_became_active":"2024-05-04T03:07:34.156905+0000","last_became_peered":"2024-05-04T03:07:34.156905+0000","last_unstale":"2024-05-04T03:08:16.245013+0000","last_undegraded":"2024-05-04T03:08:16.245013+0000","last_fullsized":"2024-05-04T03:08:16.245013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:40:41.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247601+0000","last_change":"2024-05-04T03:07:37.075559+0000","last_active":"2024-05-04T03:08:16.247601+0000","last_peered":"2024-05-04T03:08:16.247601+0000","last_clean":"2024-05-04T03:08:16.247601+0000","last_became_active":"2024-05-04T03:07:37.075220+0000","last_became_peered":"2024-05-04T03:07:37.075220+0000","last_unstale":"2024-05-04T03:08:16.247601+0000","last_undegraded":"2024-05-04T03:08:16.247601+0000","last_fullsized":"2024-05-04T03:08:16.247601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:51:23.942514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246548+0000","last_change":"2024-05-04T03:07:34.020326+0000","last_active":"2024-05-04T03:08:16.246548+0000","last_peered":"2024-05-04T03:08:16.246548+0000","last_clean":"2024-05-04T03:08:16.246548+0000","last_became_active":"2024-05-04T03:07:34.019472+0000","last_became_peered":"2024-05-04T03:07:34.019472+0000","last_unstale":"2024-05-04T03:08:16.246548+0000","last_undegraded":"2024-05-04T03:08:16.246548+0000","last_fullsized":"2024-05-04T03:08:16.246548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:33:28.375901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247007+0000","last_change":"2024-05-04T03:07:37.066223+0000","last_active":"2024-05-04T03:08:16.247007+0000","last_peered":"2024-05-04T03:08:16.247007+0000","last_clean":"2024-05-04T03:08:16.247007+0000","last_became_active":"2024-05-04T03:07:37.065859+0000","last_became_peered":"2024-05-04T03:07:37.065859+0000","last_unstale":"2024-05-04T03:08:16.247007+0000","last_undegraded":"2024-05-04T03:08:16.247007+0000","last_fullsized":"2024-05-04T03:08:16.247007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:44.902986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245539+0000","last_change":"2024-05-04T03:07:34.158674+0000","last_active":"2024-05-04T03:08:16.245539+0000","last_peered":"2024-05-04T03:08:16.245539+0000","last_clean":"2024-05-04T03:08:16.245539+0000","last_became_active":"2024-05-04T03:07:34.157952+0000","last_became_peered":"2024-05-04T03:07:34.157952+0000","last_unstale":"2024-05-04T03:08:16.245539+0000","last_undegraded":"2024-05-04T03:08:16.245539+0000","last_fullsized":"2024-05-04T03:08:16.245539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:19:11.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248252+0000","last_change":"2024-05-04T03:07:37.053897+0000","last_active":"2024-05-04T03:08:16.248252+0000","last_peered":"2024-05-04T03:08:16.248252+0000","last_clean":"2024-05-04T03:08:16.248252+0000","last_became_active":"2024-05-04T03:07:37.053466+0000","last_became_peered":"2024-05-04T03:07:37.053466+0000","last_unstale":"2024-05-04T03:08:16.248252+0000","last_undegraded":"2024-05-04T03:08:16.248252+0000","last_fullsized":"2024-05-04T03:08:16.248252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:50.650805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":137,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:41.353448+0000","last_change":"2024-05-04T03:07:34.021494+0000","last_active":"2024-05-04T03:09:41.353448+0000","last_peered":"2024-05-04T03:09:41.353448+0000","last_clean":"2024-05-04T03:09:41.353448+0000","last_became_active":"2024-05-04T03:07:34.021113+0000","last_became_peered":"2024-05-04T03:07:34.021113+0000","last_unstale":"2024-05-04T03:09:41.353448+0000","last_undegraded":"2024-05-04T03:09:41.353448+0000","last_fullsized":"2024-05-04T03:09:41.353448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:02:09.514961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248164+0000","last_change":"2024-05-04T03:07:37.068199+0000","last_active":"2024-05-04T03:08:16.248164+0000","last_peered":"2024-05-04T03:08:16.248164+0000","last_clean":"2024-05-04T03:08:16.248164+0000","last_became_active":"2024-05-04T03:07:37.067828+0000","last_became_peered":"2024-05-04T03:07:37.067828+0000","last_unstale":"2024-05-04T03:08:16.248164+0000","last_undegraded":"2024-05-04T03:08:16.248164+0000","last_fullsized":"2024-05-04T03:08:16.248164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:28.882920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248172+0000","last_change":"2024-05-04T03:07:37.069753+0000","last_active":"2024-05-04T03:08:16.248172+0000","last_peered":"2024-05-04T03:08:16.248172+0000","last_clean":"2024-05-04T03:08:16.248172+0000","last_became_active":"2024-05-04T03:07:37.069338+0000","last_became_peered":"2024-05-04T03:07:37.069338+0000","last_unstale":"2024-05-04T03:08:16.248172+0000","last_undegraded":"2024-05-04T03:08:16.248172+0000","last_fullsized":"2024-05-04T03:08:16.248172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:37:11.355749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":137,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:41.330174+0000","last_change":"2024-05-04T03:07:34.157229+0000","last_active":"2024-05-04T03:09:41.330174+0000","last_peered":"2024-05-04T03:09:41.330174+0000","last_clean":"2024-05-04T03:09:41.330174+0000","last_became_active":"2024-05-04T03:07:34.156776+0000","last_became_peered":"2024-05-04T03:07:34.156776+0000","last_unstale":"2024-05-04T03:09:41.330174+0000","last_undegraded":"2024-05-04T03:09:41.330174+0000","last_fullsized":"2024-05-04T03:09:41.330174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:26.879754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243715+0000","last_change":"2024-05-04T03:07:37.067025+0000","last_active":"2024-05-04T03:08:16.243715+0000","last_peered":"2024-05-04T03:08:16.243715+0000","last_clean":"2024-05-04T03:08:16.243715+0000","last_became_active":"2024-05-04T03:07:37.066359+0000","last_became_peered":"2024-05-04T03:07:37.066359+0000","last_unstale":"2024-05-04T03:08:16.243715+0000","last_undegraded":"2024-05-04T03:08:16.243715+0000","last_fullsized":"2024-05-04T03:08:16.243715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:44:35.604859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367184+0000","last_change":"2024-05-04T03:07:34.018019+0000","last_active":"2024-05-04T03:08:16.367184+0000","last_peered":"2024-05-04T03:08:16.367184+0000","last_clean":"2024-05-04T03:08:16.367184+0000","last_became_active":"2024-05-04T03:07:34.017556+0000","last_became_peered":"2024-05-04T03:07:34.017556+0000","last_unstale":"2024-05-04T03:08:16.367184+0000","last_undegraded":"2024-05-04T03:08:16.367184+0000","last_fullsized":"2024-05-04T03:08:16.367184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:31:34.158164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244533+0000","last_change":"2024-05-04T03:06:06.704755+0000","last_active":"2024-05-04T03:08:16.244533+0000","last_peered":"2024-05-04T03:08:16.244533+0000","last_clean":"2024-05-04T03:08:16.244533+0000","last_became_active":"2024-05-04T03:06:06.698747+0000","last_became_peered":"2024-05-04T03:06:06.698747+0000","last_unstale":"2024-05-04T03:08:16.244533+0000","last_undegraded":"2024-05-04T03:08:16.244533+0000","last_fullsized":"2024-05-04T03:08:16.244533+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_clean_scrub_stamp":"2024-05-04T03:02:43.385377+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:04:50.012414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248327+0000","last_change":"2024-05-04T03:07:37.057043+0000","last_active":"2024-05-04T03:08:16.248327+0000","last_peered":"2024-05-04T03:08:16.248327+0000","last_clean":"2024-05-04T03:08:16.248327+0000","last_became_active":"2024-05-04T03:07:37.056725+0000","last_became_peered":"2024-05-04T03:07:37.056725+0000","last_unstale":"2024-05-04T03:08:16.248327+0000","last_undegraded":"2024-05-04T03:08:16.248327+0000","last_fullsized":"2024-05-04T03:08:16.248327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:13:28.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.377000+0000","last_change":"2024-05-04T03:08:16.376510+0000","last_active":"2024-05-04T03:08:16.377000+0000","last_peered":"2024-05-04T03:08:16.377000+0000","last_clean":"2024-05-04T03:08:16.377000+0000","last_became_active":"2024-05-04T03:08:16.368625+0000","last_became_peered":"2024-05-04T03:08:16.368625+0000","last_unstale":"2024-05-04T03:08:16.377000+0000","last_undegraded":"2024-05-04T03:08:16.377000+0000","last_fullsized":"2024-05-04T03:08:16.377000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:27:50.504487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245652+0000","last_change":"2024-05-04T03:07:34.157346+0000","last_active":"2024-05-04T03:08:16.245652+0000","last_peered":"2024-05-04T03:08:16.245652+0000","last_clean":"2024-05-04T03:08:16.245652+0000","last_became_active":"2024-05-04T03:07:34.156914+0000","last_became_peered":"2024-05-04T03:07:34.156914+0000","last_unstale":"2024-05-04T03:08:16.245652+0000","last_undegraded":"2024-05-04T03:08:16.245652+0000","last_fullsized":"2024-05-04T03:08:16.245652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:54:17.059644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248761+0000","last_change":"2024-05-04T03:07:37.054373+0000","last_active":"2024-05-04T03:08:16.248761+0000","last_peered":"2024-05-04T03:08:16.248761+0000","last_clean":"2024-05-04T03:08:16.248761+0000","last_became_active":"2024-05-04T03:07:37.053764+0000","last_became_peered":"2024-05-04T03:07:37.053764+0000","last_unstale":"2024-05-04T03:08:16.248761+0000","last_undegraded":"2024-05-04T03:08:16.248761+0000","last_fullsized":"2024-05-04T03:08:16.248761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:13:47.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:21.079361+0000","last_change":"2024-05-04T03:07:34.020190+0000","last_active":"2024-05-04T03:08:21.079361+0000","last_peered":"2024-05-04T03:08:21.079361+0000","last_clean":"2024-05-04T03:08:21.079361+0000","last_became_active":"2024-05-04T03:07:34.019786+0000","last_became_peered":"2024-05-04T03:07:34.019786+0000","last_unstale":"2024-05-04T03:08:21.079361+0000","last_undegraded":"2024-05-04T03:08:21.079361+0000","last_fullsized":"2024-05-04T03:08:21.079361+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:11.243414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244079+0000","last_change":"2024-05-04T03:07:37.055274+0000","last_active":"2024-05-04T03:08:16.244079+0000","last_peered":"2024-05-04T03:08:16.244079+0000","last_clean":"2024-05-04T03:08:16.244079+0000","last_became_active":"2024-05-04T03:07:37.054968+0000","last_became_peered":"2024-05-04T03:07:37.054968+0000","last_unstale":"2024-05-04T03:08:16.244079+0000","last_undegraded":"2024-05-04T03:08:16.244079+0000","last_fullsized":"2024-05-04T03:08:16.244079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.229266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367455+0000","last_change":"2024-05-04T03:07:34.132966+0000","last_active":"2024-05-04T03:08:16.367455+0000","last_peered":"2024-05-04T03:08:16.367455+0000","last_clean":"2024-05-04T03:08:16.367455+0000","last_became_active":"2024-05-04T03:07:34.132529+0000","last_became_peered":"2024-05-04T03:07:34.132529+0000","last_unstale":"2024-05-04T03:08:16.367455+0000","last_undegraded":"2024-05-04T03:08:16.367455+0000","last_fullsized":"2024-05-04T03:08:16.367455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:31:37.122097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.368747+0000","last_change":"2024-05-04T03:08:16.368747+0000","last_active":"2024-05-04T03:08:16.368747+0000","last_peered":"2024-05-04T03:08:16.368747+0000","last_clean":"2024-05-04T03:08:16.368747+0000","last_became_active":"2024-05-04T03:08:16.368376+0000","last_became_peered":"2024-05-04T03:08:16.368376+0000","last_unstale":"2024-05-04T03:08:16.368747+0000","last_undegraded":"2024-05-04T03:08:16.368747+0000","last_fullsized":"2024-05-04T03:08:16.368747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:42:32.085650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242445+0000","last_change":"2024-05-04T03:07:34.157219+0000","last_active":"2024-05-04T03:08:16.242445+0000","last_peered":"2024-05-04T03:08:16.242445+0000","last_clean":"2024-05-04T03:08:16.242445+0000","last_became_active":"2024-05-04T03:07:34.156866+0000","last_became_peered":"2024-05-04T03:07:34.156866+0000","last_unstale":"2024-05-04T03:08:16.242445+0000","last_undegraded":"2024-05-04T03:08:16.242445+0000","last_fullsized":"2024-05-04T03:08:16.242445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:15:34.138825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245287+0000","last_change":"2024-05-04T03:07:37.058733+0000","last_active":"2024-05-04T03:08:16.245287+0000","last_peered":"2024-05-04T03:08:16.245287+0000","last_clean":"2024-05-04T03:08:16.245287+0000","last_became_active":"2024-05-04T03:07:37.058396+0000","last_became_peered":"2024-05-04T03:07:37.058396+0000","last_unstale":"2024-05-04T03:08:16.245287+0000","last_undegraded":"2024-05-04T03:08:16.245287+0000","last_fullsized":"2024-05-04T03:08:16.245287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:35.563728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243514+0000","last_change":"2024-05-04T03:07:34.016458+0000","last_active":"2024-05-04T03:08:16.243514+0000","last_peered":"2024-05-04T03:08:16.243514+0000","last_clean":"2024-05-04T03:08:16.243514+0000","last_became_active":"2024-05-04T03:07:34.016038+0000","last_became_peered":"2024-05-04T03:07:34.016038+0000","last_unstale":"2024-05-04T03:08:16.243514+0000","last_undegraded":"2024-05-04T03:08:16.243514+0000","last_fullsized":"2024-05-04T03:08:16.243514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:39:25.156047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247376+0000","last_change":"2024-05-04T03:07:37.064024+0000","last_active":"2024-05-04T03:08:16.247376+0000","last_peered":"2024-05-04T03:08:16.247376+0000","last_clean":"2024-05-04T03:08:16.247376+0000","last_became_active":"2024-05-04T03:07:37.063642+0000","last_became_peered":"2024-05-04T03:07:37.063642+0000","last_unstale":"2024-05-04T03:08:16.247376+0000","last_undegraded":"2024-05-04T03:08:16.247376+0000","last_fullsized":"2024-05-04T03:08:16.247376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:59:58.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.267243+0000","last_change":"2024-05-04T03:08:16.266763+0000","last_active":"2024-05-04T03:08:16.267243+0000","last_peered":"2024-05-04T03:08:16.267243+0000","last_clean":"2024-05-04T03:08:16.267243+0000","last_became_active":"2024-05-04T03:08:16.251212+0000","last_became_peered":"2024-05-04T03:08:16.251212+0000","last_unstale":"2024-05-04T03:08:16.267243+0000","last_undegraded":"2024-05-04T03:08:16.267243+0000","last_fullsized":"2024-05-04T03:08:16.267243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:31:27.444610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243667+0000","last_change":"2024-05-04T03:07:37.063753+0000","last_active":"2024-05-04T03:08:16.243667+0000","last_peered":"2024-05-04T03:08:16.243667+0000","last_clean":"2024-05-04T03:08:16.243667+0000","last_became_active":"2024-05-04T03:07:37.063318+0000","last_became_peered":"2024-05-04T03:07:37.063318+0000","last_unstale":"2024-05-04T03:08:16.243667+0000","last_undegraded":"2024-05-04T03:08:16.243667+0000","last_fullsized":"2024-05-04T03:08:16.243667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:48.217136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245115+0000","last_change":"2024-05-04T03:07:34.022522+0000","last_active":"2024-05-04T03:08:16.245115+0000","last_peered":"2024-05-04T03:08:16.245115+0000","last_clean":"2024-05-04T03:08:16.245115+0000","last_became_active":"2024-05-04T03:07:34.021018+0000","last_became_peered":"2024-05-04T03:07:34.021018+0000","last_unstale":"2024-05-04T03:08:16.245115+0000","last_undegraded":"2024-05-04T03:08:16.245115+0000","last_fullsized":"2024-05-04T03:08:16.245115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:47:04.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244631+0000","last_change":"2024-05-04T03:07:37.069930+0000","last_active":"2024-05-04T03:08:16.244631+0000","last_peered":"2024-05-04T03:08:16.244631+0000","last_clean":"2024-05-04T03:08:16.244631+0000","last_became_active":"2024-05-04T03:07:37.069580+0000","last_became_peered":"2024-05-04T03:07:37.069580+0000","last_unstale":"2024-05-04T03:08:16.244631+0000","last_undegraded":"2024-05-04T03:08:16.244631+0000","last_fullsized":"2024-05-04T03:08:16.244631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:15:33.815970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":137,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:41.398130+0000","last_change":"2024-05-04T03:07:34.157969+0000","last_active":"2024-05-04T03:09:41.398130+0000","last_peered":"2024-05-04T03:09:41.398130+0000","last_clean":"2024-05-04T03:09:41.398130+0000","last_became_active":"2024-05-04T03:07:34.157563+0000","last_became_peered":"2024-05-04T03:07:34.157563+0000","last_unstale":"2024-05-04T03:09:41.398130+0000","last_undegraded":"2024-05-04T03:09:41.398130+0000","last_fullsized":"2024-05-04T03:09:41.398130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:00:43.562109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245064+0000","last_change":"2024-05-04T03:07:37.071279+0000","last_active":"2024-05-04T03:08:16.245064+0000","last_peered":"2024-05-04T03:08:16.245064+0000","last_clean":"2024-05-04T03:08:16.245064+0000","last_became_active":"2024-05-04T03:07:37.070449+0000","last_became_peered":"2024-05-04T03:07:37.070449+0000","last_unstale":"2024-05-04T03:08:16.245064+0000","last_undegraded":"2024-05-04T03:08:16.245064+0000","last_fullsized":"2024-05-04T03:08:16.245064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:13:22.535788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248538+0000","last_change":"2024-05-04T03:07:34.017010+0000","last_active":"2024-05-04T03:08:16.248538+0000","last_peered":"2024-05-04T03:08:16.248538+0000","last_clean":"2024-05-04T03:08:16.248538+0000","last_became_active":"2024-05-04T03:07:34.016521+0000","last_became_peered":"2024-05-04T03:07:34.016521+0000","last_unstale":"2024-05-04T03:08:16.248538+0000","last_undegraded":"2024-05-04T03:08:16.248538+0000","last_fullsized":"2024-05-04T03:08:16.248538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:36.684656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245167+0000","last_change":"2024-05-04T03:07:37.066526+0000","last_active":"2024-05-04T03:08:16.245167+0000","last_peered":"2024-05-04T03:08:16.245167+0000","last_clean":"2024-05-04T03:08:16.245167+0000","last_became_active":"2024-05-04T03:07:37.065312+0000","last_became_peered":"2024-05-04T03:07:37.065312+0000","last_unstale":"2024-05-04T03:08:16.245167+0000","last_undegraded":"2024-05-04T03:08:16.245167+0000","last_fullsized":"2024-05-04T03:08:16.245167+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:10.077295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248927+0000","last_change":"2024-05-04T03:07:34.016898+0000","last_active":"2024-05-04T03:08:16.248927+0000","last_peered":"2024-05-04T03:08:16.248927+0000","last_clean":"2024-05-04T03:08:16.248927+0000","last_became_active":"2024-05-04T03:07:34.016524+0000","last_became_peered":"2024-05-04T03:07:34.016524+0000","last_unstale":"2024-05-04T03:08:16.248927+0000","last_undegraded":"2024-05-04T03:08:16.248927+0000","last_fullsized":"2024-05-04T03:08:16.248927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:01.236207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245122+0000","last_change":"2024-05-04T03:07:37.061501+0000","last_active":"2024-05-04T03:08:16.245122+0000","last_peered":"2024-05-04T03:08:16.245122+0000","last_clean":"2024-05-04T03:08:16.245122+0000","last_became_active":"2024-05-04T03:07:37.061077+0000","last_became_peered":"2024-05-04T03:07:37.061077+0000","last_unstale":"2024-05-04T03:08:16.245122+0000","last_undegraded":"2024-05-04T03:08:16.245122+0000","last_fullsized":"2024-05-04T03:08:16.245122+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:32.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247556+0000","last_change":"2024-05-04T03:07:37.064612+0000","last_active":"2024-05-04T03:08:16.247556+0000","last_peered":"2024-05-04T03:08:16.247556+0000","last_clean":"2024-05-04T03:08:16.247556+0000","last_became_active":"2024-05-04T03:07:37.064303+0000","last_became_peered":"2024-05-04T03:07:37.064303+0000","last_unstale":"2024-05-04T03:08:16.247556+0000","last_undegraded":"2024-05-04T03:08:16.247556+0000","last_fullsized":"2024-05-04T03:08:16.247556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:47:07.811765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250359+0000","last_change":"2024-05-04T03:07:37.061577+0000","last_active":"2024-05-04T03:08:16.250359+0000","last_peered":"2024-05-04T03:08:16.250359+0000","last_clean":"2024-05-04T03:08:16.250359+0000","last_became_active":"2024-05-04T03:07:37.061214+0000","last_became_peered":"2024-05-04T03:07:37.061214+0000","last_unstale":"2024-05-04T03:08:16.250359+0000","last_undegraded":"2024-05-04T03:08:16.250359+0000","last_fullsized":"2024-05-04T03:08:16.250359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:52:11.733257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.251385+0000","last_change":"2024-05-04T03:07:37.062906+0000","last_active":"2024-05-04T03:08:16.251385+0000","last_peered":"2024-05-04T03:08:16.251385+0000","last_clean":"2024-05-04T03:08:16.251385+0000","last_became_active":"2024-05-04T03:07:37.062510+0000","last_became_peered":"2024-05-04T03:07:37.062510+0000","last_unstale":"2024-05-04T03:08:16.251385+0000","last_undegraded":"2024-05-04T03:08:16.251385+0000","last_fullsized":"2024-05-04T03:08:16.251385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:57:58.910490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246507+0000","last_change":"2024-05-04T03:07:37.068367+0000","last_active":"2024-05-04T03:08:16.246507+0000","last_peered":"2024-05-04T03:08:16.246507+0000","last_clean":"2024-05-04T03:08:16.246507+0000","last_became_active":"2024-05-04T03:07:37.068038+0000","last_became_peered":"2024-05-04T03:07:37.068038+0000","last_unstale":"2024-05-04T03:08:16.246507+0000","last_undegraded":"2024-05-04T03:08:16.246507+0000","last_fullsized":"2024-05-04T03:08:16.246507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:11:19.195002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250759+0000","last_change":"2024-05-04T03:08:16.250759+0000","last_active":"2024-05-04T03:08:16.250759+0000","last_peered":"2024-05-04T03:08:16.250759+0000","last_clean":"2024-05-04T03:08:16.250759+0000","last_became_active":"2024-05-04T03:08:16.250379+0000","last_became_peered":"2024-05-04T03:08:16.250379+0000","last_unstale":"2024-05-04T03:08:16.250759+0000","last_undegraded":"2024-05-04T03:08:16.250759+0000","last_fullsized":"2024-05-04T03:08:16.250759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:58:09.419009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246960+0000","last_change":"2024-05-04T03:07:37.064584+0000","last_active":"2024-05-04T03:08:16.246960+0000","last_peered":"2024-05-04T03:08:16.246960+0000","last_clean":"2024-05-04T03:08:16.246960+0000","last_became_active":"2024-05-04T03:07:37.064183+0000","last_became_peered":"2024-05-04T03:07:37.064183+0000","last_unstale":"2024-05-04T03:08:16.246960+0000","last_undegraded":"2024-05-04T03:08:16.246960+0000","last_fullsized":"2024-05-04T03:08:16.246960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:17.121081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.250902+0000","last_change":"2024-05-04T03:07:37.066742+0000","last_active":"2024-05-04T03:08:16.250902+0000","last_peered":"2024-05-04T03:08:16.250902+0000","last_clean":"2024-05-04T03:08:16.250902+0000","last_became_active":"2024-05-04T03:07:37.066126+0000","last_became_peered":"2024-05-04T03:07:37.066126+0000","last_unstale":"2024-05-04T03:08:16.250902+0000","last_undegraded":"2024-05-04T03:08:16.250902+0000","last_fullsized":"2024-05-04T03:08:16.250902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:57.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244050+0000","last_change":"2024-05-04T03:07:37.068122+0000","last_active":"2024-05-04T03:08:16.244050+0000","last_peered":"2024-05-04T03:08:16.244050+0000","last_clean":"2024-05-04T03:08:16.244050+0000","last_became_active":"2024-05-04T03:07:37.066321+0000","last_became_peered":"2024-05-04T03:07:37.066321+0000","last_unstale":"2024-05-04T03:08:16.244050+0000","last_undegraded":"2024-05-04T03:08:16.244050+0000","last_fullsized":"2024-05-04T03:08:16.244050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:25:09.321853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247940+0000","last_change":"2024-05-04T03:07:37.060322+0000","last_active":"2024-05-04T03:08:16.247940+0000","last_peered":"2024-05-04T03:08:16.247940+0000","last_clean":"2024-05-04T03:08:16.247940+0000","last_became_active":"2024-05-04T03:07:37.059944+0000","last_became_peered":"2024-05-04T03:07:37.059944+0000","last_unstale":"2024-05-04T03:08:16.247940+0000","last_undegraded":"2024-05-04T03:08:16.247940+0000","last_fullsized":"2024-05-04T03:08:16.247940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.105085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243469+0000","last_change":"2024-05-04T03:07:37.056269+0000","last_active":"2024-05-04T03:08:16.243469+0000","last_peered":"2024-05-04T03:08:16.243469+0000","last_clean":"2024-05-04T03:08:16.243469+0000","last_became_active":"2024-05-04T03:07:37.055886+0000","last_became_peered":"2024-05-04T03:07:37.055886+0000","last_unstale":"2024-05-04T03:08:16.243469+0000","last_undegraded":"2024-05-04T03:08:16.243469+0000","last_fullsized":"2024-05-04T03:08:16.243469+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:55:11.292577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242733+0000","last_change":"2024-05-04T03:07:37.071013+0000","last_active":"2024-05-04T03:08:16.242733+0000","last_peered":"2024-05-04T03:08:16.242733+0000","last_clean":"2024-05-04T03:08:16.242733+0000","last_became_active":"2024-05-04T03:07:37.069346+0000","last_became_peered":"2024-05-04T03:07:37.069346+0000","last_unstale":"2024-05-04T03:08:16.242733+0000","last_undegraded":"2024-05-04T03:08:16.242733+0000","last_fullsized":"2024-05-04T03:08:16.242733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:19:11.898766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.249009+0000","last_change":"2024-05-04T03:07:37.064962+0000","last_active":"2024-05-04T03:08:16.249009+0000","last_peered":"2024-05-04T03:08:16.249009+0000","last_clean":"2024-05-04T03:08:16.249009+0000","last_became_active":"2024-05-04T03:07:37.064637+0000","last_became_peered":"2024-05-04T03:07:37.064637+0000","last_unstale":"2024-05-04T03:08:16.249009+0000","last_undegraded":"2024-05-04T03:08:16.249009+0000","last_fullsized":"2024-05-04T03:08:16.249009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:10:10.659621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246565+0000","last_change":"2024-05-04T03:07:37.056721+0000","last_active":"2024-05-04T03:08:16.246565+0000","last_peered":"2024-05-04T03:08:16.246565+0000","last_clean":"2024-05-04T03:08:16.246565+0000","last_became_active":"2024-05-04T03:07:37.056353+0000","last_became_peered":"2024-05-04T03:07:37.056353+0000","last_unstale":"2024-05-04T03:08:16.246565+0000","last_undegraded":"2024-05-04T03:08:16.246565+0000","last_fullsized":"2024-05-04T03:08:16.246565+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:30:47.476579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286351,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67160,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1074,"num_read_kb":608,"num_write":244,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710762,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":880640,"data_stored":425832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1077531,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070516,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":425694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266745,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":426072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462974,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1018590,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":425411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":486671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":426750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":486823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":425236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":425663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822728,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":1017877,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:09:44.927 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-04T03:09:44.927 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-04T03:09:44.927 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-04T03:09:44.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-04T03:09:45.209 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:45.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:45 smithi066 ceph-mon[33264]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:09:45.605 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:45 smithi039 ceph-mon[29715]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:09:45.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:45 smithi167 ceph-mon[33614]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:09:46.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:46 smithi039 ceph-mon[29715]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:46.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:46 smithi039 ceph-mon[29715]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-04T03:09:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:46 smithi066 ceph-mon[33264]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:46 smithi066 ceph-mon[33264]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-04T03:09:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:46 smithi167 ceph-mon[33614]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:09:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:46 smithi167 ceph-mon[33614]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-04T03:09:47.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 -- 172.21.15.39:0/216564076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc109870 msgr2=0x7f5abc1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:47.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 --2- 172.21.15.39:0/216564076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc109870 0x7f5abc1111d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400b0d0 tx=0x7f5ab400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 -- 172.21.15.39:0/216564076 shutdown_connections 2024-05-04T03:09:47.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 --2- 172.21.15.39:0/216564076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc109870 0x7f5abc1111d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 --2- 172.21.15.39:0/216564076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5abc1045f0 0x7f5abc104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 --2- 172.21.15.39:0/216564076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc108f20 0x7f5abc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 -- 172.21.15.39:0/216564076 >> 172.21.15.39:0/216564076 conn(0x7f5abc100040 msgr2=0x7f5abc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:47.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 -- 172.21.15.39:0/216564076 shutdown_connections 2024-05-04T03:09:47.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 -- 172.21.15.39:0/216564076 wait complete. 2024-05-04T03:09:47.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.069+0000 7f5ac2324700 1 Processor -- start 2024-05-04T03:09:47.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5ac2324700 1 -- start start 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5ac2324700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5abc1045f0 0x7f5abc19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5ac2324700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc108f20 0x7f5abc19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5ac2324700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc109870 0x7f5abc19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5ac2324700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc114350 con 0x7f5abc108f20 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5ac2324700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc1141d0 con 0x7f5abc1045f0 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5ac2324700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc1144d0 con 0x7f5abc109870 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5abb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc108f20 0x7f5abc19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5abb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc108f20 0x7f5abc19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51360/0 (socket says 172.21.15.39:51360) 2024-05-04T03:09:47.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5abb7fe700 1 -- 172.21.15.39:0/1334450131 learned_addr learned my addr 172.21.15.39:0/1334450131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5abbfff700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5abc1045f0 0x7f5abc19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.070+0000 7f5ac08c1700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc109870 0x7f5abc19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5abb7fe700 1 -- 172.21.15.39:0/1334450131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc109870 msgr2=0x7f5abc19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc109870 0x7f5abc19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5abb7fe700 1 -- 172.21.15.39:0/1334450131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5abc1045f0 msgr2=0x7f5abc19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5abc1045f0 0x7f5abc19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5abb7fe700 1 -- 172.21.15.39:0/1334450131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5abc19fd30 con 0x7f5abc108f20 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5abb7fe700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc108f20 0x7f5abc19b400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c8f0 tx=0x7f5ab000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5ab97fa700 1 -- 172.21.15.39:0/1334450131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00078d0 con 0x7f5abc108f20 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5ab97fa700 1 -- 172.21.15.39:0/1334450131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ab000f450 con 0x7f5abc108f20 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5abc1a0020 con 0x7f5abc108f20 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5ab97fa700 1 -- 172.21.15.39:0/1334450131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0018660 con 0x7f5abc108f20 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.071+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5abc1a7940 con 0x7f5abc108f20 2024-05-04T03:09:47.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.072+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5abc0512c0 con 0x7f5abc108f20 2024-05-04T03:09:47.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.075+0000 7f5ab97fa700 1 -- 172.21.15.39:0/1334450131 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5ab000f900 con 0x7f5abc108f20 2024-05-04T03:09:47.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.075+0000 7f5ab97fa700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5aa40b2880 0x7f5aa40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:47.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.076+0000 7f5abbfff700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5aa40b2880 0x7f5aa40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:47.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.076+0000 7f5ab97fa700 1 -- 172.21.15.39:0/1334450131 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5ab00e1fa0 con 0x7f5abc108f20 2024-05-04T03:09:47.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.076+0000 7f5abbfff700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5aa40b2880 0x7f5aa40b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5aac005950 tx=0x7f5aac00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:47.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.079+0000 7f5ab97fa700 1 -- 172.21.15.39:0/1334450131 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ab0014030 con 0x7f5abc108f20 2024-05-04T03:09:47.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.342+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5abc19c1c0 con 0x7f5abc108f20 2024-05-04T03:09:47.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.342+0000 7f5ab97fa700 1 -- 172.21.15.39:0/1334450131 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5ab00a3e20 con 0x7f5abc108f20 2024-05-04T03:09:47.343 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:09:47.343 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-04T03:09:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.344+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5aa40b2880 msgr2=0x7f5aa40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.344+0000 7f5ac2324700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5aa40b2880 0x7f5aa40b4d90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5aac005950 tx=0x7f5aac00a300 comp rx=0 tx=0).stop 2024-05-04T03:09:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.344+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc108f20 msgr2=0x7f5abc19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.344+0000 7f5ac2324700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc108f20 0x7f5abc19b400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c8f0 tx=0x7f5ab000cc00 comp rx=0 tx=0).stop 2024-05-04T03:09:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.345+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 shutdown_connections 2024-05-04T03:09:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.345+0000 7f5ac2324700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5aa40b2880 0x7f5aa40b4d90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.345+0000 7f5ac2324700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5abc109870 0x7f5abc19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.345+0000 7f5ac2324700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5abc1045f0 0x7f5abc19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.345+0000 7f5ac2324700 1 --2- 172.21.15.39:0/1334450131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5abc108f20 0x7f5abc19b400 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.345+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 >> 172.21.15.39:0/1334450131 conn(0x7f5abc100040 msgr2=0x7f5abc108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.345+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 shutdown_connections 2024-05-04T03:09:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:47.345+0000 7f5ac2324700 1 -- 172.21.15.39:0/1334450131 wait complete. 2024-05-04T03:09:48.066 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-04T03:09:48.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:09:48.318 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:48.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:48 smithi066 ceph-mon[33264]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-04T03:09:48.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:48 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1334450131' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-04T03:09:48.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:48 smithi039 ceph-mon[29715]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-04T03:09:48.583 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:48 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1334450131' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-04T03:09:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:48 smithi167 ceph-mon[33614]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-04T03:09:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:48 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1334450131' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 -- 172.21.15.39:0/4024190783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 msgr2=0x7f7b4c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 --2- 172.21.15.39:0/4024190783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 0x7f7b4c109320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0099c0 tx=0x7f7b3c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 -- 172.21.15.39:0/4024190783 shutdown_connections 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 --2- 172.21.15.39:0/4024190783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b4c109860 0x7f7b4c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 --2- 172.21.15.39:0/4024190783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c1045e0 0x7f7b4c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 --2- 172.21.15.39:0/4024190783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 0x7f7b4c109320 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 -- 172.21.15.39:0/4024190783 >> 172.21.15.39:0/4024190783 conn(0x7f7b4c0ffed0 msgr2=0x7f7b4c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 -- 172.21.15.39:0/4024190783 shutdown_connections 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.067+0000 7f7b5417d700 1 -- 172.21.15.39:0/4024190783 wait complete. 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b5417d700 1 Processor -- start 2024-05-04T03:09:50.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b5417d700 1 -- start start 2024-05-04T03:09:50.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b5417d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c1045e0 0x7f7b4c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:50.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b5417d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 0x7f7b4c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b5417d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b4c109860 0x7f7b4c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b5417d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c114390 con 0x7f7b4c108f10 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b5417d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c114210 con 0x7f7b4c1045e0 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b5417d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c114510 con 0x7f7b4c109860 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.068+0000 7f7b51718700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 0x7f7b4c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51718700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 0x7f7b4c19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51384/0 (socket says 172.21.15.39:51384) 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51718700 1 -- 172.21.15.39:0/1828237655 learned_addr learned my addr 172.21.15.39:0/1828237655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51f19700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c1045e0 0x7f7b4c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b5271a700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b4c109860 0x7f7b4c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51718700 1 -- 172.21.15.39:0/1828237655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b4c109860 msgr2=0x7f7b4c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51718700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b4c109860 0x7f7b4c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51718700 1 -- 172.21.15.39:0/1828237655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c1045e0 msgr2=0x7f7b4c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51718700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c1045e0 0x7f7b4c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51718700 1 -- 172.21.15.39:0/1828237655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c19fc90 con 0x7f7b4c108f10 2024-05-04T03:09:50.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b5271a700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b4c109860 0x7f7b4c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:50.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b51718700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 0x7f7b4c19b360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00b580 tx=0x7f7b3c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:50.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b42ffd700 1 -- 172.21.15.39:0/1828237655 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00f8a0 con 0x7f7b4c108f10 2024-05-04T03:09:50.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b42ffd700 1 -- 172.21.15.39:0/1828237655 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b3c004830 con 0x7f7b4c108f10 2024-05-04T03:09:50.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c19ff20 con 0x7f7b4c108f10 2024-05-04T03:09:50.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b42ffd700 1 -- 172.21.15.39:0/1828237655 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c005350 con 0x7f7b4c108f10 2024-05-04T03:09:50.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.069+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1a03f0 con 0x7f7b4c108f10 2024-05-04T03:09:50.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.070+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4c0512c0 con 0x7f7b4c108f10 2024-05-04T03:09:50.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.072+0000 7f7b42ffd700 1 -- 172.21.15.39:0/1828237655 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7b3c029060 con 0x7f7b4c108f10 2024-05-04T03:09:50.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.073+0000 7f7b42ffd700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7b380b2900 0x7f7b380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:50.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.073+0000 7f7b51f19700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7b380b2900 0x7f7b380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:50.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.073+0000 7f7b42ffd700 1 -- 172.21.15.39:0/1828237655 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7b3c0e2fa0 con 0x7f7b4c108f10 2024-05-04T03:09:50.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.073+0000 7f7b51f19700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7b380b2900 0x7f7b380b4e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b48005950 tx=0x7f7b4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:50.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.077+0000 7f7b42ffd700 1 -- 172.21.15.39:0/1828237655 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b3c013070 con 0x7f7b4c108f10 2024-05-04T03:09:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.338+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b4c02e5f0 con 0x7f7b4c108f10 2024-05-04T03:09:50.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:50 smithi039 ceph-mon[29715]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.339+0000 7f7b42ffd700 1 -- 172.21.15.39:0/1828237655 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f7b3c0aad60 con 0x7f7b4c108f10 2024-05-04T03:09:50.340 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:09:50.340 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:08:01.344495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24455,24490,24466,24472,24506]},"id":1}]} 2024-05-04T03:09:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.341+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7b380b2900 msgr2=0x7f7b380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.341+0000 7f7b5417d700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7b380b2900 0x7f7b380b4e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b48005950 tx=0x7f7b4800b410 comp rx=0 tx=0).stop 2024-05-04T03:09:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.341+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 msgr2=0x7f7b4c19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 0x7f7b4c19b360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00b580 tx=0x7f7b3c004790 comp rx=0 tx=0).stop 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 shutdown_connections 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7b380b2900 0x7f7b380b4e10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b4c109860 0x7f7b4c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b4c1045e0 0x7f7b4c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 --2- 172.21.15.39:0/1828237655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b4c108f10 0x7f7b4c19b360 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 >> 172.21.15.39:0/1828237655 conn(0x7f7b4c0ffed0 msgr2=0x7f7b4c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 shutdown_connections 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:50.342+0000 7f7b5417d700 1 -- 172.21.15.39:0/1828237655 wait complete. 2024-05-04T03:09:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-04T03:09:50.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:50 smithi066 ceph-mon[33264]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:50 smithi167 ceph-mon[33614]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:50.996 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:09:51.250 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:51.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:51 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1828237655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:51.638 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:51 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1828237655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:51.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:51 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1828237655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:52.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:52 smithi066 ceph-mon[33264]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:52.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:52 smithi039 ceph-mon[29715]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:52.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:52 smithi167 ceph-mon[33614]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:52.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 -- 172.21.15.39:0/1841226011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04108f00 msgr2=0x7f3a04109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/1841226011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04108f00 0x7f3a04109310 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0099c0 tx=0x7f39ec009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 -- 172.21.15.39:0/1841226011 shutdown_connections 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/1841226011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a04109850 0x7f3a041111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/1841226011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a041045d0 0x7f3a041049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/1841226011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04108f00 0x7f3a04109310 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 -- 172.21.15.39:0/1841226011 >> 172.21.15.39:0/1841226011 conn(0x7f3a040ffea0 msgr2=0x7f3a041022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 -- 172.21.15.39:0/1841226011 shutdown_connections 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 -- 172.21.15.39:0/1841226011 wait complete. 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.980+0000 7f3a09a0e700 1 Processor -- start 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a09a0e700 1 -- start start 2024-05-04T03:09:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a09a0e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a041045d0 0x7f3a0419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a09a0e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a04108f00 0x7f3a0419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a09a0e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04109850 0x7f3a0419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a09a0e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04114310 con 0x7f3a04109850 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a09a0e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04114190 con 0x7f3a04108f00 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a09a0e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04114490 con 0x7f3a041045d0 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04109850 0x7f3a0419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a037fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04109850 0x7f3a0419f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51392/0 (socket says 172.21.15.39:51392) 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a037fe700 1 -- 172.21.15.39:0/4265532901 learned_addr learned my addr 172.21.15.39:0/4265532901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a02ffd700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a041045d0 0x7f3a0419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a027fc700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a04108f00 0x7f3a0419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a037fe700 1 -- 172.21.15.39:0/4265532901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a041045d0 msgr2=0x7f3a0419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a037fe700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a041045d0 0x7f3a0419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a037fe700 1 -- 172.21.15.39:0/4265532901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a04108f00 msgr2=0x7f3a0419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:52.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a037fe700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a04108f00 0x7f3a0419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:52.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.981+0000 7f3a037fe700 1 -- 172.21.15.39:0/4265532901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0419fc50 con 0x7f3a04109850 2024-05-04T03:09:52.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.982+0000 7f3a037fe700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04109850 0x7f3a0419f5b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00ada0 tx=0x7f39fc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:52.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.982+0000 7f3a08a0c700 1 -- 172.21.15.39:0/4265532901 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc017020 con 0x7f3a04109850 2024-05-04T03:09:52.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.982+0000 7f3a08a0c700 1 -- 172.21.15.39:0/4265532901 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39fc00cd60 con 0x7f3a04109850 2024-05-04T03:09:52.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.982+0000 7f3a08a0c700 1 -- 172.21.15.39:0/4265532901 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc005280 con 0x7f3a04109850 2024-05-04T03:09:52.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.982+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0419ff40 con 0x7f3a04109850 2024-05-04T03:09:52.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.982+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a041a7860 con 0x7f3a04109850 2024-05-04T03:09:52.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.983+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a040512c0 con 0x7f3a04109850 2024-05-04T03:09:52.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.984+0000 7f3a08a0c700 1 -- 172.21.15.39:0/4265532901 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f39fc00c8d0 con 0x7f3a04109850 2024-05-04T03:09:52.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.985+0000 7f3a08a0c700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f39f00b2900 0x7f39f00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:52.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.985+0000 7f3a02ffd700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f39f00b2900 0x7f39f00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:52.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.985+0000 7f3a08a0c700 1 -- 172.21.15.39:0/4265532901 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f39fc0e1900 con 0x7f3a04109850 2024-05-04T03:09:52.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.986+0000 7f3a02ffd700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f39f00b2900 0x7f39f00b4e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f39f4007be0 tx=0x7f39f40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:52.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:52.990+0000 7f3a08a0c700 1 -- 172.21.15.39:0/4265532901 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39fc021020 con 0x7f3a04109850 2024-05-04T03:09:53.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.255+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a0402e5f0 con 0x7f3a04109850 2024-05-04T03:09:53.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.255+0000 7f3a08a0c700 1 -- 172.21.15.39:0/4265532901 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f39fc0a96c0 con 0x7f3a04109850 2024-05-04T03:09:53.256 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:09:53.257 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:08:01.344495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24455,24490,24466,24472,24506]},"id":1}]} 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.257+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f39f00b2900 msgr2=0x7f39f00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.257+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f39f00b2900 0x7f39f00b4e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f39f4007be0 tx=0x7f39f40073d0 comp rx=0 tx=0).stop 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.257+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04109850 msgr2=0x7f3a0419f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.257+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04109850 0x7f3a0419f5b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00ada0 tx=0x7f39fc00c330 comp rx=0 tx=0).stop 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.258+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 shutdown_connections 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.258+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f39f00b2900 0x7f39f00b4e10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.258+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a04109850 0x7f3a0419f5b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.258+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a041045d0 0x7f3a0419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.258+0000 7f3a09a0e700 1 --2- 172.21.15.39:0/4265532901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3a04108f00 0x7f3a0419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.258+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 >> 172.21.15.39:0/4265532901 conn(0x7f3a040ffea0 msgr2=0x7f3a041022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.258+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 shutdown_connections 2024-05-04T03:09:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:53.258+0000 7f3a09a0e700 1 -- 172.21.15.39:0/4265532901 wait complete. 2024-05-04T03:09:53.260 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-04T03:09:53.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:53 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4265532901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:53.625 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:53 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4265532901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:53.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:53 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4265532901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:53.960 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:09:54.214 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:54.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:54 smithi039 ceph-mon[29715]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:09:54.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:54 smithi167 ceph-mon[33614]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:09:54.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:54 smithi066 ceph-mon[33264]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-04T03:09:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 -- 172.21.15.39:0/2726084006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24107df0 msgr2=0x7f3e24108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/2726084006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24107df0 0x7f3e24108200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e180099c0 tx=0x7f3e18009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 -- 172.21.15.39:0/2726084006 shutdown_connections 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/2726084006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e24108740 0x7f3e241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/2726084006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e241034c0 0x7f3e241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/2726084006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24107df0 0x7f3e24108200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 -- 172.21.15.39:0/2726084006 >> 172.21.15.39:0/2726084006 conn(0x7f3e240fedc0 msgr2=0x7f3e241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 -- 172.21.15.39:0/2726084006 shutdown_connections 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.937+0000 7f3e2afd0700 1 -- 172.21.15.39:0/2726084006 wait complete. 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.938+0000 7f3e2afd0700 1 Processor -- start 2024-05-04T03:09:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.938+0000 7f3e2afd0700 1 -- start start 2024-05-04T03:09:55.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.938+0000 7f3e2afd0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e241034c0 0x7f3e2419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:55.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.938+0000 7f3e2afd0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e24107df0 0x7f3e2419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:55.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.938+0000 7f3e2afd0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24108740 0x7f3e2419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:55.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.938+0000 7f3e2afd0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24114370 con 0x7f3e24108740 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.938+0000 7f3e2afd0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241141f0 con 0x7f3e241034c0 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.938+0000 7f3e2afd0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241144f0 con 0x7f3e24107df0 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24108740 0x7f3e2419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24108740 0x7f3e2419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53968/0 (socket says 172.21.15.39:53968) 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 -- 172.21.15.39:0/3044145969 learned_addr learned my addr 172.21.15.39:0/3044145969 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e23fff700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e24107df0 0x7f3e2419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e28d6c700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e241034c0 0x7f3e2419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 -- 172.21.15.39:0/3044145969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e24107df0 msgr2=0x7f3e2419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e24107df0 0x7f3e2419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 -- 172.21.15.39:0/3044145969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e241034c0 msgr2=0x7f3e2419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e241034c0 0x7f3e2419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 -- 172.21.15.39:0/3044145969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2419fcb0 con 0x7f3e24108740 2024-05-04T03:09:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2956d700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24108740 0x7f3e2419f610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400d980 tx=0x7f3e1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e21ffb700 1 -- 172.21.15.39:0/3044145969 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14017070 con 0x7f3e24108740 2024-05-04T03:09:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e21ffb700 1 -- 172.21.15.39:0/3044145969 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e14005020 con 0x7f3e24108740 2024-05-04T03:09:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e21ffb700 1 -- 172.21.15.39:0/3044145969 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14009990 con 0x7f3e24108740 2024-05-04T03:09:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.939+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2419ffa0 con 0x7f3e24108740 2024-05-04T03:09:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.940+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241a78c0 con 0x7f3e24108740 2024-05-04T03:09:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.941+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e24104970 con 0x7f3e24108740 2024-05-04T03:09:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.942+0000 7f3e21ffb700 1 -- 172.21.15.39:0/3044145969 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3e14004a30 con 0x7f3e24108740 2024-05-04T03:09:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.943+0000 7f3e21ffb700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3e0c0b2900 0x7f3e0c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.943+0000 7f3e28d6c700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3e0c0b2900 0x7f3e0c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.943+0000 7f3e21ffb700 1 -- 172.21.15.39:0/3044145969 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3e140e2390 con 0x7f3e24108740 2024-05-04T03:09:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.944+0000 7f3e28d6c700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3e0c0b2900 0x7f3e0c0b4e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009f20 tx=0x7f3e1c0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:55.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:55.947+0000 7f3e21ffb700 1 -- 172.21.15.39:0/3044145969 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e14020020 con 0x7f3e24108740 2024-05-04T03:09:56.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.161+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e2402e5f0 con 0x7f3e24108740 2024-05-04T03:09:56.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.163+0000 7f3e21ffb700 1 -- 172.21.15.39:0/3044145969 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22541 (secure 0 0 0) 0x7f3e140aa140 con 0x7f3e24108740 2024-05-04T03:09:56.164 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:09:56.164 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:08:16.237923+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:09:56.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3e0c0b2900 msgr2=0x7f3e0c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:56.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3e0c0b2900 0x7f3e0c0b4e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009f20 tx=0x7f3e1c0094f0 comp rx=0 tx=0).stop 2024-05-04T03:09:56.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24108740 msgr2=0x7f3e2419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:56.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24108740 0x7f3e2419f610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400d980 tx=0x7f3e1400dc90 comp rx=0 tx=0).stop 2024-05-04T03:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 shutdown_connections 2024-05-04T03:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3e0c0b2900 0x7f3e0c0b4e10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e24108740 0x7f3e2419f610 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e241034c0 0x7f3e2419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 --2- 172.21.15.39:0/3044145969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e24107df0 0x7f3e2419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.165+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 >> 172.21.15.39:0/3044145969 conn(0x7f3e240fedc0 msgr2=0x7f3e241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.166+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 shutdown_connections 2024-05-04T03:09:56.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:56.166+0000 7f3e2afd0700 1 -- 172.21.15.39:0/3044145969 wait complete. 2024-05-04T03:09:56.529 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:56 smithi039 ceph-mon[29715]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-04T03:09:56.529 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:56 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3044145969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:09:56.620 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:56 smithi066 ceph-mon[33264]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-04T03:09:56.620 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:56 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3044145969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:09:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:56 smithi167 ceph-mon[33614]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-04T03:09:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:56 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3044145969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:09:56.863 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:09:57.116 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:58.532 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:58 smithi039 ceph-mon[29715]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:58.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:58 smithi167 ceph-mon[33614]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:58.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:58 smithi066 ceph-mon[33264]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-04T03:09:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/4034680403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8108f10 msgr2=0x7ff2e8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/4034680403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8108f10 0x7ff2e8109320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00099c0 tx=0x7ff2d0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:09:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/4034680403 shutdown_connections 2024-05-04T03:09:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/4034680403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2e8109860 0x7ff2e81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/4034680403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e81045e0 0x7ff2e81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/4034680403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8108f10 0x7ff2e8109320 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/4034680403 >> 172.21.15.39:0/4034680403 conn(0x7ff2e80ffef0 msgr2=0x7ff2e8102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/4034680403 shutdown_connections 2024-05-04T03:09:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/4034680403 wait complete. 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.938+0000 7ff2ed2ce700 1 Processor -- start 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2ed2ce700 1 -- start start 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2ed2ce700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2e81045e0 0x7ff2e8076b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2ed2ce700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8108f10 0x7ff2e80770d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2ed2ce700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8109860 0x7ff2e8077610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2ed2ce700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e8114320 con 0x7ff2e8109860 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2ed2ce700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81141a0 con 0x7ff2e81045e0 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2ed2ce700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e81144a0 con 0x7ff2e8108f10 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8109860 0x7ff2e8077610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e77fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8109860 0x7ff2e8077610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53986/0 (socket says 172.21.15.39:53986) 2024-05-04T03:09:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e77fe700 1 -- 172.21.15.39:0/1279869202 learned_addr learned my addr 172.21.15.39:0/1279869202 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e67fc700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8108f10 0x7ff2e80770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e6ffd700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2e81045e0 0x7ff2e8076b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e77fe700 1 -- 172.21.15.39:0/1279869202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8108f10 msgr2=0x7ff2e80770d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e77fe700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8108f10 0x7ff2e80770d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e77fe700 1 -- 172.21.15.39:0/1279869202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2e81045e0 msgr2=0x7ff2e8076b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e77fe700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2e81045e0 0x7ff2e8076b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.939+0000 7ff2e77fe700 1 -- 172.21.15.39:0/1279869202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e81a9140 con 0x7ff2e8109860 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.940+0000 7ff2e77fe700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8109860 0x7ff2e8077610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0007ee0 tx=0x7ff2e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.940+0000 7ff2dffff700 1 -- 172.21.15.39:0/1279869202 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e000ddd0 con 0x7ff2e8109860 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.940+0000 7ff2dffff700 1 -- 172.21.15.39:0/1279869202 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2e0004020 con 0x7ff2e8109860 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.940+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e81a9430 con 0x7ff2e8109860 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.940+0000 7ff2dffff700 1 -- 172.21.15.39:0/1279869202 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0015660 con 0x7ff2e8109860 2024-05-04T03:09:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.940+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e81a98f0 con 0x7ff2e8109860 2024-05-04T03:09:58.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.941+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e80512c0 con 0x7ff2e8109860 2024-05-04T03:09:58.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.942+0000 7ff2dffff700 1 -- 172.21.15.39:0/1279869202 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff2e00041d0 con 0x7ff2e8109860 2024-05-04T03:09:58.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.943+0000 7ff2dffff700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff2d40b2880 0x7ff2d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:58.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.943+0000 7ff2e6ffd700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff2d40b2880 0x7ff2d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:58.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.943+0000 7ff2dffff700 1 -- 172.21.15.39:0/1279869202 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff2e00e23f0 con 0x7ff2e8109860 2024-05-04T03:09:58.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.944+0000 7ff2e6ffd700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff2d40b2880 0x7ff2d40b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005950 tx=0x7ff2d80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:58.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:58.947+0000 7ff2dffff700 1 -- 172.21.15.39:0/1279869202 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2e0010070 con 0x7ff2e8109860 2024-05-04T03:09:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.207+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2e802e5f0 con 0x7ff2e8109860 2024-05-04T03:09:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.208+0000 7ff2dffff700 1 -- 172.21.15.39:0/1279869202 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7ff2e00aa120 con 0x7ff2e8109860 2024-05-04T03:09:59.209 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:09:59.209 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:08:01.344495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24455,24490,24466,24472,24506]},"id":1}]} 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff2d40b2880 msgr2=0x7ff2d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff2d40b2880 0x7ff2d40b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005950 tx=0x7ff2d80058e0 comp rx=0 tx=0).stop 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8109860 msgr2=0x7ff2e8077610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8109860 0x7ff2e8077610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0007ee0 tx=0x7ff2e000d3b0 comp rx=0 tx=0).stop 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 shutdown_connections 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff2d40b2880 0x7ff2d40b4d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e8109860 0x7ff2e8077610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2e81045e0 0x7ff2e8076b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 --2- 172.21.15.39:0/1279869202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e8108f10 0x7ff2e80770d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 >> 172.21.15.39:0/1279869202 conn(0x7ff2e80ffef0 msgr2=0x7ff2e8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 shutdown_connections 2024-05-04T03:09:59.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:09:59.210+0000 7ff2ed2ce700 1 -- 172.21.15.39:0/1279869202 wait complete. 2024-05-04T03:09:59.212 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-04T03:09:59.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:09:59 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1279869202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:59.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:09:59 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1279869202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:59.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:09:59 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1279869202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:09:59.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-04T03:07:39.087961+0000', 'modified': '2024-05-04T03:08:01.344495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24506, 'mds_1': 24490, 'mds_2': 24472, 'mds_3': 24466, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14607': {'gid': 14607, 'name': 'h', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6837/2860496376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 2860496376}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 2860496376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6839/4175892611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 4175892611}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 4175892611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14634': {'gid': 14634, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6841/1534369514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 1534369514}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 1534369514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/1077677237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 1077677237}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 1077677237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/462065299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 462065299}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 462065299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.66:6835/3463909305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 3463909305}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 3463909305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.167:6835/2699315372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 2699315372}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 2699315372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.167:6837/2270685849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2270685849}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2270685849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.167:6839/975309166', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 975309166}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 975309166}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.66:6839/4050747247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 4050747247}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 4050747247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24455, 24490, 24466, 24472, 24506]} 2024-05-04T03:09:59.918 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-04T03:09:59.918 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.e status 2024-05-04T03:10:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:00.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:00 smithi039 ceph-mon[29715]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-04T03:10:00.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:00 smithi039 ceph-mon[29715]: overall HEALTH_OK 2024-05-04T03:10:00.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:00 smithi167 ceph-mon[33614]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-04T03:10:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:00 smithi167 ceph-mon[33614]: overall HEALTH_OK 2024-05-04T03:10:00.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:00 smithi066 ceph-mon[33264]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-04T03:10:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:00 smithi066 ceph-mon[33264]: overall HEALTH_OK 2024-05-04T03:10:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 -- 172.21.15.39:0/2855000668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c103b30 msgr2=0x7f6a8c103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 --2- 172.21.15.39:0/2855000668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c103b30 0x7f6a8c103f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6a880099c0 tx=0x7f6a88009cd0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 -- 172.21.15.39:0/2855000668 shutdown_connections 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 --2- 172.21.15.39:0/2855000668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c103b30 0x7f6a8c103f40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 --2- 172.21.15.39:0/2855000668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a8c1087a0 0x7f6a8c110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 --2- 172.21.15.39:0/2855000668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c107e50 0x7f6a8c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 -- 172.21.15.39:0/2855000668 >> 172.21.15.39:0/2855000668 conn(0x7f6a8c0fe520 msgr2=0x7f6a8c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 -- 172.21.15.39:0/2855000668 shutdown_connections 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.973+0000 7f6a93be6700 1 -- 172.21.15.39:0/2855000668 wait complete. 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a93be6700 1 Processor -- start 2024-05-04T03:10:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a93be6700 1 -- start start 2024-05-04T03:10:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a93be6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a8c103b30 0x7f6a8c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a93be6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c107e50 0x7f6a8c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a93be6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c1087a0 0x7f6a8c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a93be6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c113310 con 0x7f6a8c1087a0 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a93be6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c113190 con 0x7f6a8c107e50 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a93be6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c113490 con 0x7f6a8c103b30 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a92183700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c1087a0 0x7f6a8c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a92183700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c1087a0 0x7f6a8c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54010/0 (socket says 172.21.15.39:54010) 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a92183700 1 -- 172.21.15.39:0/739505596 learned_addr learned my addr 172.21.15.39:0/739505596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a91982700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a8c103b30 0x7f6a8c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.974+0000 7f6a91181700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c107e50 0x7f6a8c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a92183700 1 -- 172.21.15.39:0/739505596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a8c103b30 msgr2=0x7f6a8c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a92183700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a8c103b30 0x7f6a8c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a92183700 1 -- 172.21.15.39:0/739505596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c107e50 msgr2=0x7f6a8c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a92183700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c107e50 0x7f6a8c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a92183700 1 -- 172.21.15.39:0/739505596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c1a9060 con 0x7f6a8c1087a0 2024-05-04T03:10:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a92183700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c1087a0 0x7f6a8c19a460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6a84007d40 tx=0x7f6a8400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a82ffd700 1 -- 172.21.15.39:0/739505596 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84016020 con 0x7f6a8c1087a0 2024-05-04T03:10:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a82ffd700 1 -- 172.21.15.39:0/739505596 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a8400acb0 con 0x7f6a8c1087a0 2024-05-04T03:10:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a93be6700 1 -- 172.21.15.39:0/739505596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8c1a9350 con 0x7f6a8c1087a0 2024-05-04T03:10:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a82ffd700 1 -- 172.21.15.39:0/739505596 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a840115e0 con 0x7f6a8c1087a0 2024-05-04T03:10:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.975+0000 7f6a93be6700 1 -- 172.21.15.39:0/739505596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8c1a9890 con 0x7f6a8c1087a0 2024-05-04T03:10:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.978+0000 7f6a82ffd700 1 -- 172.21.15.39:0/739505596 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6a84019020 con 0x7f6a8c1087a0 2024-05-04T03:10:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6a82ffd700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a780b2900 0x7f6a780b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6aa25fab80 1 Processor -- start 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6a91982700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a780b2900 0x7f6a780b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6a82ffd700 1 -- 172.21.15.39:0/739505596 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a840e2020 con 0x7f6a8c1087a0 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6aa25fab80 1 -- start start 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6a91982700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a780b2900 0x7f6a780b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b52524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.979+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524cb10 0x5627b5258db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5627b516e8f0 con 0x5627b524c1c0 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5627b50d3d50 con 0x5627b52520d0 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627b516a8b0 con 0x5627b524cb10 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54022/0 (socket says 172.21.15.39:54022) 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 -- 172.21.15.39:0/3034742149 learned_addr learned my addr 172.21.15.39:0/3034742149 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91982700 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b52524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a92183700 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524cb10 0x5627b5258db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 -- 172.21.15.39:0/3034742149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524cb10 msgr2=0x5627b5258db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524cb10 0x5627b5258db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 -- 172.21.15.39:0/3034742149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 msgr2=0x5627b52524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b52524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 -- 172.21.15.39:0/3034742149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b50bb880 con 0x5627b524c1c0 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a92183700 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524cb10 0x5627b5258db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a91181700 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524c5d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00ace0 tx=0x7f6a7c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e6be6ffe29b6d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a77fff700 1 -- 172.21.15.39:0/3034742149 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00fe50 con 0x5627b524c1c0 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a77fff700 1 -- 172.21.15.39:0/3034742149 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a7c0044b0 con 0x5627b524c1c0 2024-05-04T03:10:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.980+0000 7f6a77fff700 1 -- 172.21.15.39:0/3034742149 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c018440 con 0x5627b524c1c0 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.981+0000 7f6aa25fab80 1 -- 172.21.15.39:0/3034742149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 msgr2=0x5627b524c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.981+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524c5d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00ace0 tx=0x7f6a7c00f3c0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.981+0000 7f6aa25fab80 1 -- 172.21.15.39:0/3034742149 shutdown_connections 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.981+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524cb10 0x5627b5258db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.981+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b52524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.981+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/3034742149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524c5d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.981+0000 7f6aa25fab80 1 -- 172.21.15.39:0/3034742149 >> 172.21.15.39:0/3034742149 conn(0x5627b50cedf0 msgr2=0x5627b5247210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.981+0000 7f6aa25fab80 1 -- 172.21.15.39:0/3034742149 shutdown_connections 2024-05-04T03:10:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 -- 172.21.15.39:0/3034742149 wait complete. 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 -- start start 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 Processor -- start 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524c1c0 0x5627b52562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b52567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 0x5627b5256d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5627b52594a0 con 0x5627b524cb10 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5627b5127e80 con 0x5627b524c1c0 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.982+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627b5259620 con 0x5627b52520d0 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6aa25fab80 10 client.0 _ll_register_callbacks cb 0x5627b5106900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b52567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b52567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54032/0 (socket says 172.21.15.39:54032) 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 -- 172.21.15.39:0/4262229505 learned_addr learned my addr 172.21.15.39:0/4262229505 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91982700 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524c1c0 0x5627b52562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a92183700 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 0x5627b5256d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 -- 172.21.15.39:0/4262229505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 msgr2=0x5627b5256d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 0x5627b5256d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 -- 172.21.15.39:0/4262229505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524c1c0 msgr2=0x5627b52562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524c1c0 0x5627b52562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 -- 172.21.15.39:0/4262229505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b5125710 con 0x5627b524cb10 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a92183700 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 0x5627b5256d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a91181700 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b52567e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000c00 tx=0x7f6a7c0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804f726b3e862316 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a757fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a757fa700 1 -- 172.21.15.39:0/4262229505 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c026020 con 0x5627b524cb10 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6a757fa700 1 -- 172.21.15.39:0/4262229505 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a7c004d20 con 0x5627b524cb10 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.983+0000 7f6aa25fab80 1 -- 172.21.15.39:0/4262229505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a7c0044b0 con 0x5627b524cb10 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6a757fa700 1 -- 172.21.15.39:0/4262229505 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a7c022460 con 0x5627b524cb10 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6aa25fab80 10 client.14994 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6aa25fab80 1 -- 172.21.15.39:0/4262229505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627b52640f0 con 0x5627b524cb10 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6a757fa700 1 -- 172.21.15.39:0/4262229505 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a7c0044b0 con 0x5627b524cb10 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6a757fa700 1 -- 172.21.15.39:0/4262229505 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a7c018590 con 0x5627b524cb10 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6aa25fab80 10 client.14994 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6aa25fab80 10 client.14994 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6aa25fab80 20 client.14994 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5627b5266350 0x5627b52677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6aa25fab80 4 client.14994 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6aa25fab80 1 -- 172.21.15.39:0/4262229505 --> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627b517c2d0 con 0x5627b5266350 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.984+0000 7f6a91982700 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5627b5266350 0x5627b52677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.985+0000 7f6a757fa700 10 client.14994 ms_handle_connect on v2:172.21.15.66:6834/3463909305 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.985+0000 7f6a91982700 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5627b5266350 0x5627b52677c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4689d1ceea6abc0 server_cookie=fdfb1acde9510034 in_seq=0 out_seq=0 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6a757fa700 1 -- 172.21.15.39:0/4262229505 <== mds.4 v2:172.21.15.66:6834/3463909305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5627b517c2d0 con 0x5627b5266350 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6a757fa700 10 client.14994 handle_command_reply: tid=0 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 client.14994 shutdown 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 -- 172.21.15.39:0/4262229505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 msgr2=0x5627b52567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b52567e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000c00 tx=0x7f6a7c0054e0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 -- 172.21.15.39:0/4262229505 shutdown_connections 2024-05-04T03:10:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5627b5266350 0x5627b52677c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 0x5627b5256d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524c1c0 0x5627b52562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6a757fa700 0 client.14994 ms_handle_reset on v2:172.21.15.66:6834/3463909305 2024-05-04T03:10:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/4262229505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b52567e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 -- 172.21.15.39:0/4262229505 >> 172.21.15.39:0/4262229505 conn(0x5627b50cedf0 msgr2=0x5627b524b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 -- 172.21.15.39:0/4262229505 shutdown_connections 2024-05-04T03:10:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 1 -- 172.21.15.39:0/4262229505 wait complete. 2024-05-04T03:10:01.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:01.986+0000 7f6aa25fab80 20 client.14994 trim_cache size 0 max 16384 2024-05-04T03:10:02.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.006+0000 7f6aa25fab80 1 Processor -- start 2024-05-04T03:10:02.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.006+0000 7f6aa25fab80 1 -- start start 2024-05-04T03:10:02.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:02.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524cb10 0x5627b5255490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:02.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 0x5627b52509f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:02.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5627b5265b00 con 0x5627b524c1c0 2024-05-04T03:10:02.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5627b52661d0 con 0x5627b524cb10 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627b52668a0 con 0x5627b52520d0 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54042/0 (socket says 172.21.15.39:54042) 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 -- 172.21.15.39:0/2141440175 learned_addr learned my addr 172.21.15.39:0/2141440175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91181700 1 --2- 172.21.15.39:0/2141440175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524cb10 0x5627b5255490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 -- 172.21.15.39:0/2141440175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 msgr2=0x5627b52509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 --2- 172.21.15.39:0/2141440175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 0x5627b52509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 -- 172.21.15.39:0/2141440175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524cb10 msgr2=0x5627b5255490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 --2- 172.21.15.39:0/2141440175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524cb10 0x5627b5255490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 -- 172.21.15.39:0/2141440175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b5259060 con 0x5627b524c1c0 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a91982700 1 --2- 172.21.15.39:0/2141440175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524b820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a880058e0 tx=0x7f6a8801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd7f8de5acb3ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.007+0000 7f6a777fe700 1 -- 172.21.15.39:0/2141440175 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8803b030 con 0x5627b524c1c0 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6a777fe700 1 -- 172.21.15.39:0/2141440175 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a880361e0 con 0x5627b524c1c0 2024-05-04T03:10:02.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6a777fe700 1 -- 172.21.15.39:0/2141440175 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800f6a0 con 0x5627b524c1c0 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 -- 172.21.15.39:0/2141440175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 msgr2=0x5627b524b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/2141440175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524b820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a880058e0 tx=0x7f6a8801e000 comp rx=0 tx=0).stop 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 -- 172.21.15.39:0/2141440175 shutdown_connections 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/2141440175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524c1c0 0x5627b524b820 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/2141440175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b52520d0 0x5627b52509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/2141440175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b524cb10 0x5627b5255490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 -- 172.21.15.39:0/2141440175 >> 172.21.15.39:0/2141440175 conn(0x5627b50cedf0 msgr2=0x5627b5257950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 -- 172.21.15.39:0/2141440175 shutdown_connections 2024-05-04T03:10:02.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.008+0000 7f6aa25fab80 1 -- 172.21.15.39:0/2141440175 wait complete. 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 1 -- start start 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 1 Processor -- start 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524c1c0 0x5627b526aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b526b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b526f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5627b5267640 con 0x5627b524cb10 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5627b5266f70 con 0x5627b52520d0 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627b5267d10 con 0x5627b524c1c0 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6aa25fab80 10 client.0 _ll_register_callbacks cb 0x5627b5106900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6a91181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b526b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6a91181700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b526b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54058/0 (socket says 172.21.15.39:54058) 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.009+0000 7f6a91181700 1 -- 172.21.15.39:0/221246829 learned_addr learned my addr 172.21.15.39:0/221246829 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a92183700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b526f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a91982700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524c1c0 0x5627b526aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a91181700 1 -- 172.21.15.39:0/221246829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524c1c0 msgr2=0x5627b526aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a91181700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524c1c0 0x5627b526aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a91181700 1 -- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 msgr2=0x5627b526f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a91181700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b526f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a91181700 1 -- 172.21.15.39:0/221246829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b526f9d0 con 0x5627b524cb10 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a92183700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b526f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a91982700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524c1c0 0x5627b526aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a91181700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b526b410 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00f710 tx=0x7f6a7c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccce525cc80325f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a757fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a757fa700 1 -- 172.21.15.39:0/221246829 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c03a020 con 0x5627b524cb10 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a757fa700 1 -- 172.21.15.39:0/221246829 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a7c018640 con 0x5627b524cb10 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6aa25fab80 1 -- 172.21.15.39:0/221246829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a880361e0 con 0x5627b524cb10 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6a757fa700 1 -- 172.21.15.39:0/221246829 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a7c026040 con 0x5627b524cb10 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6aa25fab80 10 client.15000 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.010+0000 7f6aa25fab80 1 -- 172.21.15.39:0/221246829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627b52d1ec0 con 0x5627b524cb10 2024-05-04T03:10:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6a757fa700 1 -- 172.21.15.39:0/221246829 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a880361e0 con 0x5627b524cb10 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6a757fa700 1 -- 172.21.15.39:0/221246829 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a7c0104d0 con 0x5627b524cb10 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6aa25fab80 10 client.15000 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6aa25fab80 10 client.15000 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6aa25fab80 20 client.15000 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5627b52d42c0 0x5627b52d5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6aa25fab80 4 client.15000 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6aa25fab80 1 -- 172.21.15.39:0/221246829 --> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] -- command(tid 0: {"prefix": "status"}) -- 0x5627b5176370 con 0x5627b52d42c0 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6a91982700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5627b52d42c0 0x5627b52d5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6a91982700 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5627b52d42c0 0x5627b52d5730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fcb078a758c5f63 server_cookie=3cf91f5e1b112ea9 in_seq=0 out_seq=0 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.011+0000 7f6a757fa700 10 client.15000 ms_handle_connect on v2:172.21.15.66:6834/3463909305 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6a757fa700 1 -- 172.21.15.39:0/221246829 <== mds.4 v2:172.21.15.66:6834/3463909305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5627b5176370 con 0x5627b52d42c0 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6a757fa700 10 client.15000 handle_command_reply: tid=0 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 client.15000 shutdown 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 -- 172.21.15.39:0/221246829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 msgr2=0x5627b526b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:02.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b526b410 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00f710 tx=0x7f6a7c00a5c0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 -- 172.21.15.39:0/221246829 shutdown_connections 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5627b52d42c0 0x5627b52d5730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627b52520d0 0x5627b526f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6a757fa700 0 client.15000 ms_handle_reset on v2:172.21.15.66:6834/3463909305 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627b524c1c0 0x5627b526aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 --2- 172.21.15.39:0/221246829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627b524cb10 0x5627b526b410 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 -- 172.21.15.39:0/221246829 >> 172.21.15.39:0/221246829 conn(0x5627b50cedf0 msgr2=0x5627b5262310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 -- 172.21.15.39:0/221246829 shutdown_connections 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 1 -- 172.21.15.39:0/221246829 wait complete. 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.012+0000 7f6aa25fab80 20 client.15000 trim_cache size 0 max 16384 2024-05-04T03:10:02.018 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24455, 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 124.847964905, 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 183.54191372299999 2024-05-04T03:10:02.019 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 -- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a780b2900 msgr2=0x7f6a780b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a780b2900 0x7f6a780b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88005fb0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 -- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c1087a0 msgr2=0x7f6a8c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c1087a0 0x7f6a8c19a460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6a84007d40 tx=0x7f6a8400a340 comp rx=0 tx=0).stop 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 -- 172.21.15.39:0/739505596 shutdown_connections 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a780b2900 0x7f6a780b4e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a8c1087a0 0x7f6a8c19a460 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a8c103b30 0x7f6a8c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 --2- 172.21.15.39:0/739505596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a8c107e50 0x7f6a8c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 -- 172.21.15.39:0/739505596 >> 172.21.15.39:0/739505596 conn(0x7f6a8c0fe520 msgr2=0x7f6a8c106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:02.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 -- 172.21.15.39:0/739505596 shutdown_connections 2024-05-04T03:10:02.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:02.013+0000 7f6a93be6700 1 -- 172.21.15.39:0/739505596 wait complete. 2024-05-04T03:10:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:02 smithi167 ceph-mon[33614]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:02.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:02 smithi039 ceph-mon[29715]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:02.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-04T03:10:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:02 smithi066 ceph-mon[33264]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:03.027 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:04.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:04 smithi039 ceph-mon[29715]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:04.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:04 smithi167 ceph-mon[33614]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.772+0000 7fe65289f700 1 -- 172.21.15.39:0/2528667160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 msgr2=0x7fe64c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.772+0000 7fe65289f700 1 --2- 172.21.15.39:0/2528667160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 0x7fe64c1038f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0099c0 tx=0x7fe63c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 -- 172.21.15.39:0/2528667160 shutdown_connections 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 --2- 172.21.15.39:0/2528667160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 0x7fe64c1038f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 --2- 172.21.15.39:0/2528667160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe64c108760 0x7fe64c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 --2- 172.21.15.39:0/2528667160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64c107e10 0x7fe64c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 -- 172.21.15.39:0/2528667160 >> 172.21.15.39:0/2528667160 conn(0x7fe64c0fef30 msgr2=0x7fe64c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 -- 172.21.15.39:0/2528667160 shutdown_connections 2024-05-04T03:10:04.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 -- 172.21.15.39:0/2528667160 wait complete. 2024-05-04T03:10:04.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 Processor -- start 2024-05-04T03:10:04.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.773+0000 7fe65289f700 1 -- start start 2024-05-04T03:10:04.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe65289f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 0x7fe64c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe65289f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe64c107e10 0x7fe64c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe65289f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64c108760 0x7fe64c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe65289f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c113240 con 0x7fe64c1034e0 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe65289f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1130c0 con 0x7fe64c107e10 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe65289f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c1133c0 con 0x7fe64c108760 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 0x7fe64c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 0x7fe64c19adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54066/0 (socket says 172.21.15.39:54066) 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 -- 172.21.15.39:0/1336094971 learned_addr learned my addr 172.21.15.39:0/1336094971 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe64c107e10 0x7fe64c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe650e3c700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64c108760 0x7fe64c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 -- 172.21.15.39:0/1336094971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64c108760 msgr2=0x7fe64c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64c108760 0x7fe64c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 -- 172.21.15.39:0/1336094971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe64c107e10 msgr2=0x7fe64c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe64c107e10 0x7fe64c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 -- 172.21.15.39:0/1336094971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c19fc90 con 0x7fe64c1034e0 2024-05-04T03:10:04.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe64c107e10 0x7fe64c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:04.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.774+0000 7fe64bfff700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 0x7fe64c19adc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe63c000c00 tx=0x7fe63c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:04.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.775+0000 7fe6497fa700 1 -- 172.21.15.39:0/1336094971 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c00f940 con 0x7fe64c1034e0 2024-05-04T03:10:04.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.775+0000 7fe6497fa700 1 -- 172.21.15.39:0/1336094971 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe63c004830 con 0x7fe64c1034e0 2024-05-04T03:10:04.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.775+0000 7fe65289f700 1 -- 172.21.15.39:0/1336094971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c19ff20 con 0x7fe64c1034e0 2024-05-04T03:10:04.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.775+0000 7fe6497fa700 1 -- 172.21.15.39:0/1336094971 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0054b0 con 0x7fe64c1034e0 2024-05-04T03:10:04.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.775+0000 7fe65289f700 1 -- 172.21.15.39:0/1336094971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c1a03a0 con 0x7fe64c1034e0 2024-05-04T03:10:04.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.777+0000 7fe6497fa700 1 -- 172.21.15.39:0/1336094971 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe63c029060 con 0x7fe64c1034e0 2024-05-04T03:10:04.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.778+0000 7fe6497fa700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.778+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.778+0000 7fe6497fa700 1 -- 172.21.15.39:0/1336094971 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe63c0e2db0 con 0x7fe64c1034e0 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.778+0000 7fe6612b3b80 1 Processor -- start 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe6612b3b80 1 -- start start 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6340b2880 0x7fe6340b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe640005950 tx=0x7fe6400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04ddef0 0x55d0b04e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b04e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d0b03fb810 con 0x55d0b04dd5a0 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d0b039a770 con 0x55d0b04ddef0 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d0b03f77d0 con 0x55d0b04e22b0 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04dd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54080/0 (socket says 172.21.15.39:54080) 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64bfff700 1 -- 172.21.15.39:0/421423512 learned_addr learned my addr 172.21.15.39:0/421423512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe650e3c700 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b04e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04ddef0 0x55d0b04e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64bfff700 1 -- 172.21.15.39:0/421423512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 msgr2=0x55d0b04e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64bfff700 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b04e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64bfff700 1 -- 172.21.15.39:0/421423512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04ddef0 msgr2=0x55d0b04e1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64bfff700 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04ddef0 0x55d0b04e1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64bfff700 1 -- 172.21.15.39:0/421423512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b0348880 con 0x55d0b04dd5a0 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.779+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04ddef0 0x55d0b04e1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.780+0000 7fe64bfff700 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04dd9b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0a64f0 tx=0x7fe63c0a9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21c1d533f2529d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:04.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.780+0000 7fe6327fc700 1 -- 172.21.15.39:0/421423512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0aa470 con 0x55d0b04dd5a0 2024-05-04T03:10:04.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.780+0000 7fe6327fc700 1 -- 172.21.15.39:0/421423512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe63c0ae820 con 0x55d0b04dd5a0 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.780+0000 7fe6327fc700 1 -- 172.21.15.39:0/421423512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0aa8f0 con 0x55d0b04dd5a0 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 -- 172.21.15.39:0/421423512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 msgr2=0x55d0b04dd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04dd9b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0a64f0 tx=0x7fe63c0a9f80 comp rx=0 tx=0).stop 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 -- 172.21.15.39:0/421423512 shutdown_connections 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04dd9b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b04e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/421423512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04ddef0 0x55d0b04e1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 -- 172.21.15.39:0/421423512 >> 172.21.15.39:0/421423512 conn(0x55d0b035bdf0 msgr2=0x55d0b04d3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 -- 172.21.15.39:0/421423512 shutdown_connections 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 -- 172.21.15.39:0/421423512 wait complete. 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.781+0000 7fe6612b3b80 1 -- start start 2024-05-04T03:10:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe6612b3b80 1 Processor -- start 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e22b0 0x55d0b04e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 0x55d0b04e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d0b04e2aa0 con 0x55d0b04dd5a0 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d0b0361be0 con 0x55d0b04e22b0 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d0b04ea440 con 0x55d0b04e7610 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe6612b3b80 10 client.0 _ll_register_callbacks cb 0x55d0b03938d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54082/0 (socket says 172.21.15.39:54082) 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64bfff700 1 -- 172.21.15.39:0/2167389169 learned_addr learned my addr 172.21.15.39:0/2167389169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe650e3c700 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 0x55d0b04e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e22b0 0x55d0b04e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64bfff700 1 -- 172.21.15.39:0/2167389169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 msgr2=0x55d0b04e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64bfff700 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 0x55d0b04e7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64bfff700 1 -- 172.21.15.39:0/2167389169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e22b0 msgr2=0x55d0b04e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64bfff700 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e22b0 0x55d0b04e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.782+0000 7fe64bfff700 1 -- 172.21.15.39:0/2167389169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b0397640 con 0x55d0b04dd5a0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e22b0 0x55d0b04e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe64bfff700 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e9d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0921a0 tx=0x7fe63c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a82b145eaa515a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe62b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe62b7fe700 1 -- 172.21.15.39:0/2167389169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0b2db0 con 0x55d0b04dd5a0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe62b7fe700 1 -- 172.21.15.39:0/2167389169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe63c0b2f50 con 0x55d0b04dd5a0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe6612b3b80 1 -- 172.21.15.39:0/2167389169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe63c0ae820 con 0x55d0b04dd5a0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe62b7fe700 1 -- 172.21.15.39:0/2167389169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe63c0a4b90 con 0x55d0b04dd5a0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe6612b3b80 10 client.15012 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe6612b3b80 1 -- 172.21.15.39:0/2167389169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d0b04f4e80 con 0x55d0b04dd5a0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe62b7fe700 1 -- 172.21.15.39:0/2167389169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe63c0ae820 con 0x55d0b04dd5a0 2024-05-04T03:10:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.783+0000 7fe62b7fe700 1 -- 172.21.15.39:0/2167389169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe63c00fae0 con 0x55d0b04dd5a0 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe6612b3b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe6612b3b80 10 client.15012 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe6612b3b80 20 client.15012 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55d0b04f7220 0x55d0b04f8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe6612b3b80 4 client.15012 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe6612b3b80 1 -- 172.21.15.39:0/2167389169 --> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b04091f0 con 0x55d0b04f7220 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55d0b04f7220 0x55d0b04f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe62b7fe700 10 client.15012 ms_handle_connect on v2:172.21.15.167:6834/2699315372 2024-05-04T03:10:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.784+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55d0b04f7220 0x55d0b04f8690 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c32de55867a3ac21 server_cookie=d8e7b58cc4e510bd in_seq=0 out_seq=0 2024-05-04T03:10:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.785+0000 7fe62b7fe700 1 -- 172.21.15.39:0/2167389169 <== mds.3 v2:172.21.15.167:6834/2699315372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0b04091f0 con 0x55d0b04f7220 2024-05-04T03:10:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.785+0000 7fe62b7fe700 10 client.15012 handle_command_reply: tid=0 2024-05-04T03:10:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.785+0000 7fe6612b3b80 1 client.15012 shutdown 2024-05-04T03:10:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 -- 172.21.15.39:0/2167389169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 msgr2=0x55d0b04e9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e9d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0921a0 tx=0x7fe63c009f10 comp rx=0 tx=0).stop 2024-05-04T03:10:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 -- 172.21.15.39:0/2167389169 shutdown_connections 2024-05-04T03:10:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55d0b04f7220 0x55d0b04f8690 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e9d50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 0x55d0b04e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/2167389169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e22b0 0x55d0b04e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 -- 172.21.15.39:0/2167389169 >> 172.21.15.39:0/2167389169 conn(0x55d0b035bdf0 msgr2=0x55d0b04dd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:04.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe62b7fe700 0 client.15012 ms_handle_reset on v2:172.21.15.167:6834/2699315372 2024-05-04T03:10:04.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 -- 172.21.15.39:0/2167389169 shutdown_connections 2024-05-04T03:10:04.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 1 -- 172.21.15.39:0/2167389169 wait complete. 2024-05-04T03:10:04.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.786+0000 7fe6612b3b80 20 client.15012 trim_cache size 0 max 16384 2024-05-04T03:10:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.807+0000 7fe6612b3b80 1 Processor -- start 2024-05-04T03:10:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.807+0000 7fe6612b3b80 1 -- start start 2024-05-04T03:10:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04e22b0 0x55d0b04f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 0x55d0b04e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d0b04f69d0 con 0x55d0b04e22b0 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d0b04f70a0 con 0x55d0b04dd5a0 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d0b04f7770 con 0x55d0b04e7610 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04e22b0 0x55d0b04f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04e22b0 0x55d0b04f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54090/0 (socket says 172.21.15.39:54090) 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 -- 172.21.15.39:0/787880508 learned_addr learned my addr 172.21.15.39:0/787880508 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64bfff700 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe650e3c700 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 0x55d0b04e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 -- 172.21.15.39:0/787880508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 msgr2=0x55d0b04e1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 0x55d0b04e1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 -- 172.21.15.39:0/787880508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04dd5a0 msgr2=0x55d0b04e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 -- 172.21.15.39:0/787880508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b04e63f0 con 0x55d0b04e22b0 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04e22b0 0x55d0b04f3550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55d0b04d5460 tx=0x7fe640015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c816e0900b89f75b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe631ffb700 1 -- 172.21.15.39:0/787880508 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64002b070 con 0x55d0b04e22b0 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe631ffb700 1 -- 172.21.15.39:0/787880508 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe640004540 con 0x55d0b04e22b0 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.808+0000 7fe631ffb700 1 -- 172.21.15.39:0/787880508 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640027980 con 0x55d0b04e22b0 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 -- 172.21.15.39:0/787880508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04e22b0 msgr2=0x55d0b04f3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04e22b0 0x55d0b04f3550 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55d0b04d5460 tx=0x7fe640015000 comp rx=0 tx=0).stop 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 -- 172.21.15.39:0/787880508 shutdown_connections 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e7610 0x55d0b04e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04dd5a0 0x55d0b04e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/787880508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04e22b0 0x55d0b04f3550 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 -- 172.21.15.39:0/787880508 >> 172.21.15.39:0/787880508 conn(0x55d0b035bdf0 msgr2=0x55d0b04e6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 -- 172.21.15.39:0/787880508 shutdown_connections 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.809+0000 7fe6612b3b80 1 -- 172.21.15.39:0/787880508 wait complete. 2024-05-04T03:10:04.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 1 -- start start 2024-05-04T03:10:04.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 1 Processor -- start 2024-05-04T03:10:04.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b0501ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b05020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e7610 0x55d0b04fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d0b04f8510 con 0x55d0b04dd5a0 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d0b04f7e40 con 0x55d0b04e7610 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d0b04f8be0 con 0x55d0b04e22b0 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe64bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b0501ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe6612b3b80 10 client.0 _ll_register_callbacks cb 0x55d0b03938d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe64bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b0501ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54104/0 (socket says 172.21.15.39:54104) 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe64bfff700 1 -- 172.21.15.39:0/428026071 learned_addr learned my addr 172.21.15.39:0/428026071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.810+0000 7fe650e3c700 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e7610 0x55d0b04fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b05020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe64bfff700 1 -- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 msgr2=0x55d0b05020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:04.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe64bfff700 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b05020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe64bfff700 1 -- 172.21.15.39:0/428026071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e7610 msgr2=0x55d0b04fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe64bfff700 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e7610 0x55d0b04fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe64bfff700 1 -- 172.21.15.39:0/428026071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b04fc740 con 0x55d0b04dd5a0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b05020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe64bfff700 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b0501ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe63c092170 tx=0x7fe63c0e0910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a50a34c1ee11f53 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe62b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe62b7fe700 1 -- 172.21.15.39:0/428026071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0ae820 con 0x55d0b04dd5a0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe62b7fe700 1 -- 172.21.15.39:0/428026071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe63c0b2db0 con 0x55d0b04dd5a0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe6612b3b80 1 -- 172.21.15.39:0/428026071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe640004540 con 0x55d0b04dd5a0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe62b7fe700 1 -- 172.21.15.39:0/428026071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe63c0b2fb0 con 0x55d0b04dd5a0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe6612b3b80 10 client.15018 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe6612b3b80 1 -- 172.21.15.39:0/428026071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d0b04fc740 con 0x55d0b04dd5a0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.811+0000 7fe62b7fe700 1 -- 172.21.15.39:0/428026071 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe640004540 con 0x55d0b04dd5a0 2024-05-04T03:10:04.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe62b7fe700 1 -- 172.21.15.39:0/428026071 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe63c0a9740 con 0x55d0b04dd5a0 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe6612b3b80 10 client.15018 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe6612b3b80 10 client.15018 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe6612b3b80 20 client.15018 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55d0b05650f0 0x55d0b0566560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe6612b3b80 4 client.15018 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe6612b3b80 1 -- 172.21.15.39:0/428026071 --> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] -- command(tid 0: {"prefix": "status"}) -- 0x55d0b0403290 con 0x55d0b05650f0 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55d0b05650f0 0x55d0b0566560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe64b7fe700 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55d0b05650f0 0x55d0b0566560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=71476d944bba66de server_cookie=4f6ea4549414db0 in_seq=0 out_seq=0 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.812+0000 7fe62b7fe700 10 client.15018 ms_handle_connect on v2:172.21.15.167:6834/2699315372 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe62b7fe700 1 -- 172.21.15.39:0/428026071 <== mds.3 v2:172.21.15.167:6834/2699315372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55d0b0403290 con 0x55d0b05650f0 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe62b7fe700 10 client.15018 handle_command_reply: tid=0 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 client.15018 shutdown 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 -- 172.21.15.39:0/428026071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 msgr2=0x55d0b0501ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b0501ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe63c092170 tx=0x7fe63c0e0910 comp rx=0 tx=0).stop 2024-05-04T03:10:04.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 -- 172.21.15.39:0/428026071 shutdown_connections 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55d0b05650f0 0x55d0b0566560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0b04dd5a0 0x55d0b0501ba0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d0b04e7610 0x55d0b04fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 --2- 172.21.15.39:0/428026071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d0b04e22b0 0x55d0b05020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe62b7fe700 0 client.15018 ms_handle_reset on v2:172.21.15.167:6834/2699315372 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 -- 172.21.15.39:0/428026071 >> 172.21.15.39:0/428026071 conn(0x55d0b035bdf0 msgr2=0x55d0b04cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 -- 172.21.15.39:0/428026071 shutdown_connections 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 1 -- 172.21.15.39:0/428026071 wait complete. 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.813+0000 7fe6612b3b80 20 client.15018 trim_cache size 0 max 16384 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24466, 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 128.66536266, 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 183.55513171499999 2024-05-04T03:10:04.821 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 -- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6340b2880 msgr2=0x7fe6340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6340b2880 0x7fe6340b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe640005950 tx=0x7fe6400058e0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 -- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 msgr2=0x7fe64c19adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 0x7fe64c19adc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe63c000c00 tx=0x7fe63c00f510 comp rx=0 tx=0).stop 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 -- 172.21.15.39:0/1336094971 shutdown_connections 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6340b2880 0x7fe6340b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe64c1034e0 0x7fe64c19adc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe64c108760 0x7fe64c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 --2- 172.21.15.39:0/1336094971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe64c107e10 0x7fe64c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 -- 172.21.15.39:0/1336094971 >> 172.21.15.39:0/1336094971 conn(0x7fe64c0fef30 msgr2=0x7fe64c1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 -- 172.21.15.39:0/1336094971 shutdown_connections 2024-05-04T03:10:04.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:04.814+0000 7fe65289f700 1 -- 172.21.15.39:0/1336094971 wait complete. 2024-05-04T03:10:04.823 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:04 smithi066 ceph-mon[33264]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:05.588 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-04T03:10:05.838 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:06.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:06 smithi167 ceph-mon[33614]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:06 smithi039 ceph-mon[29715]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:06.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:06 smithi066 ceph-mon[33264]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:07.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.602+0000 7fa07f9c2700 1 -- 172.21.15.39:0/3602760883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa078108f00 msgr2=0x7fa078109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.602+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/3602760883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa078108f00 0x7fa078109310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0099c0 tx=0x7fa06c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.602+0000 7fa07f9c2700 1 -- 172.21.15.39:0/3602760883 shutdown_connections 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.602+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/3602760883 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa078109850 0x7fa0781111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.602+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/3602760883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0781045d0 0x7fa0781049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.602+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/3602760883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa078108f00 0x7fa078109310 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.602+0000 7fa07f9c2700 1 -- 172.21.15.39:0/3602760883 >> 172.21.15.39:0/3602760883 conn(0x7fa0780ffea0 msgr2=0x7fa0781022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.603+0000 7fa07f9c2700 1 -- 172.21.15.39:0/3602760883 shutdown_connections 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.603+0000 7fa07f9c2700 1 -- 172.21.15.39:0/3602760883 wait complete. 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07f9c2700 1 Processor -- start 2024-05-04T03:10:07.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07f9c2700 1 -- start start 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07f9c2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0781045d0 0x7fa07819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07f9c2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078108f00 0x7fa07819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07f9c2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa078109850 0x7fa07819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07f9c2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa078114310 con 0x7fa0781045d0 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07f9c2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa078114190 con 0x7fa078109850 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07f9c2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa078114490 con 0x7fa078108f00 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07d75e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0781045d0 0x7fa07819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07d75e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0781045d0 0x7fa07819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40134/0 (socket says 172.21.15.39:40134) 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.604+0000 7fa07d75e700 1 -- 172.21.15.39:0/1553948261 learned_addr learned my addr 172.21.15.39:0/1553948261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa07d75e700 1 -- 172.21.15.39:0/1553948261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078108f00 msgr2=0x7fa07819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa07d75e700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078108f00 0x7fa07819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa07d75e700 1 -- 172.21.15.39:0/1553948261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa078109850 msgr2=0x7fa07819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa07d75e700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa078109850 0x7fa07819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa07d75e700 1 -- 172.21.15.39:0/1553948261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07819fad0 con 0x7fa0781045d0 2024-05-04T03:10:07.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa07d75e700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0781045d0 0x7fa07819ad60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa06800e9a0 tx=0x7fa06800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:07.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa0767fc700 1 -- 172.21.15.39:0/1553948261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06800cbd0 con 0x7fa0781045d0 2024-05-04T03:10:07.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa07f9c2700 1 -- 172.21.15.39:0/1553948261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07819fdc0 con 0x7fa0781045d0 2024-05-04T03:10:07.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa0767fc700 1 -- 172.21.15.39:0/1553948261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa06800cd70 con 0x7fa0781045d0 2024-05-04T03:10:07.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa0767fc700 1 -- 172.21.15.39:0/1553948261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068018680 con 0x7fa0781045d0 2024-05-04T03:10:07.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.605+0000 7fa07f9c2700 1 -- 172.21.15.39:0/1553948261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0781a0350 con 0x7fa0781045d0 2024-05-04T03:10:07.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.608+0000 7fa0767fc700 1 -- 172.21.15.39:0/1553948261 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa068018820 con 0x7fa0781045d0 2024-05-04T03:10:07.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.609+0000 7fa0767fc700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0600b2880 0x7fa0600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.609+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0600b2880 0x7fa0600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.609+0000 7fa0767fc700 1 -- 172.21.15.39:0/1553948261 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa0680e2530 con 0x7fa0781045d0 2024-05-04T03:10:07.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.609+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0600b2880 0x7fa0600b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0098e0 tx=0x7fa06c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa08e3d6b80 1 Processor -- start 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa08e3d6b80 1 -- start start 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf46f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf4691c0 0x55efdf4695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf475db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55efdf38b8f0 con 0x55efdf4691c0 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55efdf2f0d50 con 0x55efdf469b10 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55efdf3878b0 con 0x55efdf46f0d0 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07cf5d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf4691c0 0x55efdf4695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07cf5d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf4691c0 0x55efdf4695d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40140/0 (socket says 172.21.15.39:40140) 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07cf5d700 1 -- 172.21.15.39:0/3584359157 learned_addr learned my addr 172.21.15.39:0/3584359157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07d75e700 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf46f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07df5f700 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf475db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07cf5d700 1 -- 172.21.15.39:0/3584359157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 msgr2=0x55efdf46f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf46f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07cf5d700 1 -- 172.21.15.39:0/3584359157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 msgr2=0x55efdf475db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf475db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.610+0000 7fa07cf5d700 1 -- 172.21.15.39:0/3584359157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efdf2d8880 con 0x55efdf4691c0 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.611+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf4691c0 0x55efdf4695d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa06c015000 tx=0x7fa06c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631aecda6cfdb2ea server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:07.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.611+0000 7fa05f7fe700 1 -- 172.21.15.39:0/3584359157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0038f0 con 0x55efdf4691c0 2024-05-04T03:10:07.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.611+0000 7fa05f7fe700 1 -- 172.21.15.39:0/3584359157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa06c00b790 con 0x55efdf4691c0 2024-05-04T03:10:07.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.611+0000 7fa05f7fe700 1 -- 172.21.15.39:0/3584359157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c02d580 con 0x55efdf4691c0 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/3584359157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf4691c0 msgr2=0x55efdf4695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf4691c0 0x55efdf4695d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa06c015000 tx=0x7fa06c01c000 comp rx=0 tx=0).stop 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/3584359157 shutdown_connections 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf475db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf46f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/3584359157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf4691c0 0x55efdf4695d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/3584359157 >> 172.21.15.39:0/3584359157 conn(0x55efdf2ebdf0 msgr2=0x55efdf464210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/3584359157 shutdown_connections 2024-05-04T03:10:07.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/3584359157 wait complete. 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 -- start start 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.612+0000 7fa08e3d6b80 1 Processor -- start 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 0x55efdf473310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf473850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf46f0d0 0x55efdf473d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55efdf476440 con 0x55efdf469b10 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55efdf344e80 con 0x55efdf46f0d0 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55efdf4765c0 con 0x55efdf4691c0 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa08e3d6b80 10 client.0 _ll_register_callbacks cb 0x55efdf323900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07cf5d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf473850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07cf5d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf473850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40150/0 (socket says 172.21.15.39:40150) 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07cf5d700 1 -- 172.21.15.39:0/2906882961 learned_addr learned my addr 172.21.15.39:0/2906882961 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07d75e700 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 0x55efdf473310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07df5f700 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf46f0d0 0x55efdf473d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07cf5d700 1 -- 172.21.15.39:0/2906882961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 msgr2=0x55efdf473310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 0x55efdf473310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07cf5d700 1 -- 172.21.15.39:0/2906882961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf46f0d0 msgr2=0x55efdf473d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf46f0d0 0x55efdf473d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.613+0000 7fa07cf5d700 1 -- 172.21.15.39:0/2906882961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efdf342710 con 0x55efdf469b10 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf473850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa06c005a30 tx=0x7fa06c02d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be90b04e200df2db server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa05cff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa05cff9700 1 -- 172.21.15.39:0/2906882961 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00bbd0 con 0x55efdf469b10 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa05cff9700 1 -- 172.21.15.39:0/2906882961 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa06c00bd70 con 0x55efdf469b10 2024-05-04T03:10:07.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2906882961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa06c00b790 con 0x55efdf469b10 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa05cff9700 1 -- 172.21.15.39:0/2906882961 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa06c02d7c0 con 0x55efdf469b10 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa08e3d6b80 10 client.15030 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2906882961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55efdf4810a0 con 0x55efdf469b10 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa05cff9700 1 -- 172.21.15.39:0/2906882961 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa06c00b790 con 0x55efdf469b10 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.614+0000 7fa05cff9700 1 -- 172.21.15.39:0/2906882961 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa06c00f910 con 0x55efdf469b10 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa08e3d6b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa08e3d6b80 10 client.15030 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa08e3d6b80 20 client.15030 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x55efdf4823f0 0x55efdf4845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa08e3d6b80 4 client.15030 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2906882961 --> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efdf3992d0 con 0x55efdf4823f0 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa07d75e700 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x55efdf4823f0 0x55efdf4845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa07d75e700 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x55efdf4823f0 0x55efdf4845d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ddaa4de43934d28 server_cookie=bda583dd8455463f in_seq=0 out_seq=0 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.615+0000 7fa05cff9700 10 client.15030 ms_handle_connect on v2:172.21.15.167:6836/2270685849 2024-05-04T03:10:07.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.616+0000 7fa05cff9700 1 -- 172.21.15.39:0/2906882961 <== mds.2 v2:172.21.15.167:6836/2270685849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55efdf3992d0 con 0x55efdf4823f0 2024-05-04T03:10:07.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.616+0000 7fa05cff9700 10 client.15030 handle_command_reply: tid=0 2024-05-04T03:10:07.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.616+0000 7fa08e3d6b80 1 client.15030 shutdown 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2906882961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 msgr2=0x55efdf473850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf473850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa06c005a30 tx=0x7fa06c02d460 comp rx=0 tx=0).stop 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2906882961 shutdown_connections 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x55efdf4823f0 0x55efdf4845d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf46f0d0 0x55efdf473d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 0x55efdf473310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa05cff9700 0 client.15030 ms_handle_reset on v2:172.21.15.167:6836/2270685849 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2906882961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf473850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2906882961 >> 172.21.15.39:0/2906882961 conn(0x55efdf2ebdf0 msgr2=0x55efdf4681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2906882961 shutdown_connections 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2906882961 wait complete. 2024-05-04T03:10:07.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.617+0000 7fa08e3d6b80 20 client.15030 trim_cache size 0 max 16384 2024-05-04T03:10:07.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.638+0000 7fa08e3d6b80 1 Processor -- start 2024-05-04T03:10:07.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.638+0000 7fa08e3d6b80 1 -- start start 2024-05-04T03:10:07.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.638+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf4691c0 0x55efdf4753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.638+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf45e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.638+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf473880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.638+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55efdf482910 con 0x55efdf469b10 2024-05-04T03:10:07.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.638+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55efdf482fe0 con 0x55efdf4691c0 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.638+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55efdf4836b0 con 0x55efdf46f0d0 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf45e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf45e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40162/0 (socket says 172.21.15.39:40162) 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 -- 172.21.15.39:0/515336070 learned_addr learned my addr 172.21.15.39:0/515336070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 -- 172.21.15.39:0/515336070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 msgr2=0x55efdf473880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07df5f700 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf473880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07d75e700 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf4691c0 0x55efdf4753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf473880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 -- 172.21.15.39:0/515336070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf4691c0 msgr2=0x55efdf4753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf4691c0 0x55efdf4753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 -- 172.21.15.39:0/515336070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efdf476080 con 0x55efdf469b10 2024-05-04T03:10:07.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07df5f700 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf473880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07d75e700 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf4691c0 0x55efdf4753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf45e5f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa06c005270 tx=0x7fa06c0059c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d8dc85c788bc49 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa05effd700 1 -- 172.21.15.39:0/515336070 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00b810 con 0x55efdf469b10 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa05effd700 1 -- 172.21.15.39:0/515336070 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa06c02ddf0 con 0x55efdf469b10 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.639+0000 7fa05effd700 1 -- 172.21.15.39:0/515336070 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c035bc0 con 0x55efdf469b10 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/515336070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 msgr2=0x55efdf45e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf45e5f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa06c005270 tx=0x7fa06c0059c0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/515336070 shutdown_connections 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf46f0d0 0x55efdf473880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf4691c0 0x55efdf4753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/515336070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf469b10 0x55efdf45e5f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/515336070 >> 172.21.15.39:0/515336070 conn(0x55efdf2ebdf0 msgr2=0x55efdf468820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/515336070 shutdown_connections 2024-05-04T03:10:07.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.640+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/515336070 wait complete. 2024-05-04T03:10:07.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 1 -- start start 2024-05-04T03:10:07.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 1 Processor -- start 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 0x55efdf48dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf487fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf46f0d0 0x55efdf4884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55efdf484450 con 0x55efdf46f0d0 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55efdf483d80 con 0x55efdf469b10 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55efdf484b20 con 0x55efdf4691c0 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa08e3d6b80 10 client.0 _ll_register_callbacks cb 0x55efdf323900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa07df5f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf46f0d0 0x55efdf4884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa07df5f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf46f0d0 0x55efdf4884e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40172/0 (socket says 172.21.15.39:40172) 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa07cf5d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf487fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa07d75e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 0x55efdf48dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.641+0000 7fa07df5f700 1 -- 172.21.15.39:0/2981143274 learned_addr learned my addr 172.21.15.39:0/2981143274 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa07df5f700 1 -- 172.21.15.39:0/2981143274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 msgr2=0x55efdf48dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa07df5f700 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 0x55efdf48dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa07df5f700 1 -- 172.21.15.39:0/2981143274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 msgr2=0x55efdf487fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa07df5f700 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf487fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa07df5f700 1 -- 172.21.15.39:0/2981143274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efdf488a20 con 0x55efdf46f0d0 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa07cf5d700 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf487fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:10:07.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa07df5f700 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf46f0d0 0x55efdf4884e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa070007a80 tx=0x7fa070009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f355687b0be76c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa05cff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa05cff9700 1 -- 172.21.15.39:0/2981143274 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07001b070 con 0x55efdf46f0d0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2981143274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa06c02ddf0 con 0x55efdf46f0d0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa05cff9700 1 -- 172.21.15.39:0/2981143274 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa0700138d0 con 0x55efdf46f0d0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa05cff9700 1 -- 172.21.15.39:0/2981143274 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa070027050 con 0x55efdf46f0d0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa08e3d6b80 10 client.15036 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2981143274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55efdf4ef0d0 con 0x55efdf46f0d0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.642+0000 7fa05cff9700 1 -- 172.21.15.39:0/2981143274 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa06c02ddf0 con 0x55efdf46f0d0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa05cff9700 1 -- 172.21.15.39:0/2981143274 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa070017b20 con 0x55efdf46f0d0 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa08e3d6b80 10 client.15036 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa08e3d6b80 10 client.15036 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa08e3d6b80 20 client.15036 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x55efdf4f1590 0x55efdf4f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa08e3d6b80 4 client.15036 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2981143274 --> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] -- command(tid 0: {"prefix": "status"}) -- 0x55efdf393370 con 0x55efdf4f1590 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa07d75e700 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x55efdf4f1590 0x55efdf4f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa05cff9700 10 client.15036 ms_handle_connect on v2:172.21.15.167:6836/2270685849 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.643+0000 7fa07d75e700 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x55efdf4f1590 0x55efdf4f2a00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=321da553a82c3b57 server_cookie=28864251eabfce92 in_seq=0 out_seq=0 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa05cff9700 1 -- 172.21.15.39:0/2981143274 <== mds.2 v2:172.21.15.167:6836/2270685849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55efdf393370 con 0x55efdf4f1590 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa05cff9700 10 client.15036 handle_command_reply: tid=0 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 client.15036 shutdown 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2981143274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf46f0d0 msgr2=0x55efdf4884e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf46f0d0 0x55efdf4884e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa070007a80 tx=0x7fa070009f90 comp rx=0 tx=0).stop 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2981143274 shutdown_connections 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x55efdf4f1590 0x55efdf4f2a00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55efdf46f0d0 0x55efdf4884e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55efdf4691c0 0x55efdf48dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 --2- 172.21.15.39:0/2981143274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efdf469b10 0x55efdf487fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2981143274 >> 172.21.15.39:0/2981143274 conn(0x55efdf2ebdf0 msgr2=0x55efdf475180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa05cff9700 0 client.15036 ms_handle_reset on v2:172.21.15.167:6836/2270685849 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2981143274 shutdown_connections 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 1 -- 172.21.15.39:0/2981143274 wait complete. 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.644+0000 7fa08e3d6b80 20 client.15036 trim_cache size 0 max 16384 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24472, 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:07.653 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:07.654 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:07.654 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 132.51251077399999, 2024-05-04T03:10:07.654 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:07.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:07.654 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:07.654 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 183.61426727899999 2024-05-04T03:10:07.654 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:07.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 -- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0600b2880 msgr2=0x7fa0600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0600b2880 0x7fa0600b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0098e0 tx=0x7fa06c000bc0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 -- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0781045d0 msgr2=0x7fa07819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0781045d0 0x7fa07819ad60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa06800e9a0 tx=0x7fa06800ed60 comp rx=0 tx=0).stop 2024-05-04T03:10:07.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 -- 172.21.15.39:0/1553948261 shutdown_connections 2024-05-04T03:10:07.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0600b2880 0x7fa0600b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0781045d0 0x7fa07819ad60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa078109850 0x7fa07819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 --2- 172.21.15.39:0/1553948261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa078108f00 0x7fa07819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 -- 172.21.15.39:0/1553948261 >> 172.21.15.39:0/1553948261 conn(0x7fa0780ffea0 msgr2=0x7fa0781022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:07.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 -- 172.21.15.39:0/1553948261 shutdown_connections 2024-05-04T03:10:07.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:07.645+0000 7fa07f9c2700 1 -- 172.21.15.39:0/1553948261 wait complete. 2024-05-04T03:10:08.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-04T03:10:08.644 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:08.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:08 smithi167 ceph-mon[33614]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:08.680 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:08 smithi039 ceph-mon[29715]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:08.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:08 smithi066 ceph-mon[33264]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 -- 172.21.15.39:0/2366432618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105480 msgr2=0x7f5a10105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 --2- 172.21.15.39:0/2366432618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105480 0x7f5a10105890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009990 tx=0x7f5a0c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 -- 172.21.15.39:0/2366432618 shutdown_connections 2024-05-04T03:10:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 --2- 172.21.15.39:0/2366432618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105480 0x7f5a10105890 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 --2- 172.21.15.39:0/2366432618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1010e940 0x7f5a10110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 --2- 172.21.15.39:0/2366432618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a10105e60 0x7f5a1010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 -- 172.21.15.39:0/2366432618 >> 172.21.15.39:0/2366432618 conn(0x7f5a100fef50 msgr2=0x7f5a10101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 -- 172.21.15.39:0/2366432618 shutdown_connections 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.444+0000 7f5a1808a700 1 -- 172.21.15.39:0/2366432618 wait complete. 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.445+0000 7f5a1808a700 1 Processor -- start 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.445+0000 7f5a1808a700 1 -- start start 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.445+0000 7f5a1808a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a10105480 0x7f5a1019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.445+0000 7f5a1808a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105e60 0x7f5a1019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.445+0000 7f5a1808a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1010e940 0x7f5a1019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.445+0000 7f5a1808a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10114350 con 0x7f5a10105e60 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.445+0000 7f5a1808a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101141d0 con 0x7f5a10105480 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.445+0000 7f5a1808a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101144d0 con 0x7f5a1010e940 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105e60 0x7f5a1019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105e60 0x7f5a1019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40180/0 (socket says 172.21.15.39:40180) 2024-05-04T03:10:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 -- 172.21.15.39:0/3098059029 learned_addr learned my addr 172.21.15.39:0/3098059029 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a16627700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1010e940 0x7f5a1019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15e26700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a10105480 0x7f5a1019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 -- 172.21.15.39:0/3098059029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1010e940 msgr2=0x7f5a1019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1010e940 0x7f5a1019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 -- 172.21.15.39:0/3098059029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a10105480 msgr2=0x7f5a1019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a10105480 0x7f5a1019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 -- 172.21.15.39:0/3098059029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1019fc30 con 0x7f5a10105e60 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a15625700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105e60 0x7f5a1019b300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c8f0 tx=0x7f5a0000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a06ffd700 1 -- 172.21.15.39:0/3098059029 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00007940 con 0x7f5a10105e60 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a06ffd700 1 -- 172.21.15.39:0/3098059029 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a0000f450 con 0x7f5a10105e60 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a1808a700 1 -- 172.21.15.39:0/3098059029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1019ff20 con 0x7f5a10105e60 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a06ffd700 1 -- 172.21.15.39:0/3098059029 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00018660 con 0x7f5a10105e60 2024-05-04T03:10:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.446+0000 7f5a1808a700 1 -- 172.21.15.39:0/3098059029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a101a7840 con 0x7f5a10105e60 2024-05-04T03:10:10.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.449+0000 7f5a06ffd700 1 -- 172.21.15.39:0/3098059029 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5a00018800 con 0x7f5a10105e60 2024-05-04T03:10:10.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.449+0000 7f5a06ffd700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59fc0b2880 0x7f59fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a15e26700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59fc0b2880 0x7f59fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a06ffd700 1 -- 172.21.15.39:0/3098059029 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5a000e2130 con 0x7f5a10105e60 2024-05-04T03:10:10.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a26a9eb80 1 Processor -- start 2024-05-04T03:10:10.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a26a9eb80 1 -- start start 2024-05-04T03:10:10.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a15e26700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59fc0b2880 0x7f59fc0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c006010 tx=0x7f5a0c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:10.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0866da0 0x563df08671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0860ef0 0x563df0861300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563df0783810 con 0x563df0861840 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563df0722770 con 0x563df0860ef0 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.450+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563df077f7d0 con 0x563df0866da0 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40190/0 (socket says 172.21.15.39:40190) 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 -- 172.21.15.39:0/2963631786 learned_addr learned my addr 172.21.15.39:0/2963631786 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a15625700 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0860ef0 0x563df0861300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a15e26700 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0866da0 0x563df08671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 -- 172.21.15.39:0/2963631786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0866da0 msgr2=0x563df08671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0866da0 0x563df08671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 -- 172.21.15.39:0/2963631786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0860ef0 msgr2=0x563df0861300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0860ef0 0x563df0861300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 -- 172.21.15.39:0/2963631786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df06d0880 con 0x563df0861840 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f5a16627700 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086d9f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5a08003fa0 tx=0x7f5a08009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c216710f9663ee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f59fbfff700 1 -- 172.21.15.39:0/2963631786 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08004bb0 con 0x563df0861840 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f59fbfff700 1 -- 172.21.15.39:0/2963631786 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a080048e0 con 0x563df0861840 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.451+0000 7f59fbfff700 1 -- 172.21.15.39:0/2963631786 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08007cc0 con 0x563df0861840 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.452+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2963631786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 msgr2=0x563df086d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.452+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086d9f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5a08003fa0 tx=0x7f5a08009ba0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.452+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2963631786 shutdown_connections 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.452+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086d9f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.452+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0866da0 0x563df08671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.452+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2963631786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0860ef0 0x563df0861300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.452+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2963631786 >> 172.21.15.39:0/2963631786 conn(0x563df06e3df0 msgr2=0x563df085bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.452+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2963631786 shutdown_connections 2024-05-04T03:10:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2963631786 wait complete. 2024-05-04T03:10:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 -- start start 2024-05-04T03:10:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 Processor -- start 2024-05-04T03:10:10.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 0x563df086aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df086b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563df086e170 con 0x563df0861840 2024-05-04T03:10:10.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563df06e9be0 con 0x563df0866da0 2024-05-04T03:10:10.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.453+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563df086e2f0 con 0x563df0860ef0 2024-05-04T03:10:10.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a26a9eb80 10 client.0 _ll_register_callbacks cb 0x563df071b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40204/0 (socket says 172.21.15.39:40204) 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 -- 172.21.15.39:0/2087179185 learned_addr learned my addr 172.21.15.39:0/2087179185 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15e26700 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 0x563df086aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a16627700 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df086b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 -- 172.21.15.39:0/2087179185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 msgr2=0x563df086aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 0x563df086aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 -- 172.21.15.39:0/2087179185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 msgr2=0x563df086b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df086b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 -- 172.21.15.39:0/2087179185 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df071f640 con 0x563df0861840 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a15625700 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086b410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c8c0 tx=0x7f5a000a9210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed878a1104f9f9b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:10.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f59f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f59f97fa700 1 -- 172.21.15.39:0/2087179185 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000c2030 con 0x563df0861840 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f59f97fa700 1 -- 172.21.15.39:0/2087179185 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5a000a5cf0 con 0x563df0861840 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2087179185 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a080048e0 con 0x563df0861840 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f59f97fa700 1 -- 172.21.15.39:0/2087179185 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a000a9700 con 0x563df0861840 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a26a9eb80 10 client.15048 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.454+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2087179185 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563df0878e30 con 0x563df0861840 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f59f97fa700 1 -- 172.21.15.39:0/2087179185 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a080048e0 con 0x563df0861840 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f59f97fa700 1 -- 172.21.15.39:0/2087179185 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5a0008fce0 con 0x563df0861840 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f5a26a9eb80 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f5a26a9eb80 10 client.15048 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f5a26a9eb80 20 client.15048 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x563df087b4a0 0x563df087c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f5a26a9eb80 4 client.15048 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2087179185 --> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563df07911f0 con 0x563df087b4a0 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.455+0000 7f5a15e26700 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x563df087b4a0 0x563df087c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.456+0000 7f59f97fa700 10 client.15048 ms_handle_connect on v2:172.21.15.167:6838/975309166 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.456+0000 7f5a15e26700 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x563df087b4a0 0x563df087c910 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f018e28a2321e24e server_cookie=624bb66c283dac4a in_seq=0 out_seq=0 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f59f97fa700 1 -- 172.21.15.39:0/2087179185 <== mds.1 v2:172.21.15.167:6838/975309166 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563df07911f0 con 0x563df087b4a0 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f59f97fa700 10 client.15048 handle_command_reply: tid=0 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 client.15048 shutdown 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2087179185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 msgr2=0x563df086b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086b410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c8c0 tx=0x7f5a000a9210 comp rx=0 tx=0).stop 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2087179185 shutdown_connections 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x563df087b4a0 0x563df087c910 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df086b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 0x563df086aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f59f97fa700 0 client.15048 ms_handle_reset on v2:172.21.15.167:6838/975309166 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2087179185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df086b410 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2087179185 >> 172.21.15.39:0/2087179185 conn(0x563df06e3df0 msgr2=0x563df085fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2087179185 shutdown_connections 2024-05-04T03:10:10.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2087179185 wait complete. 2024-05-04T03:10:10.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.457+0000 7f5a26a9eb80 20 client.15048 trim_cache size 0 max 16384 2024-05-04T03:10:10.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.478+0000 7f5a26a9eb80 1 Processor -- start 2024-05-04T03:10:10.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.478+0000 7f5a26a9eb80 1 -- start start 2024-05-04T03:10:10.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:10 smithi039 ceph-mon[29715]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:10.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.478+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0860ef0 0x563df0865680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.478+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0861840 0x563df08775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.478+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df0868870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.478+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563df087ac50 con 0x563df0860ef0 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.478+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563df087b320 con 0x563df0866da0 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.478+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563df087b9f0 con 0x563df0861840 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0860ef0 0x563df0865680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0860ef0 0x563df0865680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40210/0 (socket says 172.21.15.39:40210) 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 -- 172.21.15.39:0/2682278640 learned_addr learned my addr 172.21.15.39:0/2682278640 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a16627700 1 --2- 172.21.15.39:0/2682278640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df0868870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 -- 172.21.15.39:0/2682278640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0861840 msgr2=0x563df08775f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 --2- 172.21.15.39:0/2682278640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0861840 0x563df08775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 -- 172.21.15.39:0/2682278640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 msgr2=0x563df0868870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 --2- 172.21.15.39:0/2682278640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df0868870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 -- 172.21.15.39:0/2682278640 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df0869440 con 0x563df0860ef0 2024-05-04T03:10:10.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f5a15e26700 1 --2- 172.21.15.39:0/2682278640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0860ef0 0x563df0865680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c000c00 tx=0x7f5a0c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4e8030864db268 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:10.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f59fb7fe700 1 -- 172.21.15.39:0/2682278640 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c03a070 con 0x563df0860ef0 2024-05-04T03:10:10.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f59fb7fe700 1 -- 172.21.15.39:0/2682278640 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a0c0053d0 con 0x563df0860ef0 2024-05-04T03:10:10.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.479+0000 7f59fb7fe700 1 -- 172.21.15.39:0/2682278640 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c034730 con 0x563df0860ef0 2024-05-04T03:10:10.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.480+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2682278640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0860ef0 msgr2=0x563df0865680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.480+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2682278640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0860ef0 0x563df0865680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c000c00 tx=0x7f5a0c021000 comp rx=0 tx=0).stop 2024-05-04T03:10:10.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.480+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2682278640 shutdown_connections 2024-05-04T03:10:10.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.480+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2682278640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0860ef0 0x563df0865680 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.480+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2682278640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df0868870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.480+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/2682278640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0861840 0x563df08775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.480+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2682278640 >> 172.21.15.39:0/2682278640 conn(0x563df06e3df0 msgr2=0x563df085b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.481+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2682278640 shutdown_connections 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.481+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/2682278640 wait complete. 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.481+0000 7f5a26a9eb80 1 -- start start 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.481+0000 7f5a26a9eb80 1 Processor -- start 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.481+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 0x563df0885d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df08862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a26a9eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df0880d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563df087c790 con 0x563df0861840 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563df087c0c0 con 0x563df0866da0 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a26a9eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563df087ce60 con 0x563df0860ef0 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df08862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df08862d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40216/0 (socket says 172.21.15.39:40216) 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 -- 172.21.15.39:0/1417287052 learned_addr learned my addr 172.21.15.39:0/1417287052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a26a9eb80 10 client.0 _ll_register_callbacks cb 0x563df071b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15e26700 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 0x563df0885d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a16627700 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df0880d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 -- 172.21.15.39:0/1417287052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 msgr2=0x563df0885d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 0x563df0885d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 -- 172.21.15.39:0/1417287052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 msgr2=0x563df0880d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df0880d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 -- 172.21.15.39:0/1417287052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df08813e0 con 0x563df0861840 2024-05-04T03:10:10.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a15625700 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df08862d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000c00 tx=0x7f5a00007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66250731796419d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f59f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f59f97fa700 1 -- 172.21.15.39:0/1417287052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000c2020 con 0x563df0861840 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f59f97fa700 1 -- 172.21.15.39:0/1417287052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5a00091300 con 0x563df0861840 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.482+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/1417287052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c0053d0 con 0x563df0861840 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f59f97fa700 1 -- 172.21.15.39:0/1417287052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a000a9960 con 0x563df0861840 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a26a9eb80 10 client.15054 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/1417287052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563df08e7220 con 0x563df0861840 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f59f97fa700 1 -- 172.21.15.39:0/1417287052 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a0c0053d0 con 0x563df0861840 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f59f97fa700 1 -- 172.21.15.39:0/1417287052 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5a0008a070 con 0x563df0861840 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a26a9eb80 10 client.15054 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a26a9eb80 10 client.15054 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a26a9eb80 20 client.15054 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x563df08e94b0 0x563df08ea920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a26a9eb80 4 client.15054 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/1417287052 --> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] -- command(tid 0: {"prefix": "status"}) -- 0x563df078b290 con 0x563df08e94b0 2024-05-04T03:10:10.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.483+0000 7f5a15e26700 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x563df08e94b0 0x563df08ea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a15e26700 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x563df08e94b0 0x563df08ea920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e6b04286e124ad6 server_cookie=940337746e51088b in_seq=0 out_seq=0 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f59f97fa700 10 client.15054 ms_handle_connect on v2:172.21.15.167:6838/975309166 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f59f97fa700 1 -- 172.21.15.39:0/1417287052 <== mds.1 v2:172.21.15.167:6838/975309166 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563df078b290 con 0x563df08e94b0 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f59f97fa700 10 client.15054 handle_command_reply: tid=0 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 client.15054 shutdown 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/1417287052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 msgr2=0x563df08862d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df08862d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5a00000c00 tx=0x7f5a00007da0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/1417287052 shutdown_connections 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x563df08e94b0 0x563df08ea920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563df0866da0 0x563df0880d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563df0860ef0 0x563df0885d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f59f97fa700 0 client.15054 ms_handle_reset on v2:172.21.15.167:6838/975309166 2024-05-04T03:10:10.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 --2- 172.21.15.39:0/1417287052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563df0861840 0x563df08862d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/1417287052 >> 172.21.15.39:0/1417287052 conn(0x563df06e3df0 msgr2=0x563df08696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:10.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/1417287052 shutdown_connections 2024-05-04T03:10:10.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.484+0000 7f5a26a9eb80 1 -- 172.21.15.39:0/1417287052 wait complete. 2024-05-04T03:10:10.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a26a9eb80 20 client.15054 trim_cache size 0 max 16384 2024-05-04T03:10:10.491 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24490, 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 136.36040305099999, 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 180.82355197300001 2024-05-04T03:10:10.492 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:10.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 -- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59fc0b2880 msgr2=0x7f59fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59fc0b2880 0x7f59fc0b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c006010 tx=0x7f5a0c00b540 comp rx=0 tx=0).stop 2024-05-04T03:10:10.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 -- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105e60 msgr2=0x7f5a1019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:10.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105e60 0x7f5a1019b300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c8f0 tx=0x7f5a0000cc00 comp rx=0 tx=0).stop 2024-05-04T03:10:10.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 -- 172.21.15.39:0/3098059029 shutdown_connections 2024-05-04T03:10:10.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f59fc0b2880 0x7f59fc0b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a1010e940 0x7f5a1019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a10105480 0x7f5a1019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 --2- 172.21.15.39:0/3098059029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a10105e60 0x7f5a1019b300 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:10.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 -- 172.21.15.39:0/3098059029 >> 172.21.15.39:0/3098059029 conn(0x7f5a100fef50 msgr2=0x7f5a100ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:10.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 -- 172.21.15.39:0/3098059029 shutdown_connections 2024-05-04T03:10:10.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:10.485+0000 7f5a1808a700 1 -- 172.21.15.39:0/3098059029 wait complete. 2024-05-04T03:10:10.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:10 smithi167 ceph-mon[33614]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:10.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:10 smithi066 ceph-mon[33264]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:11.314 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-04T03:10:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:12.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:12 smithi167 ceph-mon[33614]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:12.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:12 smithi039 ceph-mon[29715]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:12.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:12 smithi066 ceph-mon[33264]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:13.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 -- 172.21.15.39:0/2541929886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2308108740 msgr2=0x7f23081100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 --2- 172.21.15.39:0/2541929886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2308108740 0x7f23081100a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f230000b0d0 tx=0x7f230000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 -- 172.21.15.39:0/2541929886 shutdown_connections 2024-05-04T03:10:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 --2- 172.21.15.39:0/2541929886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2308108740 0x7f23081100a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 --2- 172.21.15.39:0/2541929886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23081034c0 0x7f23081038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 --2- 172.21.15.39:0/2541929886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2308107df0 0x7f2308108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 -- 172.21.15.39:0/2541929886 >> 172.21.15.39:0/2541929886 conn(0x7f23080fedc0 msgr2=0x7f23081011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 -- 172.21.15.39:0/2541929886 shutdown_connections 2024-05-04T03:10:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.398+0000 7f2310744700 1 -- 172.21.15.39:0/2541929886 wait complete. 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.399+0000 7f2310744700 1 Processor -- start 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.399+0000 7f2310744700 1 -- start start 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.399+0000 7f2310744700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23081034c0 0x7f230819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.399+0000 7f2310744700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2308107df0 0x7f230819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.399+0000 7f2310744700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2308108740 0x7f230819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.399+0000 7f2310744700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2308113220 con 0x7f23081034c0 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.399+0000 7f2310744700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f23081130a0 con 0x7f2308108740 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.399+0000 7f2310744700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23081133a0 con 0x7f2308107df0 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23081034c0 0x7f230819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23081034c0 0x7f230819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40236/0 (socket says 172.21.15.39:40236) 2024-05-04T03:10:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 -- 172.21.15.39:0/3138222085 learned_addr learned my addr 172.21.15.39:0/3138222085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230dcdf700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2308107df0 0x7f230819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230ece1700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2308108740 0x7f230819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 -- 172.21.15.39:0/3138222085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2308107df0 msgr2=0x7f230819c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2308107df0 0x7f230819c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 -- 172.21.15.39:0/3138222085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2308108740 msgr2=0x7f230819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2308108740 0x7f230819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 -- 172.21.15.39:0/3138222085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23081a6e90 con 0x7f23081034c0 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230ece1700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2308108740 0x7f230819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f230e4e0700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23081034c0 0x7f230819bb40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f230400d980 tx=0x7f230400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f22ff7fe700 1 -- 172.21.15.39:0/3138222085 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304016070 con 0x7f23081034c0 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f22ff7fe700 1 -- 172.21.15.39:0/3138222085 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2304004830 con 0x7f23081034c0 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f2310744700 1 -- 172.21.15.39:0/3138222085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23081a7180 con 0x7f23081034c0 2024-05-04T03:10:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.400+0000 7f22ff7fe700 1 -- 172.21.15.39:0/3138222085 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304005440 con 0x7f23081034c0 2024-05-04T03:10:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.401+0000 7f2310744700 1 -- 172.21.15.39:0/3138222085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23081a76c0 con 0x7f23081034c0 2024-05-04T03:10:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.403+0000 7f22ff7fe700 1 -- 172.21.15.39:0/3138222085 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f23040055e0 con 0x7f23081034c0 2024-05-04T03:10:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.404+0000 7f22ff7fe700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22f40b2880 0x7f22f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.404+0000 7f230dcdf700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22f40b2880 0x7f22f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.404+0000 7f22ff7fe700 1 -- 172.21.15.39:0/3138222085 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f23040e1ff0 con 0x7f23081034c0 2024-05-04T03:10:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.404+0000 7f231f158b80 1 Processor -- start 2024-05-04T03:10:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f230dcdf700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22f40b2880 0x7f22f40b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2308102440 tx=0x7f22f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f231f158b80 1 -- start start 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f231f158b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f231f158b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b21c0 0x55c0a77b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f231f158b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f231f158b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c0a76d48f0 con 0x55c0a77b2b10 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f231f158b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c0a7639d50 con 0x55c0a77b21c0 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f231f158b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0a76d08b0 con 0x55c0a77b80d0 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f230ece1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f230ece1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77bedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40248/0 (socket says 172.21.15.39:40248) 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f230e4e0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f230ece1700 1 -- 172.21.15.39:0/2581050422 learned_addr learned my addr 172.21.15.39:0/2581050422 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.405+0000 7f230dcdf700 1 --2- 172.21.15.39:0/2581050422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b21c0 0x55c0a77b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f230e4e0700 1 -- 172.21.15.39:0/2581050422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b21c0 msgr2=0x55c0a77b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f230e4e0700 1 --2- 172.21.15.39:0/2581050422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b21c0 0x55c0a77b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f230e4e0700 1 -- 172.21.15.39:0/2581050422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 msgr2=0x55c0a77bedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f230e4e0700 1 --2- 172.21.15.39:0/2581050422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77bedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f230e4e0700 1 -- 172.21.15.39:0/2581050422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a7621880 con 0x55c0a77b80d0 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f230e4e0700 1 --2- 172.21.15.39:0/2581050422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77b84e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f23040dfb50 tx=0x7f23040a9220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76d2816a9114aa69 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f22fcff9700 1 -- 172.21.15.39:0/2581050422 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040ad9d0 con 0x55c0a77b80d0 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f22fcff9700 1 -- 172.21.15.39:0/2581050422 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23040b2480 con 0x55c0a77b80d0 2024-05-04T03:10:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.406+0000 7f22fcff9700 1 -- 172.21.15.39:0/2581050422 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040a9880 con 0x55c0a77b80d0 2024-05-04T03:10:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 -- 172.21.15.39:0/2581050422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 msgr2=0x55c0a77b84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 --2- 172.21.15.39:0/2581050422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77b84e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f23040dfb50 tx=0x7f23040a9220 comp rx=0 tx=0).stop 2024-05-04T03:10:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 -- 172.21.15.39:0/2581050422 shutdown_connections 2024-05-04T03:10:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 --2- 172.21.15.39:0/2581050422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77bedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 --2- 172.21.15.39:0/2581050422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77b84e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 --2- 172.21.15.39:0/2581050422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b21c0 0x55c0a77b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 -- 172.21.15.39:0/2581050422 >> 172.21.15.39:0/2581050422 conn(0x55c0a7634df0 msgr2=0x55c0a77ad210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 -- 172.21.15.39:0/2581050422 shutdown_connections 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.407+0000 7f231f158b80 1 -- 172.21.15.39:0/2581050422 wait complete. 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 1 -- start start 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 1 Processor -- start 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b21c0 0x55c0a77bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 0x55c0a77bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77bce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c0a77bf4a0 con 0x55c0a77b21c0 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c0a768de80 con 0x55c0a77b2b10 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0a77bf620 con 0x55c0a77b80d0 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f231f158b80 10 client.0 _ll_register_callbacks cb 0x55c0a766c900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f230e4e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b21c0 0x55c0a77bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f230e4e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b21c0 0x55c0a77bc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40260/0 (socket says 172.21.15.39:40260) 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f230e4e0700 1 -- 172.21.15.39:0/2969662041 learned_addr learned my addr 172.21.15.39:0/2969662041 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.408+0000 7f230ece1700 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f230dcdf700 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 0x55c0a77bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f230e4e0700 1 -- 172.21.15.39:0/2969662041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 msgr2=0x55c0a77bce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f230e4e0700 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77bce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f230e4e0700 1 -- 172.21.15.39:0/2969662041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 msgr2=0x55c0a77bc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f230e4e0700 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 0x55c0a77bc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f230e4e0700 1 -- 172.21.15.39:0/2969662041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a768b710 con 0x55c0a77b21c0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f230e4e0700 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b21c0 0x55c0a77bc3a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2304000c00 tx=0x7f2304004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253e586d2b460002 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f22f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f22f1ffb700 1 -- 172.21.15.39:0/2969662041 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040c8060 con 0x55c0a77b21c0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f22f1ffb700 1 -- 172.21.15.39:0/2969662041 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f23040a9880 con 0x55c0a77b21c0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f231f158b80 1 -- 172.21.15.39:0/2969662041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23040b2480 con 0x55c0a77b21c0 2024-05-04T03:10:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f22f1ffb700 1 -- 172.21.15.39:0/2969662041 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2304088050 con 0x55c0a77b21c0 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f231f158b80 10 client.15066 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.409+0000 7f231f158b80 1 -- 172.21.15.39:0/2969662041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0a77ca1c0 con 0x55c0a77b21c0 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f22f1ffb700 1 -- 172.21.15.39:0/2969662041 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23040a9a80 con 0x55c0a77b21c0 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f22f1ffb700 1 -- 172.21.15.39:0/2969662041 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f23040a8880 con 0x55c0a77b21c0 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f231f158b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f231f158b80 10 client.15066 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f231f158b80 20 client.15066 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f231f158b80 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c0a77cc2d0 0x55c0a77cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f231f158b80 4 client.15066 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f231f158b80 1 -- 172.21.15.39:0/2969662041 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f23040b2480 con 0x55c0a77cc2d0 2024-05-04T03:10:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.410+0000 7f230dcdf700 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c0a77cc2d0 0x55c0a77cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.411+0000 7f230dcdf700 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c0a77cc2d0 0x55c0a77cd740 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c215d83b480f942 server_cookie=232d049b345f4d2f in_seq=0 out_seq=0 2024-05-04T03:10:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.411+0000 7f22f1ffb700 10 client.15066 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f22f1ffb700 1 -- 172.21.15.39:0/2969662041 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f23040b2480 con 0x55c0a77cc2d0 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f22f1ffb700 10 client.15066 handle_command_reply: tid=0 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 client.15066 shutdown 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 -- 172.21.15.39:0/2969662041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b21c0 msgr2=0x55c0a77bc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b21c0 0x55c0a77bc3a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2304000c00 tx=0x7f2304004ef0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 -- 172.21.15.39:0/2969662041 shutdown_connections 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c0a77cc2d0 0x55c0a77cd740 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b21c0 0x55c0a77bc3a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b80d0 0x55c0a77bce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f22f1ffb700 0 client.15066 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 --2- 172.21.15.39:0/2969662041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 0x55c0a77bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 -- 172.21.15.39:0/2969662041 >> 172.21.15.39:0/2969662041 conn(0x55c0a7634df0 msgr2=0x55c0a77b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 -- 172.21.15.39:0/2969662041 shutdown_connections 2024-05-04T03:10:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 1 -- 172.21.15.39:0/2969662041 wait complete. 2024-05-04T03:10:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.412+0000 7f231f158b80 20 client.15066 trim_cache size 0 max 16384 2024-05-04T03:10:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.433+0000 7f231f158b80 1 Processor -- start 2024-05-04T03:10:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.433+0000 7f231f158b80 1 -- start start 2024-05-04T03:10:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f231f158b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 0x55c0a77bcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f231f158b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b80d0 0x55c0a77b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f231f158b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 0x55c0a77b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f231f158b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c0a77cba80 con 0x55c0a77b80d0 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f231f158b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c0a77cc150 con 0x55c0a77b2b10 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f231f158b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0a77cc820 con 0x55c0a77b18c0 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b80d0 0x55c0a77b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b80d0 0x55c0a77b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40262/0 (socket says 172.21.15.39:40262) 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 -- 172.21.15.39:0/2122801399 learned_addr learned my addr 172.21.15.39:0/2122801399 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230e4e0700 1 --2- 172.21.15.39:0/2122801399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 0x55c0a77bcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 -- 172.21.15.39:0/2122801399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 msgr2=0x55c0a77b9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 --2- 172.21.15.39:0/2122801399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 0x55c0a77b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 -- 172.21.15.39:0/2122801399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 msgr2=0x55c0a77bcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 --2- 172.21.15.39:0/2122801399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 0x55c0a77bcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 -- 172.21.15.39:0/2122801399 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a77ae8a0 con 0x55c0a77b80d0 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f230dcdf700 1 --2- 172.21.15.39:0/2122801399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b80d0 0x55c0a77b69f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55c0a77aea70 tx=0x7f22f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5105682c2fdffa6c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.434+0000 7f22f3fff700 1 -- 172.21.15.39:0/2122801399 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8011cb0 con 0x55c0a77b80d0 2024-05-04T03:10:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f22f3fff700 1 -- 172.21.15.39:0/2122801399 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22f8011e50 con 0x55c0a77b80d0 2024-05-04T03:10:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f22f3fff700 1 -- 172.21.15.39:0/2122801399 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8027260 con 0x55c0a77b80d0 2024-05-04T03:10:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 -- 172.21.15.39:0/2122801399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b80d0 msgr2=0x55c0a77b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 --2- 172.21.15.39:0/2122801399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b80d0 0x55c0a77b69f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55c0a77aea70 tx=0x7f22f8013000 comp rx=0 tx=0).stop 2024-05-04T03:10:13.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 -- 172.21.15.39:0/2122801399 shutdown_connections 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 --2- 172.21.15.39:0/2122801399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 0x55c0a77b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 --2- 172.21.15.39:0/2122801399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b2b10 0x55c0a77bcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 --2- 172.21.15.39:0/2122801399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b80d0 0x55c0a77b69f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 -- 172.21.15.39:0/2122801399 >> 172.21.15.39:0/2122801399 conn(0x55c0a7634df0 msgr2=0x55c0a77b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 -- 172.21.15.39:0/2122801399 shutdown_connections 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.435+0000 7f231f158b80 1 -- 172.21.15.39:0/2122801399 wait complete. 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 1 -- start start 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 1 Processor -- start 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 0x55c0a77d6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b80d0 0x55c0a77d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c0a77cd5c0 con 0x55c0a77b2b10 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c0a77ccef0 con 0x55c0a77b80d0 2024-05-04T03:10:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0a77cdc90 con 0x55c0a77b18c0 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f231f158b80 10 client.0 _ll_register_callbacks cb 0x55c0a766c900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f230dcdf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f230dcdf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40278/0 (socket says 172.21.15.39:40278) 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f230dcdf700 1 -- 172.21.15.39:0/409197211 learned_addr learned my addr 172.21.15.39:0/409197211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.436+0000 7f230ece1700 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b80d0 0x55c0a77d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f230e4e0700 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 0x55c0a77d6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f230dcdf700 1 -- 172.21.15.39:0/409197211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 msgr2=0x55c0a77d6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f230dcdf700 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 0x55c0a77d6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f230dcdf700 1 -- 172.21.15.39:0/409197211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b80d0 msgr2=0x55c0a77d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f230dcdf700 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b80d0 0x55c0a77d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f230dcdf700 1 -- 172.21.15.39:0/409197211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a77d25f0 con 0x55c0a77b2b10 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f230dcdf700 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77d1b70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005580 tx=0x7f22f80055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c21573b3160ad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f22f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f22f1ffb700 1 -- 172.21.15.39:0/409197211 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f802c040 con 0x55c0a77b2b10 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f22f1ffb700 1 -- 172.21.15.39:0/409197211 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f22f8027320 con 0x55c0a77b2b10 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f231f158b80 1 -- 172.21.15.39:0/409197211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f8011e50 con 0x55c0a77b2b10 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f22f1ffb700 1 -- 172.21.15.39:0/409197211 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22f8027520 con 0x55c0a77b2b10 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f231f158b80 10 client.15072 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f231f158b80 1 -- 172.21.15.39:0/409197211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0a78382a0 con 0x55c0a77b2b10 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.437+0000 7f22f1ffb700 1 -- 172.21.15.39:0/409197211 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22f8011e50 con 0x55c0a77b2b10 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f22f1ffb700 1 -- 172.21.15.39:0/409197211 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f22f802f070 con 0x55c0a77b2b10 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f231f158b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f231f158b80 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f231f158b80 20 client.15072 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f231f158b80 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c0a783a390 0x55c0a783b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f231f158b80 4 client.15072 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f231f158b80 1 -- 172.21.15.39:0/409197211 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "status"}) -- 0x55c0a76e22d0 con 0x55c0a783a390 2024-05-04T03:10:13.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f230e4e0700 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c0a783a390 0x55c0a783b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f22f1ffb700 10 client.15072 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.438+0000 7f230e4e0700 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c0a783a390 0x55c0a783b800 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80910fd7650ebac server_cookie=f659f5863ea89b36 in_seq=0 out_seq=0 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f22f1ffb700 1 -- 172.21.15.39:0/409197211 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c0a76e22d0 con 0x55c0a783a390 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f22f1ffb700 10 client.15072 handle_command_reply: tid=0 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 client.15072 shutdown 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 -- 172.21.15.39:0/409197211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 msgr2=0x55c0a77d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77d1b70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005580 tx=0x7f22f80055b0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 -- 172.21.15.39:0/409197211 shutdown_connections 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c0a783a390 0x55c0a783b800 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c0a77b80d0 0x55c0a77d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0a77b18c0 0x55c0a77d6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f22f1ffb700 0 client.15072 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 --2- 172.21.15.39:0/409197211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a77b2b10 0x55c0a77d1b70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 -- 172.21.15.39:0/409197211 >> 172.21.15.39:0/409197211 conn(0x55c0a7634df0 msgr2=0x55c0a77c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 -- 172.21.15.39:0/409197211 shutdown_connections 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 1 -- 172.21.15.39:0/409197211 wait complete. 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f231f158b80 20 client.15072 trim_cache size 0 max 16384 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24506, 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:13.445 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:13.446 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 153.37121907900001, 2024-05-04T03:10:13.446 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:13.446 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:13.446 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:13.446 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 172.223681422 2024-05-04T03:10:13.446 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:13.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f2310744700 1 -- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22f40b2880 msgr2=0x7f22f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f2310744700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22f40b2880 0x7f22f40b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2308102440 tx=0x7f22f800b410 comp rx=0 tx=0).stop 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f2310744700 1 -- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23081034c0 msgr2=0x7f230819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.439+0000 7f2310744700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23081034c0 0x7f230819bb40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f230400d980 tx=0x7f230400dc90 comp rx=0 tx=0).stop 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.440+0000 7f2310744700 1 -- 172.21.15.39:0/3138222085 shutdown_connections 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.440+0000 7f2310744700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22f40b2880 0x7f22f40b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.440+0000 7f2310744700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23081034c0 0x7f230819bb40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.440+0000 7f2310744700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2308108740 0x7f230819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.440+0000 7f2310744700 1 --2- 172.21.15.39:0/3138222085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2308107df0 0x7f230819c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.440+0000 7f2310744700 1 -- 172.21.15.39:0/3138222085 >> 172.21.15.39:0/3138222085 conn(0x7f23080fedc0 msgr2=0x7f23081075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.440+0000 7f2310744700 1 -- 172.21.15.39:0/3138222085 shutdown_connections 2024-05-04T03:10:13.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:13.440+0000 7f2310744700 1 -- 172.21.15.39:0/3138222085 wait complete. 2024-05-04T03:10:14.236 INFO:tasks.cephadm:Setup complete, yielding 2024-05-04T03:10:14.236 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-04T03:10:14.246 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T03:10:14.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-04T03:10:14.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:14 smithi039 ceph-mon[29715]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:14 smithi167 ceph-mon[33614]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:14.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:14 smithi066 ceph-mon[33264]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:16.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 -- 172.21.15.39:0/3481322690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 msgr2=0x7ff7ac103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/3481322690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 0x7ff7ac103050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009990 tx=0x7ff7a0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 -- 172.21.15.39:0/3481322690 shutdown_connections 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/3481322690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 0x7ff7ac103050 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/3481322690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ac103ee0 0x7ff7ac1042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/3481322690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ac103590 0x7ff7ac1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 -- 172.21.15.39:0/3481322690 >> 172.21.15.39:0/3481322690 conn(0x7ff7ac0fe890 msgr2=0x7ff7ac100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 -- 172.21.15.39:0/3481322690 shutdown_connections 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.347+0000 7ff7b2a11700 1 -- 172.21.15.39:0/3481322690 wait complete. 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7b2a11700 1 Processor -- start 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7b2a11700 1 -- start start 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7b2a11700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 0x7ff7ac19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7b2a11700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ac103590 0x7ff7ac19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7b2a11700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ac103ee0 0x7ff7ac1a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7b2a11700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac111e40 con 0x7ff7ac102c40 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7b2a11700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac111cc0 con 0x7ff7ac103ee0 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7b2a11700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac111fc0 con 0x7ff7ac103590 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 0x7ff7ac19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 0x7ff7ac19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55736/0 (socket says 172.21.15.39:55736) 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7abfff700 1 -- 172.21.15.39:0/2925368653 learned_addr learned my addr 172.21.15.39:0/2925368653 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.348+0000 7ff7ab7fe700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ac103590 0x7ff7ac19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7b0fae700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ac103ee0 0x7ff7ac1a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7abfff700 1 -- 172.21.15.39:0/2925368653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ac103590 msgr2=0x7ff7ac19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7abfff700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ac103590 0x7ff7ac19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7abfff700 1 -- 172.21.15.39:0/2925368653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ac103ee0 msgr2=0x7ff7ac1a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:16.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7abfff700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ac103ee0 0x7ff7ac1a3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7abfff700 1 -- 172.21.15.39:0/2925368653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac1a45d0 con 0x7ff7ac102c40 2024-05-04T03:10:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7abfff700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 0x7ff7ac19ca70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0000c00 tx=0x7ff7a0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7a97fa700 1 -- 172.21.15.39:0/2925368653 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0027070 con 0x7ff7ac102c40 2024-05-04T03:10:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7a97fa700 1 -- 172.21.15.39:0/2925368653 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7a00040a0 con 0x7ff7ac102c40 2024-05-04T03:10:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac1a4860 con 0x7ff7ac102c40 2024-05-04T03:10:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7a97fa700 1 -- 172.21.15.39:0/2925368653 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0022d00 con 0x7ff7ac102c40 2024-05-04T03:10:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.349+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac1a4d10 con 0x7ff7ac102c40 2024-05-04T03:10:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.350+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ac0512c0 con 0x7ff7ac102c40 2024-05-04T03:10:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.352+0000 7ff7a97fa700 1 -- 172.21.15.39:0/2925368653 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff7a002a070 con 0x7ff7ac102c40 2024-05-04T03:10:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.352+0000 7ff7a97fa700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff7940b2900 0x7ff7940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.352+0000 7ff7ab7fe700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff7940b2900 0x7ff7940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.352+0000 7ff7a97fa700 1 -- 172.21.15.39:0/2925368653 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff7a00e39d0 con 0x7ff7ac102c40 2024-05-04T03:10:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.353+0000 7ff7ab7fe700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff7940b2900 0x7ff7940b4e10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff79c005950 tx=0x7ff79c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:16.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.356+0000 7ff7a97fa700 1 -- 172.21.15.39:0/2925368653 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7a0013070 con 0x7ff7ac102c40 2024-05-04T03:10:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.598+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff7ac063c00 con 0x7ff7940b2900 2024-05-04T03:10:16.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.600+0000 7ff7a97fa700 1 -- 172.21.15.39:0/2925368653 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff7ac063c00 con 0x7ff7940b2900 2024-05-04T03:10:16.600 INFO:teuthology.orchestra.run.smithi039.stdout:Backend: cephadm 2024-05-04T03:10:16.600 INFO:teuthology.orchestra.run.smithi039.stdout:Available: Yes 2024-05-04T03:10:16.601 INFO:teuthology.orchestra.run.smithi039.stdout:Paused: No 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff7940b2900 msgr2=0x7ff7940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff7940b2900 0x7ff7940b4e10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff79c005950 tx=0x7ff79c0058e0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 msgr2=0x7ff7ac19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 0x7ff7ac19ca70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0000c00 tx=0x7ff7a0009510 comp rx=0 tx=0).stop 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 shutdown_connections 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff7940b2900 0x7ff7940b4e10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ac102c40 0x7ff7ac19ca70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7ac103ee0 0x7ff7ac1a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 --2- 172.21.15.39:0/2925368653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ac103590 0x7ff7ac19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 >> 172.21.15.39:0/2925368653 conn(0x7ff7ac0fe890 msgr2=0x7ff7ac10b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 shutdown_connections 2024-05-04T03:10:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:16.602+0000 7ff7b2a11700 1 -- 172.21.15.39:0/2925368653 wait complete. 2024-05-04T03:10:16.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:16 smithi167 ceph-mon[33614]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:16.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:16 smithi039 ceph-mon[29715]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:16.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:16 smithi066 ceph-mon[33264]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:17.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-04T03:10:17.771 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:18.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:18 smithi039 ceph-mon[29715]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:18.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:18 smithi039 ceph-mon[29715]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:18 smithi167 ceph-mon[33614]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:18 smithi167 ceph-mon[33614]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:18.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:18 smithi066 ceph-mon[33264]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:18 smithi066 ceph-mon[33264]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 -- 172.21.15.39:0/25136930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54104380 msgr2=0x7f2c54104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 --2- 172.21.15.39:0/25136930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54104380 0x7f2c54104790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009990 tx=0x7f2c48009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 -- 172.21.15.39:0/25136930 shutdown_connections 2024-05-04T03:10:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 --2- 172.21.15.39:0/25136930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c54104d20 0x7f2c5410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 --2- 172.21.15.39:0/25136930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c54103250 0x7f2c54103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 --2- 172.21.15.39:0/25136930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54104380 0x7f2c54104790 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 -- 172.21.15.39:0/25136930 >> 172.21.15.39:0/25136930 conn(0x7f2c540fe890 msgr2=0x7f2c54100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 -- 172.21.15.39:0/25136930 shutdown_connections 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.488+0000 7f2c595b4700 1 -- 172.21.15.39:0/25136930 wait complete. 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c595b4700 1 Processor -- start 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c595b4700 1 -- start start 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c595b4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54103250 0x7f2c5419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c595b4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c54104380 0x7f2c5419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c595b4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c54104d20 0x7f2c541a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c595b4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410fce0 con 0x7f2c54103250 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c595b4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410fb60 con 0x7f2c54104d20 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c595b4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5410fe60 con 0x7f2c54104380 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c52ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54103250 0x7f2c5419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c537fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c54104d20 0x7f2c541a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c52ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54103250 0x7f2c5419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55756/0 (socket says 172.21.15.39:55756) 2024-05-04T03:10:19.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c52ffd700 1 -- 172.21.15.39:0/1102023088 learned_addr learned my addr 172.21.15.39:0/1102023088 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.489+0000 7f2c527fc700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c54104380 0x7f2c5419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c52ffd700 1 -- 172.21.15.39:0/1102023088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c54104380 msgr2=0x7f2c5419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c52ffd700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c54104380 0x7f2c5419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c52ffd700 1 -- 172.21.15.39:0/1102023088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c54104d20 msgr2=0x7f2c541a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c52ffd700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c54104d20 0x7f2c541a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c52ffd700 1 -- 172.21.15.39:0/1102023088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c541a2450 con 0x7f2c54103250 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c52ffd700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54103250 0x7f2c5419a8f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b4d0 tx=0x7f2c4400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c3bfff700 1 -- 172.21.15.39:0/1102023088 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44015020 con 0x7f2c54103250 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c541a2740 con 0x7f2c54103250 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c3bfff700 1 -- 172.21.15.39:0/1102023088 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c44011cc0 con 0x7f2c54103250 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c3bfff700 1 -- 172.21.15.39:0/1102023088 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44010b80 con 0x7f2c54103250 2024-05-04T03:10:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.490+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c541a2d00 con 0x7f2c54103250 2024-05-04T03:10:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.491+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c540512c0 con 0x7f2c54103250 2024-05-04T03:10:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.493+0000 7f2c3bfff700 1 -- 172.21.15.39:0/1102023088 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2c44018020 con 0x7f2c54103250 2024-05-04T03:10:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.494+0000 7f2c3bfff700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c3c0b2880 0x7f2c3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.494+0000 7f2c527fc700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c3c0b2880 0x7f2c3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.494+0000 7f2c3bfff700 1 -- 172.21.15.39:0/1102023088 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2c440e1e90 con 0x7f2c54103250 2024-05-04T03:10:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.494+0000 7f2c527fc700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c3c0b2880 0x7f2c3c0b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c48006010 tx=0x7f2c4801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.497+0000 7f2c3bfff700 1 -- 172.21.15.39:0/1102023088 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c4401f020 con 0x7f2c54103250 2024-05-04T03:10:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.720+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2c54063c00 con 0x7f2c3c0b2880 2024-05-04T03:10:19.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.734+0000 7f2c3bfff700 1 -- 172.21.15.39:0/1102023088 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f2c54063c00 con 0x7f2c3c0b2880 2024-05-04T03:10:19.735 INFO:teuthology.orchestra.run.smithi039.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-04T03:10:19.735 INFO:teuthology.orchestra.run.smithi039.stdout:mds.a smithi039 running (3m) 38s ago 3m 86.4M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 2d319958b6f3 2024-05-04T03:10:19.735 INFO:teuthology.orchestra.run.smithi039.stdout:mds.b smithi066 running (2m) 44s ago 2m 93.8M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 e870791c88c2 2024-05-04T03:10:19.735 INFO:teuthology.orchestra.run.smithi039.stdout:mds.c smithi167 running (3m) 44s ago 3m 87.2M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 c63b4a6b05de 2024-05-04T03:10:19.735 INFO:teuthology.orchestra.run.smithi039.stdout:mds.d smithi039 running (3m) 38s ago 3m 83.7M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 712b077342d1 2024-05-04T03:10:19.735 INFO:teuthology.orchestra.run.smithi039.stdout:mds.e smithi066 running (3m) 44s ago 3m 87.1M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 2ac28a1a403f 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mds.f smithi167 running (3m) 44s ago 3m 88.5M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 4c4e8da1ce11 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mds.g smithi039 running (3m) 38s ago 3m 86.4M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 cfacbc7484ee 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mds.h smithi066 running (3m) 44s ago 3m 89.2M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 3405fdf73df9 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mds.i smithi167 running (3m) 44s ago 3m 87.5M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 bee2384c7f1a 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mds.j smithi039 running (3m) 38s ago 3m 84.6M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 a7f4071649bb 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mds.k smithi066 running (3m) 44s ago 3m 90.1M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 ba2bcf5d2559 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mds.l smithi167 running (3m) 44s ago 3m 89.0M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 5db8322d980a 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.x smithi039 *:9283,8765 running (10m) 38s ago 10m 596M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 4730d304cd2a 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.y smithi066 *:8443,8765 running (8m) 44s ago 8m 532M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 d3214c33ee95 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.z smithi167 *:8443,8765 running (8m) 44s ago 8m 534M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 fb82acedd078 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a smithi039 running (10m) 38s ago 10m 121M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 8812331cba60 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b smithi066 running (9m) 44s ago 9m 107M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 b24312c8fd41 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c smithi167 running (9m) 44s ago 9m 107M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 e7adc71bf1ee 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 smithi039 running (8m) 38s ago 8m 126M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 f34163066569 2024-05-04T03:10:19.736 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 smithi066 running (8m) 44s ago 8m 126M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 d089e88590e1 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 smithi167 running (7m) 44s ago 7m 124M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 eaea622546bf 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 smithi039 running (7m) 38s ago 7m 110M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 56615f8fd63f 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 smithi066 running (6m) 44s ago 6m 124M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 c10393e99505 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 smithi167 running (6m) 44s ago 6m 121M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 b475e2986cbd 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 smithi039 running (6m) 38s ago 6m 110M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 bb46f39f70ef 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 smithi066 running (5m) 44s ago 5m 122M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 55669f7d9140 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 smithi167 running (5m) 44s ago 5m 122M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 719dedb0ab04 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 smithi039 running (4m) 38s ago 4m 121M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 c0d76dc23384 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 smithi066 running (4m) 44s ago 4m 110M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 92d584f108a2 2024-05-04T03:10:19.737 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 smithi167 running (3m) 44s ago 3m 118M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 0978b4fcde11 2024-05-04T03:10:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c3c0b2880 msgr2=0x7f2c3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c3c0b2880 0x7f2c3c0b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c48006010 tx=0x7f2c4801a040 comp rx=0 tx=0).stop 2024-05-04T03:10:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54103250 msgr2=0x7f2c5419a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54103250 0x7f2c5419a8f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400b4d0 tx=0x7f2c4400b7e0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 shutdown_connections 2024-05-04T03:10:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c3c0b2880 0x7f2c3c0b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c54103250 0x7f2c5419a8f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c54104d20 0x7f2c541a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 --2- 172.21.15.39:0/1102023088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c54104380 0x7f2c5419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.736+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 >> 172.21.15.39:0/1102023088 conn(0x7f2c540fe890 msgr2=0x7f2c541075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.737+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 shutdown_connections 2024-05-04T03:10:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:19.737+0000 7f2c595b4700 1 -- 172.21.15.39:0/1102023088 wait complete. 2024-05-04T03:10:20.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-04T03:10:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:20 smithi167 ceph-mon[33614]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:20.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:20 smithi039 ceph-mon[29715]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:20.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:20 smithi066 ceph-mon[33264]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:21.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:21 smithi167 ceph-mon[33614]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:21.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:21 smithi039 ceph-mon[29715]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:21.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:21 smithi066 ceph-mon[33264]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:22.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 -- 172.21.15.39:0/380691429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8104d10 msgr2=0x7ff4a810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:22.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/380691429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8104d10 0x7ff4a810b5c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000b0a0 tx=0x7ff4a000b3b0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 -- 172.21.15.39:0/380691429 shutdown_connections 2024-05-04T03:10:22.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/380691429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8104d10 0x7ff4a810b5c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/380691429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4a8103240 0x7ff4a8103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/380691429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8104370 0x7ff4a8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 -- 172.21.15.39:0/380691429 >> 172.21.15.39:0/380691429 conn(0x7ff4a80fe880 msgr2=0x7ff4a8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:22.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 -- 172.21.15.39:0/380691429 shutdown_connections 2024-05-04T03:10:22.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.605+0000 7ff4af1ba700 1 -- 172.21.15.39:0/380691429 wait complete. 2024-05-04T03:10:22.608 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:22 smithi039 ceph-mon[29715]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:22.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.606+0000 7ff4af1ba700 1 Processor -- start 2024-05-04T03:10:22.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.606+0000 7ff4af1ba700 1 -- start start 2024-05-04T03:10:22.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4af1ba700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8103240 0x7ff4a819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:22.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4af1ba700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8104370 0x7ff4a819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:22.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4af1ba700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4a8104d10 0x7ff4a81a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:22.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4af1ba700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a810eb50 con 0x7ff4a8103240 2024-05-04T03:10:22.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4af1ba700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a810e9d0 con 0x7ff4a8104d10 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4af1ba700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a810ecd0 con 0x7ff4a8104370 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8103240 0x7ff4a819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8103240 0x7ff4a819ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55762/0 (socket says 172.21.15.39:55762) 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 -- 172.21.15.39:0/2814018335 learned_addr learned my addr 172.21.15.39:0/2814018335 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4a7fff700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8104370 0x7ff4a819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4ad757700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4a8104d10 0x7ff4a81a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 -- 172.21.15.39:0/2814018335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8104370 msgr2=0x7ff4a819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8104370 0x7ff4a819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 -- 172.21.15.39:0/2814018335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4a8104d10 msgr2=0x7ff4a81a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4a8104d10 0x7ff4a81a3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 -- 172.21.15.39:0/2814018335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81a44e0 con 0x7ff4a8103240 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4ad757700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4a8104d10 0x7ff4a81a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.607+0000 7ff4acf56700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8103240 0x7ff4a819ca10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff49800c970 tx=0x7ff49800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:22.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.608+0000 7ff4a5ffb700 1 -- 172.21.15.39:0/2814018335 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498007b00 con 0x7ff4a8103240 2024-05-04T03:10:22.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.608+0000 7ff4a5ffb700 1 -- 172.21.15.39:0/2814018335 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff498007ca0 con 0x7ff4a8103240 2024-05-04T03:10:22.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.608+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a81a47d0 con 0x7ff4a8103240 2024-05-04T03:10:22.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.608+0000 7ff4a5ffb700 1 -- 172.21.15.39:0/2814018335 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498018660 con 0x7ff4a8103240 2024-05-04T03:10:22.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.608+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81a4d10 con 0x7ff4a8103240 2024-05-04T03:10:22.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.609+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a80512c0 con 0x7ff4a8103240 2024-05-04T03:10:22.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.611+0000 7ff4a5ffb700 1 -- 172.21.15.39:0/2814018335 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff498004020 con 0x7ff4a8103240 2024-05-04T03:10:22.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.611+0000 7ff4a5ffb700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff48c0b2880 0x7ff48c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:22.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.611+0000 7ff4a7fff700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff48c0b2880 0x7ff48c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:22.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.611+0000 7ff4a5ffb700 1 -- 172.21.15.39:0/2814018335 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff4980e2300 con 0x7ff4a8103240 2024-05-04T03:10:22.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.612+0000 7ff4a7fff700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff48c0b2880 0x7ff48c0b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff494005950 tx=0x7ff4940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:22.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.615+0000 7ff4a5ffb700 1 -- 172.21.15.39:0/2814018335 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff498053df0 con 0x7ff4a8103240 2024-05-04T03:10:22.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:22 smithi167 ceph-mon[33614]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:22.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:22 smithi066 ceph-mon[33264]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:22.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.840+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff4a81a4fd0 con 0x7ff48c0b2880 2024-05-04T03:10:22.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.846+0000 7ff4a5ffb700 1 -- 172.21.15.39:0/2814018335 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff4a81a4fd0 con 0x7ff48c0b2880 2024-05-04T03:10:22.847 INFO:teuthology.orchestra.run.smithi039.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-04T03:10:22.847 INFO:teuthology.orchestra.run.smithi039.stdout:mds.all 12/12 47s ago 3m smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-05-04T03:10:22.847 INFO:teuthology.orchestra.run.smithi039.stdout:mgr 3/3 47s ago 9m smithi039=x;smithi066=y;smithi167=z;count:3 2024-05-04T03:10:22.847 INFO:teuthology.orchestra.run.smithi039.stdout:mon 3/3 47s ago 9m smithi039:172.21.15.39=a;smithi066:172.21.15.66=b;smithi167:172.21.15.167=c;count:3 2024-05-04T03:10:22.847 INFO:teuthology.orchestra.run.smithi039.stdout:osd 12 47s ago - 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff48c0b2880 msgr2=0x7ff48c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff48c0b2880 0x7ff48c0b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff494005950 tx=0x7ff4940058e0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8103240 msgr2=0x7ff4a819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8103240 0x7ff4a819ca10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff49800c970 tx=0x7ff49800cd30 comp rx=0 tx=0).stop 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 shutdown_connections 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff48c0b2880 0x7ff48c0b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4a8103240 0x7ff4a819ca10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4a8104d10 0x7ff4a81a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 --2- 172.21.15.39:0/2814018335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4a8104370 0x7ff4a819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:22.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 >> 172.21.15.39:0/2814018335 conn(0x7ff4a80fe880 msgr2=0x7ff4a81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:22.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 shutdown_connections 2024-05-04T03:10:22.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:22.849+0000 7ff4af1ba700 1 -- 172.21.15.39:0/2814018335 wait complete. 2024-05-04T03:10:23.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-04T03:10:23.947 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:24.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:24 smithi039 ceph-mon[29715]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:24.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:24 smithi039 ceph-mon[29715]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:24.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:24 smithi066 ceph-mon[33264]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:24 smithi066 ceph-mon[33264]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:24.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:24 smithi167 ceph-mon[33614]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:24.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:24 smithi167 ceph-mon[33614]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 -- 172.21.15.39:0/2716934390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7424076650 msgr2=0x7f7424106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 --2- 172.21.15.39:0/2716934390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7424076650 0x7f7424106f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7414009990 tx=0x7f7414009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 -- 172.21.15.39:0/2716934390 shutdown_connections 2024-05-04T03:10:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 --2- 172.21.15.39:0/2716934390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7424076650 0x7f7424106f20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 --2- 172.21.15.39:0/2716934390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f742410a000 0x7f742410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 --2- 172.21.15.39:0/2716934390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74241074f0 0x7f7424109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 -- 172.21.15.39:0/2716934390 >> 172.21.15.39:0/2716934390 conn(0x7f74240fe890 msgr2=0x7f7424100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 -- 172.21.15.39:0/2716934390 shutdown_connections 2024-05-04T03:10:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.797+0000 7f7429557700 1 -- 172.21.15.39:0/2716934390 wait complete. 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7429557700 1 Processor -- start 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7429557700 1 -- start start 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7429557700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7424076650 0x7f742419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7429557700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74241074f0 0x7f742419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7429557700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742410a000 0x7f74241a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7429557700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f742410fc50 con 0x7f742410a000 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7429557700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f742410fad0 con 0x7f74241074f0 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7429557700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f742410fdd0 con 0x7f7424076650 2024-05-04T03:10:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f74237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742410a000 0x7f74241a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f74237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742410a000 0x7f74241a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33420/0 (socket says 172.21.15.39:33420) 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f74237fe700 1 -- 172.21.15.39:0/3350593228 learned_addr learned my addr 172.21.15.39:0/3350593228 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.798+0000 7f7422ffd700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7424076650 0x7f742419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f74227fc700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74241074f0 0x7f742419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f74237fe700 1 -- 172.21.15.39:0/3350593228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7424076650 msgr2=0x7f742419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f74237fe700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7424076650 0x7f742419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f74237fe700 1 -- 172.21.15.39:0/3350593228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74241074f0 msgr2=0x7f742419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f74237fe700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74241074f0 0x7f742419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f74237fe700 1 -- 172.21.15.39:0/3350593228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74241a2320 con 0x7f742410a000 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f74237fe700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742410a000 0x7f74241a1d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f741c007d70 tx=0x7f741c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f740bfff700 1 -- 172.21.15.39:0/3350593228 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c016020 con 0x7f742410a000 2024-05-04T03:10:25.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f740bfff700 1 -- 172.21.15.39:0/3350593228 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f741c00ad40 con 0x7f742410a000 2024-05-04T03:10:25.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74241a2610 con 0x7f742410a000 2024-05-04T03:10:25.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f740bfff700 1 -- 172.21.15.39:0/3350593228 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c0115e0 con 0x7f742410a000 2024-05-04T03:10:25.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.799+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74241a2b00 con 0x7f742410a000 2024-05-04T03:10:25.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.800+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74240512c0 con 0x7f742410a000 2024-05-04T03:10:25.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.803+0000 7f740bfff700 1 -- 172.21.15.39:0/3350593228 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f741c019020 con 0x7f742410a000 2024-05-04T03:10:25.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.803+0000 7f740bfff700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f740c0b2880 0x7f740c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:25.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.803+0000 7f7422ffd700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f740c0b2880 0x7f740c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:25.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.803+0000 7f740bfff700 1 -- 172.21.15.39:0/3350593228 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f741c0e26a0 con 0x7f742410a000 2024-05-04T03:10:25.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.804+0000 7f7422ffd700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f740c0b2880 0x7f740c0b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f741400b5c0 tx=0x7f741401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:25.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:25.807+0000 7f740bfff700 1 -- 172.21.15.39:0/3350593228 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f741c020020 con 0x7f742410a000 2024-05-04T03:10:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.027+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7424063c00 con 0x7f740c0b2880 2024-05-04T03:10:26.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.029+0000 7f740bfff700 1 -- 172.21.15.39:0/3350593228 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7424063c00 con 0x7f740c0b2880 2024-05-04T03:10:26.029 INFO:teuthology.orchestra.run.smithi039.stdout:HOST ADDR LABELS STATUS 2024-05-04T03:10:26.030 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 172.21.15.39 2024-05-04T03:10:26.030 INFO:teuthology.orchestra.run.smithi039.stdout:smithi066 172.21.15.66 2024-05-04T03:10:26.030 INFO:teuthology.orchestra.run.smithi039.stdout:smithi167 172.21.15.167 2024-05-04T03:10:26.030 INFO:teuthology.orchestra.run.smithi039.stdout:3 hosts in cluster 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f740c0b2880 msgr2=0x7f740c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f740c0b2880 0x7f740c0b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f741400b5c0 tx=0x7f741401a040 comp rx=0 tx=0).stop 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742410a000 msgr2=0x7f74241a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742410a000 0x7f74241a1d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f741c007d70 tx=0x7f741c00a340 comp rx=0 tx=0).stop 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 shutdown_connections 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f740c0b2880 0x7f740c0b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742410a000 0x7f74241a1d30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7424076650 0x7f742419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 --2- 172.21.15.39:0/3350593228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74241074f0 0x7f742419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:26.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 >> 172.21.15.39:0/3350593228 conn(0x7f74240fe890 msgr2=0x7f7424105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:26.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 shutdown_connections 2024-05-04T03:10:26.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:26.031+0000 7f7429557700 1 -- 172.21.15.39:0/3350593228 wait complete. 2024-05-04T03:10:26.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:26 smithi039 ceph-mon[29715]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:26.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:26 smithi066 ceph-mon[33264]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:26.829 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-04T03:10:26.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:26 smithi167 ceph-mon[33614]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:27.556 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:27 smithi039 ceph-mon[29715]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:27.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:27 smithi066 ceph-mon[33264]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:27.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:27 smithi167 ceph-mon[33614]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:28.652 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:28 smithi039 ceph-mon[29715]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:28.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:28 smithi066 ceph-mon[33264]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:28.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 -- 172.21.15.39:0/3377513825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c104380 msgr2=0x7f722c104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:28.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 --2- 172.21.15.39:0/3377513825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c104380 0x7f722c104790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f721c009990 tx=0x7f721c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:28.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 -- 172.21.15.39:0/3377513825 shutdown_connections 2024-05-04T03:10:28.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 --2- 172.21.15.39:0/3377513825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f722c104d20 0x7f722c10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:28.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 --2- 172.21.15.39:0/3377513825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f722c103250 0x7f722c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:28.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 --2- 172.21.15.39:0/3377513825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c104380 0x7f722c104790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:28.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 -- 172.21.15.39:0/3377513825 >> 172.21.15.39:0/3377513825 conn(0x7f722c0fe890 msgr2=0x7f722c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 -- 172.21.15.39:0/3377513825 shutdown_connections 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.904+0000 7f7233c14700 1 -- 172.21.15.39:0/3377513825 wait complete. 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f7233c14700 1 Processor -- start 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f7233c14700 1 -- start start 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f7233c14700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c103250 0x7f722c107010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f7233c14700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f722c104380 0x7f722c1056c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f7233c14700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f722c104d20 0x7f722c105c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f7233c14700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f722c10ec60 con 0x7f722c103250 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f7233c14700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f722c10eae0 con 0x7f722c104380 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f7233c14700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f722c10ede0 con 0x7f722c104d20 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f72319b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c103250 0x7f722c107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f72319b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c103250 0x7f722c107010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33438/0 (socket says 172.21.15.39:33438) 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f72311af700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f722c104380 0x7f722c1056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:28.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f72319b0700 1 -- 172.21.15.39:0/1482811919 learned_addr learned my addr 172.21.15.39:0/1482811919 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:28.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.905+0000 7f72321b1700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f722c104d20 0x7f722c105c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f72319b0700 1 -- 172.21.15.39:0/1482811919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f722c104d20 msgr2=0x7f722c105c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f72319b0700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f722c104d20 0x7f722c105c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f72319b0700 1 -- 172.21.15.39:0/1482811919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f722c104380 msgr2=0x7f722c1056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f72319b0700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f722c104380 0x7f722c1056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f72319b0700 1 -- 172.21.15.39:0/1482811919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c106360 con 0x7f722c103250 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f72311af700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f722c104380 0x7f722c1056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f72319b0700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c103250 0x7f722c107010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f722800b4d0 tx=0x7f722800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f7222ffd700 1 -- 172.21.15.39:0/1482811919 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228015020 con 0x7f722c103250 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f7222ffd700 1 -- 172.21.15.39:0/1482811919 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7228011cc0 con 0x7f722c103250 2024-05-04T03:10:28.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722c1a4a90 con 0x7f722c103250 2024-05-04T03:10:28.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f7222ffd700 1 -- 172.21.15.39:0/1482811919 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228010b80 con 0x7f722c103250 2024-05-04T03:10:28.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.906+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f722c1a4ff0 con 0x7f722c103250 2024-05-04T03:10:28.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.907+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722c0512c0 con 0x7f722c103250 2024-05-04T03:10:28.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.910+0000 7f7222ffd700 1 -- 172.21.15.39:0/1482811919 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7228018020 con 0x7f722c103250 2024-05-04T03:10:28.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.910+0000 7f7222ffd700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72180b2900 0x7f72180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:28.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.910+0000 7f72311af700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72180b2900 0x7f72180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:28.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.911+0000 7f7222ffd700 1 -- 172.21.15.39:0/1482811919 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f72280e1dc0 con 0x7f722c103250 2024-05-04T03:10:28.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.911+0000 7f72311af700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72180b2900 0x7f72180b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f721c006010 tx=0x7f721c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:28.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:28.914+0000 7f7222ffd700 1 -- 172.21.15.39:0/1482811919 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f722801f020 con 0x7f722c103250 2024-05-04T03:10:28.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:28 smithi167 ceph-mon[33614]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:29.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.139+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f722c1a5280 con 0x7f72180b2900 2024-05-04T03:10:29.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.141+0000 7f7222ffd700 1 -- 172.21.15.39:0/1482811919 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f722c1a5280 con 0x7f72180b2900 2024-05-04T03:10:29.142 INFO:teuthology.orchestra.run.smithi039.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-04T03:10:29.142 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102QX4P0KGN 3726G No 4m ago Has a FileSystem, LVM detected 2024-05-04T03:10:29.142 INFO:teuthology.orchestra.run.smithi039.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-04T03:10:29.142 INFO:teuthology.orchestra.run.smithi039.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72180b2900 msgr2=0x7f72180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72180b2900 0x7f72180b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f721c006010 tx=0x7f721c01a040 comp rx=0 tx=0).stop 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c103250 msgr2=0x7f722c107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c103250 0x7f722c107010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f722800b4d0 tx=0x7f722800b7e0 comp rx=0 tx=0).stop 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 shutdown_connections 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f72180b2900 0x7f72180b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722c103250 0x7f722c107010 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f722c104d20 0x7f722c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 --2- 172.21.15.39:0/1482811919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f722c104380 0x7f722c1056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 >> 172.21.15.39:0/1482811919 conn(0x7f722c0fe890 msgr2=0x7f722c1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:29.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 shutdown_connections 2024-05-04T03:10:29.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:29.143+0000 7f7233c14700 1 -- 172.21.15.39:0/1482811919 wait complete. 2024-05-04T03:10:29.860 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-04T03:10:29.869 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T03:10:29.869 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-04T03:10:30.121 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:30.500 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:30 smithi039 ceph-mon[29715]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:30.500 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:30 smithi039 ceph-mon[29715]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:30.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:30 smithi066 ceph-mon[33264]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:30 smithi066 ceph-mon[33264]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:30.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:30 smithi167 ceph-mon[33614]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:30.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:30 smithi167 ceph-mon[33614]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 -- 172.21.15.39:0/269252003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 msgr2=0x7f2a5c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/269252003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 0x7f2a5c104750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009990 tx=0x7f2a54009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 -- 172.21.15.39:0/269252003 shutdown_connections 2024-05-04T03:10:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/269252003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 0x7f2a5c104750 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/269252003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c105e10 0x7f2a5c10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/269252003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a5c105470 0x7f2a5c105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 -- 172.21.15.39:0/269252003 >> 172.21.15.39:0/269252003 conn(0x7f2a5c0ff9c0 msgr2=0x7f2a5c101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 -- 172.21.15.39:0/269252003 shutdown_connections 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.966+0000 7f2a63e5e700 1 -- 172.21.15.39:0/269252003 wait complete. 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a63e5e700 1 Processor -- start 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a63e5e700 1 -- start start 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a63e5e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 0x7f2a5c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a63e5e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c105470 0x7f2a5c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a63e5e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a5c105e10 0x7f2a5c1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a63e5e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10fc80 con 0x7f2a5c104340 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a63e5e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10fb00 con 0x7f2a5c105470 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a63e5e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10fe00 con 0x7f2a5c105e10 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a61bfa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 0x7f2a5c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a61bfa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 0x7f2a5c19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33470/0 (socket says 172.21.15.39:33470) 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.967+0000 7f2a61bfa700 1 -- 172.21.15.39:0/4128365157 learned_addr learned my addr 172.21.15.39:0/4128365157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a613f9700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c105470 0x7f2a5c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a623fb700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a5c105e10 0x7f2a5c1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a61bfa700 1 -- 172.21.15.39:0/4128365157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a5c105e10 msgr2=0x7f2a5c1a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a61bfa700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a5c105e10 0x7f2a5c1a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a61bfa700 1 -- 172.21.15.39:0/4128365157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c105470 msgr2=0x7f2a5c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a61bfa700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c105470 0x7f2a5c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a61bfa700 1 -- 172.21.15.39:0/4128365157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c1a2340 con 0x7f2a5c104340 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a613f9700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c105470 0x7f2a5c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a623fb700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a5c105e10 0x7f2a5c1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a61bfa700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 0x7f2a5c19a8e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a54006010 tx=0x7f2a54009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a5affd700 1 -- 172.21.15.39:0/4128365157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54026070 con 0x7f2a5c104340 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c1a25d0 con 0x7f2a5c104340 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a5affd700 1 -- 172.21.15.39:0/4128365157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a540040a0 con 0x7f2a5c104340 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a5affd700 1 -- 172.21.15.39:0/4128365157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400fca0 con 0x7f2a5c104340 2024-05-04T03:10:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.968+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c1a2a20 con 0x7f2a5c104340 2024-05-04T03:10:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.969+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c0512c0 con 0x7f2a5c104340 2024-05-04T03:10:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.971+0000 7f2a5affd700 1 -- 172.21.15.39:0/4128365157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2a54029070 con 0x7f2a5c104340 2024-05-04T03:10:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.972+0000 7f2a5affd700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a440b2880 0x7f2a440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.972+0000 7f2a613f9700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a440b2880 0x7f2a440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.972+0000 7f2a5affd700 1 -- 172.21.15.39:0/4128365157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2a540e3260 con 0x7f2a5c104340 2024-05-04T03:10:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.973+0000 7f2a613f9700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a440b2880 0x7f2a440b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c19b990 tx=0x7f2a50009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:31.975+0000 7f2a5affd700 1 -- 172.21.15.39:0/4128365157 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a54013070 con 0x7f2a5c104340 2024-05-04T03:10:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.238+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2a5c0643d0 con 0x7f2a5c104340 2024-05-04T03:10:32.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.239+0000 7f2a5affd700 1 -- 172.21.15.39:0/4128365157 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3090 (secure 0 0 0) 0x7f2a540ab020 con 0x7f2a5c104340 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:e27 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:legacy client fscid: 1 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem 'cephfs' (1) 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:fs_name cephfs 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 27 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-05-04T03:07:39.087961+0000 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-05-04T03:08:01.344495+0000 2024-05-04T03:10:32.240 INFO:teuthology.orchestra.run.smithi039.stdout:tableserver 0 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:root 0 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:session_timeout 300 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:session_autoclose 300 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:max_file_size 1099511627776 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:max_xattr_size 65536 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:required_client_features {} 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure 0 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure_osd_epoch 0 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:max_mds 5 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:in 0,1,2,3,4 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:up {0=24506,1=24490,2=24472,3=24466,4=24455} 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:failed 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:damaged 2024-05-04T03:10:32.241 INFO:teuthology.orchestra.run.smithi039.stdout:stopped 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:data_pools [3] 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:metadata_pool 2 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:inline_data disabled 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:balancer 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:bal_rank_mask -1 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:standby_count_wanted 0 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:qdb_cluster leader: 24506 members: 24455,24490,24466,24472,24506 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b{0:24506} state up:active seq 6 addr [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.k{0:24428} state up:standby-replay seq 1 addr [v2:172.21.15.66:6832/462065299,v1:172.21.15.66:6833/462065299] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.i{1:24490} state up:active seq 13 addr [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.l{1:24427} state up:standby-replay seq 1 addr [v2:172.21.15.167:6832/1077677237,v1:172.21.15.167:6833/1077677237] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c{2:24472} state up:active seq 14 addr [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a{2:14634} state up:standby-replay seq 1 addr [v2:172.21.15.39:6840/1534369514,v1:172.21.15.39:6841/1534369514] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.242 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.f{3:24466} state up:active seq 15 addr [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.g{3:14622} state up:standby-replay seq 1 addr [v2:172.21.15.39:6838/4175892611,v1:172.21.15.39:6839/4175892611] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.e{4:24455} state up:active seq 16 addr [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.h{4:14607} state up:standby-replay seq 1 addr [v2:172.21.15.66:6836/2860496376,v1:172.21.15.66:6837/2860496376] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout:Standby daemons: 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.j{-1:14562} state up:standby seq 1 addr [v2:172.21.15.39:6834/3587796449,v1:172.21.15.39:6835/3587796449] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.243 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d{-1:14574} state up:standby seq 1 addr [v2:172.21.15.39:6836/2949312289,v1:172.21.15.39:6837/2949312289] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a440b2880 msgr2=0x7f2a440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a440b2880 0x7f2a440b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c19b990 tx=0x7f2a50009040 comp rx=0 tx=0).stop 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 msgr2=0x7f2a5c19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 0x7f2a5c19a8e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a54006010 tx=0x7f2a54009510 comp rx=0 tx=0).stop 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 shutdown_connections 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a440b2880 0x7f2a440b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a5c104340 0x7f2a5c19a8e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a5c105e10 0x7f2a5c1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 --2- 172.21.15.39:0/4128365157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a5c105470 0x7f2a5c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 >> 172.21.15.39:0/4128365157 conn(0x7f2a5c0ff9c0 msgr2=0x7f2a5c1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 shutdown_connections 2024-05-04T03:10:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:32.242+0000 7f2a63e5e700 1 -- 172.21.15.39:0/4128365157 wait complete. 2024-05-04T03:10:32.245 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-04T03:10:32.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:32 smithi039 ceph-mon[29715]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:32.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:32 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4128365157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-04T03:10:32.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:32 smithi066 ceph-mon[33264]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:32.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:32 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4128365157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-04T03:10:32.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:32 smithi167 ceph-mon[33614]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:32.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:32 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4128365157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-04T03:10:33.046 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-04T03:10:33.301 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:34.784 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:34 smithi039 ceph-mon[29715]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:34.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:34 smithi066 ceph-mon[33264]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:34.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:34 smithi167 ceph-mon[33614]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 -- 172.21.15.39:0/1504047674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4104750 msgr2=0x7fb5c4104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 --2- 172.21.15.39:0/1504047674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4104750 0x7fb5c4104b80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009990 tx=0x7fb5b4009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 -- 172.21.15.39:0/1504047674 shutdown_connections 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 --2- 172.21.15.39:0/1504047674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4104750 0x7fb5c4104b80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 --2- 172.21.15.39:0/1504047674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c4103db0 0x7fb5c410b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 --2- 172.21.15.39:0/1504047674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c4103460 0x7fb5c4103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 -- 172.21.15.39:0/1504047674 >> 172.21.15.39:0/1504047674 conn(0x7fb5c40fe020 msgr2=0x7fb5c41004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 -- 172.21.15.39:0/1504047674 shutdown_connections 2024-05-04T03:10:35.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.049+0000 7fb5c9989700 1 -- 172.21.15.39:0/1504047674 wait complete. 2024-05-04T03:10:35.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c9989700 1 Processor -- start 2024-05-04T03:10:35.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c9989700 1 -- start start 2024-05-04T03:10:35.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c9989700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c4103460 0x7fb5c4107100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:35.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c9989700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4103db0 0x7fb5c41057b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:35.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c9989700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c4104750 0x7fb5c4105cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:35.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c9989700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410eb70 con 0x7fb5c4103db0 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c9989700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410e9f0 con 0x7fb5c4103460 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c9989700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410ecf0 con 0x7fb5c4104750 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4103db0 0x7fb5c41057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c4103460 0x7fb5c4107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4103db0 0x7fb5c41057b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33492/0 (socket says 172.21.15.39:33492) 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c4103460 0x7fb5c4107100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:49682/0 (socket says 172.21.15.39:49682) 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.050+0000 7fb5c27fc700 1 -- 172.21.15.39:0/3861711739 learned_addr learned my addr 172.21.15.39:0/3861711739 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c37fe700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c4104750 0x7fb5c4105cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c2ffd700 1 -- 172.21.15.39:0/3861711739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c4104750 msgr2=0x7fb5c4105cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c2ffd700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c4104750 0x7fb5c4105cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c2ffd700 1 -- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4103db0 msgr2=0x7fb5c41057b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c2ffd700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4103db0 0x7fb5c41057b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c2ffd700 1 -- 172.21.15.39:0/3861711739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c41063c0 con 0x7fb5c4103460 2024-05-04T03:10:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c37fe700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c4104750 0x7fb5c4105cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:35.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c27fc700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4103db0 0x7fb5c41057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:10:35.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c2ffd700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c4103460 0x7fb5c4107100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:35.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c8987700 1 -- 172.21.15.39:0/3861711739 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4027070 con 0x7fb5c4103460 2024-05-04T03:10:35.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41a4930 con 0x7fb5c4103460 2024-05-04T03:10:35.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c8987700 1 -- 172.21.15.39:0/3861711739 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5b4004020 con 0x7fb5c4103460 2024-05-04T03:10:35.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41a4cd0 con 0x7fb5c4103460 2024-05-04T03:10:35.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.051+0000 7fb5c8987700 1 -- 172.21.15.39:0/3861711739 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4022c10 con 0x7fb5c4103460 2024-05-04T03:10:35.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.052+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c40512c0 con 0x7fb5c4103460 2024-05-04T03:10:35.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.054+0000 7fb5c8987700 1 -- 172.21.15.39:0/3861711739 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb5b402a070 con 0x7fb5c4103460 2024-05-04T03:10:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.055+0000 7fb5c8987700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5ac0b2880 0x7fb5ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.055+0000 7fb5c27fc700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5ac0b2880 0x7fb5ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.055+0000 7fb5c8987700 1 -- 172.21.15.39:0/3861711739 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb5b40e3a10 con 0x7fb5c4103460 2024-05-04T03:10:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.055+0000 7fb5c27fc700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5ac0b2880 0x7fb5ac0b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4106ab0 tx=0x7fb5b800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:35.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.059+0000 7fb5c8987700 1 -- 172.21.15.39:0/3861711739 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5b4031080 con 0x7fb5c4103460 2024-05-04T03:10:35.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.275+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb5c41a5060 con 0x7fb5c4103460 2024-05-04T03:10:35.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.276+0000 7fb5c8987700 1 -- 172.21.15.39:0/3861711739 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) ==== 56+0+5262 (secure 0 0 0) 0x7fb5b4013070 con 0x7fb5c4103460 2024-05-04T03:10:35.277 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 80 2024-05-04T03:10:35.277 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-05-04T02:59:20.440407+0000 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-05-04T03:08:16.237923+0000 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 26 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client luminous 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release squid 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-04T03:10:35.278 INFO:teuthology.orchestra.run.smithi039.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 12 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6802/837850170,v1:172.21.15.39:6803/837850170] [v2:172.21.15.39:6804/837850170,v1:172.21.15.39:6805/837850170] exists,up c9940446-c2ae-4eb5-b6fe-313baf8466d2 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 13 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/2695940383,v1:172.21.15.66:6801/2695940383] [v2:172.21.15.66:6802/2695940383,v1:172.21.15.66:6803/2695940383] exists,up 6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 18 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/2785227057,v1:172.21.15.167:6801/2785227057] [v2:172.21.15.167:6802/2785227057,v1:172.21.15.167:6803/2785227057] exists,up 339eb4b8-1e7f-490b-9d54-63886da97018 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 up in weight 1 up_from 25 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6810/747919958,v1:172.21.15.39:6811/747919958] [v2:172.21.15.39:6812/747919958,v1:172.21.15.39:6813/747919958] exists,up 118295e4-04a9-4997-bfa5-a589e4849058 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 up in weight 1 up_from 31 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/1509400752,v1:172.21.15.66:6809/1509400752] [v2:172.21.15.66:6810/1509400752,v1:172.21.15.66:6811/1509400752] exists,up 14d54715-138c-4f72-837d-ecea4f0ef3e8 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 up in weight 1 up_from 36 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/3200709973,v1:172.21.15.167:6809/3200709973] [v2:172.21.15.167:6810/3200709973,v1:172.21.15.167:6811/3200709973] exists,up 3aa01070-d35f-4964-9ccd-f956e1c9e03c 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 up in weight 1 up_from 42 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6818/519893434,v1:172.21.15.39:6819/519893434] [v2:172.21.15.39:6820/519893434,v1:172.21.15.39:6821/519893434] exists,up e73a0efc-8e1e-42dc-9c24-f45d08af466c 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 up in weight 1 up_from 47 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/931306967,v1:172.21.15.66:6817/931306967] [v2:172.21.15.66:6818/931306967,v1:172.21.15.66:6819/931306967] exists,up 56289675-217d-4d0e-9875-fb5e48254898 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 up in weight 1 up_from 53 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/948600298,v1:172.21.15.167:6817/948600298] [v2:172.21.15.167:6818/948600298,v1:172.21.15.167:6819/948600298] exists,up 755cbc15-ebe2-408f-8009-518e6f31b975 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 up in weight 1 up_from 59 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6826/1483154283,v1:172.21.15.39:6827/1483154283] [v2:172.21.15.39:6828/1483154283,v1:172.21.15.39:6829/1483154283] exists,up 956d274d-166b-4474-92d3-d31bda097f21 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 up in weight 1 up_from 64 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/3028550986,v1:172.21.15.66:6825/3028550986] [v2:172.21.15.66:6826/3028550986,v1:172.21.15.66:6827/3028550986] exists,up 2c1ff6cd-7475-4b79-83bb-1f671fe132cf 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 up in weight 1 up_from 70 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/3994242029,v1:172.21.15.167:6825/3994242029] [v2:172.21.15.167:6826/3994242029,v1:172.21.15.167:6827/3994242029] exists,up d8940073-e9c7-4d49-af48-0a4c9a4b6f6e 2024-05-04T03:10:35.279 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.0 [8,4] 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.a [2,11] 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.13 [2,11] 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.16 [2,4] 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.6 [3,6] 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.1b [4,8,5,9] 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.24 [4,9] 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.36 [4,9] 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/3594679658 expires 2024-05-05T03:00:14.875322+0000 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/3124453164 expires 2024-05-05T02:59:47.622763+0000 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/3124453164 expires 2024-05-05T02:59:47.622763+0000 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/3594679658 expires 2024-05-05T03:00:14.875322+0000 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/163870032 expires 2024-05-05T03:00:14.875322+0000 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/518372945 expires 2024-05-05T02:59:47.622763+0000 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1251709695 expires 2024-05-05T02:59:47.622763+0000 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/402196413 expires 2024-05-05T03:00:14.875322+0000 2024-05-04T03:10:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1831352319 expires 2024-05-05T02:59:47.622763+0000 2024-05-04T03:10:35.281 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1379489577 expires 2024-05-05T03:00:14.875322+0000 2024-05-04T03:10:35.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5ac0b2880 msgr2=0x7fb5ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:35.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5ac0b2880 0x7fb5ac0b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4106ab0 tx=0x7fb5b800b500 comp rx=0 tx=0).stop 2024-05-04T03:10:35.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c4103460 msgr2=0x7fb5c4107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:35.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c4103460 0x7fb5c4107100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b40057c0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 shutdown_connections 2024-05-04T03:10:35.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb5ac0b2880 0x7fb5ac0b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5c4104750 0x7fb5c4105cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c4103460 0x7fb5c4107100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 --2- 172.21.15.39:0/3861711739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5c4103db0 0x7fb5c41057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:35.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.278+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 >> 172.21.15.39:0/3861711739 conn(0x7fb5c40fe020 msgr2=0x7fb5c40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:35.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.279+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 shutdown_connections 2024-05-04T03:10:35.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:35.279+0000 7fb5c9989700 1 -- 172.21.15.39:0/3861711739 wait complete. 2024-05-04T03:10:35.741 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:35 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3861711739' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-04T03:10:35.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:35 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3861711739' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-04T03:10:35.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:35 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3861711739' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-04T03:10:36.185 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-04T03:10:36.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:10:36.453 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:36.779 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:36 smithi039 ceph-mon[29715]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:36.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:36 smithi066 ceph-mon[33264]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:36.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:36 smithi167 ceph-mon[33614]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 -- 172.21.15.39:0/53967471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37781034c0 msgr2=0x7f37781038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 --2- 172.21.15.39:0/53967471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37781034c0 0x7f37781038d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f37680099c0 tx=0x7f3768009cd0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 -- 172.21.15.39:0/53967471 shutdown_connections 2024-05-04T03:10:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 --2- 172.21.15.39:0/53967471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37781034c0 0x7f37781038d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 --2- 172.21.15.39:0/53967471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778108740 0x7f37781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 --2- 172.21.15.39:0/53967471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3778107df0 0x7f3778108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 -- 172.21.15.39:0/53967471 >> 172.21.15.39:0/53967471 conn(0x7f37780fedf0 msgr2=0x7f3778101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 -- 172.21.15.39:0/53967471 shutdown_connections 2024-05-04T03:10:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.249+0000 7f377d7bf700 1 -- 172.21.15.39:0/53967471 wait complete. 2024-05-04T03:10:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.250+0000 7f377d7bf700 1 Processor -- start 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.250+0000 7f377d7bf700 1 -- start start 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.250+0000 7f377d7bf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37781034c0 0x7f377819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.250+0000 7f377d7bf700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3778107df0 0x7f377819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.250+0000 7f377d7bf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3778108740 0x7f3778199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.250+0000 7f377d7bf700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3778113220 con 0x7f3778107df0 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f377d7bf700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37781130a0 con 0x7f3778108740 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f377d7bf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37781133a0 con 0x7f37781034c0 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3778107df0 0x7f377819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3778107df0 0x7f377819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50186/0 (socket says 172.21.15.39:50186) 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 -- 172.21.15.39:0/3200044690 learned_addr learned my addr 172.21.15.39:0/3200044690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f37777fe700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3778108740 0x7f3778199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f3776ffd700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37781034c0 0x7f377819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 -- 172.21.15.39:0/3200044690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37781034c0 msgr2=0x7f377819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37781034c0 0x7f377819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 -- 172.21.15.39:0/3200044690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3778108740 msgr2=0x7f3778199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3778108740 0x7f3778199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 -- 172.21.15.39:0/3200044690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377819a0c0 con 0x7f3778107df0 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f3776ffd700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37781034c0 0x7f377819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.251+0000 7f376ffff700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3778107df0 0x7f377819fda0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f376000d980 tx=0x7f376000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.252+0000 7f3774ff9700 1 -- 172.21.15.39:0/3200044690 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760017070 con 0x7f3778107df0 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.252+0000 7f3774ff9700 1 -- 172.21.15.39:0/3200044690 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3760009dc0 con 0x7f3778107df0 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.252+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377819a3b0 con 0x7f3778107df0 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.252+0000 7f3774ff9700 1 -- 172.21.15.39:0/3200044690 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376000f3c0 con 0x7f3778107df0 2024-05-04T03:10:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.252+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37780759f0 con 0x7f3778107df0 2024-05-04T03:10:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.253+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3778104970 con 0x7f3778107df0 2024-05-04T03:10:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.254+0000 7f3774ff9700 1 -- 172.21.15.39:0/3200044690 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3760009930 con 0x7f3778107df0 2024-05-04T03:10:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.255+0000 7f3774ff9700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37640b2880 0x7f37640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.255+0000 7f3776ffd700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37640b2880 0x7f37640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.255+0000 7f3774ff9700 1 -- 172.21.15.39:0/3200044690 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f37600e1d30 con 0x7f3778107df0 2024-05-04T03:10:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.256+0000 7f3776ffd700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37640b2880 0x7f37640b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3768009990 tx=0x7f3768005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.260+0000 7f3774ff9700 1 -- 172.21.15.39:0/3200044690 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37600a9a60 con 0x7f3778107df0 2024-05-04T03:10:38.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.520+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f377819afb0 con 0x7f3778107df0 2024-05-04T03:10:38.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:38 smithi039 ceph-mon[29715]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:38.522 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:10:38.522 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:08:01.344495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24455,24490,24466,24472,24506]},"id":1}]} 2024-05-04T03:10:38.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.521+0000 7f3774ff9700 1 -- 172.21.15.39:0/3200044690 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f37600a9360 con 0x7f3778107df0 2024-05-04T03:10:38.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37640b2880 msgr2=0x7f37640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:38.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37640b2880 0x7f37640b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3768009990 tx=0x7f3768005940 comp rx=0 tx=0).stop 2024-05-04T03:10:38.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3778107df0 msgr2=0x7f377819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:38.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3778107df0 0x7f377819fda0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f376000d980 tx=0x7f376000dc90 comp rx=0 tx=0).stop 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 shutdown_connections 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f37640b2880 0x7f37640b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3778108740 0x7f3778199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37781034c0 0x7f377819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 --2- 172.21.15.39:0/3200044690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3778107df0 0x7f377819fda0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 >> 172.21.15.39:0/3200044690 conn(0x7f37780fedf0 msgr2=0x7f3778101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 shutdown_connections 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:38.523+0000 7f377d7bf700 1 -- 172.21.15.39:0/3200044690 wait complete. 2024-05-04T03:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-04T03:10:38.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:38 smithi066 ceph-mon[33264]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:38.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:38 smithi167 ceph-mon[33614]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:39.175 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:10:39.458 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:39.753 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:39 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3200044690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:10:39.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:39 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3200044690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:10:39.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:39 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3200044690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:10:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:40 smithi039 ceph-mon[29715]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:40.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:40 smithi066 ceph-mon[33264]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:40.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:40 smithi167 ceph-mon[33614]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 -- 172.21.15.39:0/2086903673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 msgr2=0x7fc4e8102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 --2- 172.21.15.39:0/2086903673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 0x7fc4e8102dd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009990 tx=0x7fc4d8009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 -- 172.21.15.39:0/2086903673 shutdown_connections 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 --2- 172.21.15.39:0/2086903673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e8103310 0x7fc4e810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 --2- 172.21.15.39:0/2086903673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4e81089f0 0x7fc4e8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 --2- 172.21.15.39:0/2086903673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 0x7fc4e8102dd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 -- 172.21.15.39:0/2086903673 >> 172.21.15.39:0/2086903673 conn(0x7fc4e80fe4e0 msgr2=0x7fc4e8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 -- 172.21.15.39:0/2086903673 shutdown_connections 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.365+0000 7fc4efb49700 1 -- 172.21.15.39:0/2086903673 wait complete. 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4efb49700 1 Processor -- start 2024-05-04T03:10:41.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4efb49700 1 -- start start 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4efb49700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 0x7fc4e819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4efb49700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e8103310 0x7fc4e819c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4efb49700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4e81089f0 0x7fc4e81a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4efb49700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e81129a0 con 0x7fc4e81029c0 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4efb49700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e8112820 con 0x7fc4e8103310 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4efb49700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e8112b20 con 0x7fc4e81089f0 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4ed8e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 0x7fc4e819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4ed8e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 0x7fc4e819bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50202/0 (socket says 172.21.15.39:50202) 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.366+0000 7fc4ed8e5700 1 -- 172.21.15.39:0/4209813771 learned_addr learned my addr 172.21.15.39:0/4209813771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ed0e4700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e8103310 0x7fc4e819c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ee0e6700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4e81089f0 0x7fc4e81a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ed8e5700 1 -- 172.21.15.39:0/4209813771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4e81089f0 msgr2=0x7fc4e81a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ed8e5700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4e81089f0 0x7fc4e81a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ed8e5700 1 -- 172.21.15.39:0/4209813771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e8103310 msgr2=0x7fc4e819c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:41.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ed8e5700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e8103310 0x7fc4e819c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ed8e5700 1 -- 172.21.15.39:0/4209813771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e819c910 con 0x7fc4e81029c0 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ee0e6700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4e81089f0 0x7fc4e81a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4ed8e5700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 0x7fc4e819bb20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400e970 tx=0x7fc4e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4deffd700 1 -- 172.21.15.39:0/4209813771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400cbd0 con 0x7fc4e81029c0 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4deffd700 1 -- 172.21.15.39:0/4209813771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4e400cd70 con 0x7fc4e81029c0 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e81a7200 con 0x7fc4e81029c0 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4deffd700 1 -- 172.21.15.39:0/4209813771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4005570 con 0x7fc4e81029c0 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.367+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e81a76c0 con 0x7fc4e81029c0 2024-05-04T03:10:41.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.368+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e80512c0 con 0x7fc4e81029c0 2024-05-04T03:10:41.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.370+0000 7fc4deffd700 1 -- 172.21.15.39:0/4209813771 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc4e4004750 con 0x7fc4e81029c0 2024-05-04T03:10:41.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.371+0000 7fc4deffd700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc4d40b2880 0x7fc4d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:41.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.371+0000 7fc4ed0e4700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc4d40b2880 0x7fc4d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:41.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.371+0000 7fc4deffd700 1 -- 172.21.15.39:0/4209813771 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc4e40e25c0 con 0x7fc4e81029c0 2024-05-04T03:10:41.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.371+0000 7fc4ed0e4700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc4d40b2880 0x7fc4d40b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800b2e0 tx=0x7fc4d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:41.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.375+0000 7fc4deffd700 1 -- 172.21.15.39:0/4209813771 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc4e4054df0 con 0x7fc4e81029c0 2024-05-04T03:10:41.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.640+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4e8105a90 con 0x7fc4e81029c0 2024-05-04T03:10:41.641 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:41 smithi039 ceph-mon[29715]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:41.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.641+0000 7fc4deffd700 1 -- 172.21.15.39:0/4209813771 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7fc4e40aa2f0 con 0x7fc4e81029c0 2024-05-04T03:10:41.642 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:10:41.642 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:08:01.344495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24455,24490,24466,24472,24506]},"id":1}]} 2024-05-04T03:10:41.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.643+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc4d40b2880 msgr2=0x7fc4d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.643+0000 7fc4efb49700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc4d40b2880 0x7fc4d40b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800b2e0 tx=0x7fc4d8005fb0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.643+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 msgr2=0x7fc4e819bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.643+0000 7fc4efb49700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 0x7fc4e819bb20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400e970 tx=0x7fc4e400ec80 comp rx=0 tx=0).stop 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.644+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 shutdown_connections 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.644+0000 7fc4efb49700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc4d40b2880 0x7fc4d40b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.644+0000 7fc4efb49700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e81029c0 0x7fc4e819bb20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.644+0000 7fc4efb49700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4e81089f0 0x7fc4e81a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.644+0000 7fc4efb49700 1 --2- 172.21.15.39:0/4209813771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4e8103310 0x7fc4e819c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.644+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 >> 172.21.15.39:0/4209813771 conn(0x7fc4e80fe4e0 msgr2=0x7fc4e8100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.644+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 shutdown_connections 2024-05-04T03:10:41.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:41.644+0000 7fc4efb49700 1 -- 172.21.15.39:0/4209813771 wait complete. 2024-05-04T03:10:41.646 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-05-04T03:10:41.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:41 smithi066 ceph-mon[33264]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:41.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:41 smithi167 ceph-mon[33614]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:42.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:10:42.631 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:42.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:42 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4209813771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:10:42.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:42 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:42.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:42 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4209813771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:10:42.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:42 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:42.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:42 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4209813771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:10:42.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:42 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:44.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:44.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:44 smithi066 ceph-mon[33264]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:44.318 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:44 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:44.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:44.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:44 smithi039 ceph-mon[29715]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:44.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:44 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:44.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 -- 172.21.15.39:0/3892635207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac1034e0 msgr2=0x7f22ac1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:44.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/3892635207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac1034e0 0x7f22ac1038f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f229c0099c0 tx=0x7f229c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 -- 172.21.15.39:0/3892635207 shutdown_connections 2024-05-04T03:10:44.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/3892635207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac1034e0 0x7f22ac1038f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/3892635207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22ac108760 0x7f22ac1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/3892635207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22ac107e10 0x7f22ac108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 -- 172.21.15.39:0/3892635207 >> 172.21.15.39:0/3892635207 conn(0x7f22ac0fef30 msgr2=0x7f22ac101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:44.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 -- 172.21.15.39:0/3892635207 shutdown_connections 2024-05-04T03:10:44.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.349+0000 7f22b1f6e700 1 -- 172.21.15.39:0/3892635207 wait complete. 2024-05-04T03:10:44.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22b1f6e700 1 Processor -- start 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22b1f6e700 1 -- start start 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22b1f6e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22ac1034e0 0x7f22ac19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22b1f6e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac107e10 0x7f22ac19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22b1f6e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22ac108760 0x7f22ac19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22b1f6e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac113330 con 0x7f22ac107e10 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22b1f6e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1131b0 con 0x7f22ac108760 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22b1f6e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1134b0 con 0x7f22ac1034e0 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.350+0000 7f22aaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac107e10 0x7f22ac19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22aaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac107e10 0x7f22ac19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50210/0 (socket says 172.21.15.39:50210) 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22aaffd700 1 -- 172.21.15.39:0/942498434 learned_addr learned my addr 172.21.15.39:0/942498434 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22ab7fe700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22ac1034e0 0x7f22ac19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22abfff700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22ac108760 0x7f22ac19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22aaffd700 1 -- 172.21.15.39:0/942498434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22ac1034e0 msgr2=0x7f22ac19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22aaffd700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22ac1034e0 0x7f22ac19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22aaffd700 1 -- 172.21.15.39:0/942498434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22ac108760 msgr2=0x7f22ac19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22aaffd700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22ac108760 0x7f22ac19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22aaffd700 1 -- 172.21.15.39:0/942498434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac19fd20 con 0x7f22ac107e10 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22aaffd700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac107e10 0x7f22ac19b3f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f22a000b500 tx=0x7f22a000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22a8ff9700 1 -- 172.21.15.39:0/942498434 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0015020 con 0x7f22ac107e10 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22a8ff9700 1 -- 172.21.15.39:0/942498434 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22a0011cc0 con 0x7f22ac107e10 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ac1a0010 con 0x7f22ac107e10 2024-05-04T03:10:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22a8ff9700 1 -- 172.21.15.39:0/942498434 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0010c80 con 0x7f22ac107e10 2024-05-04T03:10:44.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.351+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ac1a7930 con 0x7f22ac107e10 2024-05-04T03:10:44.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.352+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ac0512c0 con 0x7f22ac107e10 2024-05-04T03:10:44.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.354+0000 7f22a8ff9700 1 -- 172.21.15.39:0/942498434 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f22a0018020 con 0x7f22ac107e10 2024-05-04T03:10:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.355+0000 7f22a8ff9700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22940b2880 0x7f22940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.355+0000 7f22ab7fe700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22940b2880 0x7f22940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.355+0000 7f22a8ff9700 1 -- 172.21.15.39:0/942498434 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f22a00e2000 con 0x7f22ac107e10 2024-05-04T03:10:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.355+0000 7f22ab7fe700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22940b2880 0x7f22940b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f229c000c00 tx=0x7f229c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:44.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.358+0000 7f22a8ff9700 1 -- 172.21.15.39:0/942498434 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22a001f020 con 0x7f22ac107e10 2024-05-04T03:10:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:44 smithi167 ceph-mon[33614]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:44 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:44.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.604+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22ac19c1b0 con 0x7f22ac107e10 2024-05-04T03:10:44.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.607+0000 7f22a8ff9700 1 -- 172.21.15.39:0/942498434 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22541 (secure 0 0 0) 0x7f22a00a9d30 con 0x7f22ac107e10 2024-05-04T03:10:44.608 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:10:44.608 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:08:16.237923+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:10:44.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22940b2880 msgr2=0x7f22940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:44.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22940b2880 0x7f22940b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f229c000c00 tx=0x7f229c005fb0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac107e10 msgr2=0x7f22ac19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:44.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac107e10 0x7f22ac19b3f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f22a000b500 tx=0x7f22a000b8c0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 shutdown_connections 2024-05-04T03:10:44.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f22940b2880 0x7f22940b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22ac108760 0x7f22ac19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22ac1034e0 0x7f22ac19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 --2- 172.21.15.39:0/942498434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ac107e10 0x7f22ac19b3f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:44.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 >> 172.21.15.39:0/942498434 conn(0x7f22ac0fef30 msgr2=0x7f22ac1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:44.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 shutdown_connections 2024-05-04T03:10:44.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:44.609+0000 7f22b1f6e700 1 -- 172.21.15.39:0/942498434 wait complete. 2024-05-04T03:10:45.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-04T03:07:39.087961+0000', 'modified': '2024-05-04T03:08:01.344495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24506, 'mds_1': 24490, 'mds_2': 24472, 'mds_3': 24466, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14607': {'gid': 14607, 'name': 'h', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6837/2860496376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 2860496376}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 2860496376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6839/4175892611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 4175892611}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 4175892611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14634': {'gid': 14634, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6841/1534369514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 1534369514}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 1534369514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/1077677237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 1077677237}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 1077677237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/462065299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 462065299}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 462065299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.66:6835/3463909305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 3463909305}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 3463909305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.167:6835/2699315372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 2699315372}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 2699315372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.167:6837/2270685849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2270685849}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2270685849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.167:6839/975309166', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 975309166}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 975309166}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.66:6839/4050747247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 4050747247}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 4050747247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24455, 24490, 24466, 24472, 24506]} 2024-05-04T03:10:45.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-04T03:10:45.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.e status 2024-05-04T03:10:45.638 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:45.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:45.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:45 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:45.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:45 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/942498434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:10:45.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:45.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:45 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:45.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:45 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/942498434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:10:45.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:45.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:45 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:45.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:45 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/942498434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:10:46.805 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:46 smithi039 ceph-mon[29715]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:46.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:46.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:46.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:46.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:46 smithi066 ceph-mon[33264]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:46 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:46 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:46 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:46.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:46 smithi167 ceph-mon[33614]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:46.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:46 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:46.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:46 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:46.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:46 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:10:47.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/869745280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc108740 msgr2=0x7ffbcc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/869745280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc108740 0x7ffbcc1100a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400b0d0 tx=0x7ffbc400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/869745280 shutdown_connections 2024-05-04T03:10:47.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/869745280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc108740 0x7ffbcc1100a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/869745280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbcc1034c0 0x7ffbcc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/869745280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc107df0 0x7ffbcc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/869745280 >> 172.21.15.39:0/869745280 conn(0x7ffbcc0fedf0 msgr2=0x7ffbcc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:47.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/869745280 shutdown_connections 2024-05-04T03:10:47.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.414+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/869745280 wait complete. 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbd0c6a700 1 Processor -- start 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbd0c6a700 1 -- start start 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbd0c6a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc1034c0 0x7ffbcc19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbd0c6a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbcc107df0 0x7ffbcc19c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbd0c6a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc108740 0x7ffbcc19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbd0c6a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc113270 con 0x7ffbcc1034c0 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbd0c6a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1130f0 con 0x7ffbcc108740 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbd0c6a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1133f0 con 0x7ffbcc107df0 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc1034c0 0x7ffbcc19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.415+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc1034c0 0x7ffbcc19bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43096/0 (socket says 172.21.15.39:43096) 2024-05-04T03:10:47.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbca59c700 1 -- 172.21.15.39:0/1549541507 learned_addr learned my addr 172.21.15.39:0/1549541507 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbcad9d700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc108740 0x7ffbcc19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbcc107df0 0x7ffbcc19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbca59c700 1 -- 172.21.15.39:0/1549541507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbcc107df0 msgr2=0x7ffbcc19c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbca59c700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbcc107df0 0x7ffbcc19c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbca59c700 1 -- 172.21.15.39:0/1549541507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc108740 msgr2=0x7ffbcc19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbca59c700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc108740 0x7ffbcc19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbca59c700 1 -- 172.21.15.39:0/1549541507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc1a6f20 con 0x7ffbcc1034c0 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbca59c700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc1034c0 0x7ffbcc19bbd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00d950 tx=0x7ffbbc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbc37fe700 1 -- 172.21.15.39:0/1549541507 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc016070 con 0x7ffbcc1034c0 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbc37fe700 1 -- 172.21.15.39:0/1549541507 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbbc009dd0 con 0x7ffbcc1034c0 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/1549541507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc1a7210 con 0x7ffbcc1034c0 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbc37fe700 1 -- 172.21.15.39:0/1549541507 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc00f3c0 con 0x7ffbcc1034c0 2024-05-04T03:10:47.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.416+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/1549541507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc1a77a0 con 0x7ffbcc1034c0 2024-05-04T03:10:47.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.419+0000 7ffbc37fe700 1 -- 172.21.15.39:0/1549541507 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffbbc019070 con 0x7ffbcc1034c0 2024-05-04T03:10:47.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.420+0000 7ffbc37fe700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ffbb80b2900 0x7ffbb80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.420+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ffbb80b2900 0x7ffbb80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.420+0000 7ffbc37fe700 1 -- 172.21.15.39:0/1549541507 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ffbbc0e1e40 con 0x7ffbcc1034c0 2024-05-04T03:10:47.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.420+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ffbb80b2900 0x7ffbb80b4e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005fd0 tx=0x7ffbb4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:47.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbdf67eb80 1 Processor -- start 2024-05-04T03:10:47.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbdf67eb80 1 -- start start 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe389da0 0x560dfe38a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe383ef0 0x560dfe384300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe3909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dfe2a6810 con 0x560dfe389da0 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560dfe245770 con 0x560dfe383ef0 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560dfe2a27d0 con 0x560dfe384840 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe389da0 0x560dfe38a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe389da0 0x560dfe38a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43098/0 (socket says 172.21.15.39:43098) 2024-05-04T03:10:47.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbca59c700 1 -- 172.21.15.39:0/2578123942 learned_addr learned my addr 172.21.15.39:0/2578123942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.421+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe383ef0 0x560dfe384300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbcad9d700 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe3909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbca59c700 1 -- 172.21.15.39:0/2578123942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 msgr2=0x560dfe3909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbca59c700 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe3909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbca59c700 1 -- 172.21.15.39:0/2578123942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe383ef0 msgr2=0x560dfe384300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbca59c700 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe383ef0 0x560dfe384300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbca59c700 1 -- 172.21.15.39:0/2578123942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dfe1f3880 con 0x560dfe389da0 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbcad9d700 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe3909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbca59c700 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe389da0 0x560dfe38a1b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0df830 tx=0x7ffbbc0a9b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71a38ac9898147e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbc0ff9700 1 -- 172.21.15.39:0/2578123942 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc020070 con 0x560dfe389da0 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbc0ff9700 1 -- 172.21.15.39:0/2578123942 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbbc0a9410 con 0x560dfe389da0 2024-05-04T03:10:47.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.422+0000 7ffbc0ff9700 1 -- 172.21.15.39:0/2578123942 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0b1fe0 con 0x560dfe389da0 2024-05-04T03:10:47.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2578123942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe389da0 msgr2=0x560dfe38a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe389da0 0x560dfe38a1b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0df830 tx=0x7ffbbc0a9b10 comp rx=0 tx=0).stop 2024-05-04T03:10:47.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2578123942 shutdown_connections 2024-05-04T03:10:47.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe389da0 0x560dfe38a1b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe3909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2578123942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe383ef0 0x560dfe384300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2578123942 >> 172.21.15.39:0/2578123942 conn(0x560dfe206df0 msgr2=0x560dfe37ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2578123942 shutdown_connections 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2578123942 wait complete. 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 -- start start 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.423+0000 7ffbdf67eb80 1 Processor -- start 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 0x560dfe38e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 0x560dfe38e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dfe391170 con 0x560dfe383ef0 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560dfe20cbe0 con 0x560dfe384840 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560dfe3912f0 con 0x560dfe389da0 2024-05-04T03:10:47.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbdf67eb80 10 client.0 _ll_register_callbacks cb 0x560dfe23e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43102/0 (socket says 172.21.15.39:43102) 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 -- 172.21.15.39:0/590911578 learned_addr learned my addr 172.21.15.39:0/590911578 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 0x560dfe38e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbcad9d700 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 0x560dfe38e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 -- 172.21.15.39:0/590911578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 msgr2=0x560dfe38e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 0x560dfe38e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 -- 172.21.15.39:0/590911578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 msgr2=0x560dfe38e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 0x560dfe38e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 -- 172.21.15.39:0/590911578 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dfe242640 con 0x560dfe383ef0 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbca59c700 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38dee0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0a5490 tx=0x7ffbbc0ad800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce05b2eff839ed31 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.424+0000 7ffbb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/590911578 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0c2020 con 0x560dfe383ef0 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/590911578 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ffbbc0b1de0 con 0x560dfe383ef0 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/590911578 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbbc0a9410 con 0x560dfe383ef0 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/590911578 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc020070 con 0x560dfe383ef0 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 10 client.15144 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/590911578 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560dfe39bd00 con 0x560dfe383ef0 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/590911578 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbbc0a9410 con 0x560dfe383ef0 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/590911578 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffbbc0b0b40 con 0x560dfe383ef0 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 10 client.15144 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 20 client.15144 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x560dfe39d050 0x560dfe39f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 4 client.15144 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.425+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/590911578 --> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dfe2b41f0 con 0x560dfe39d050 2024-05-04T03:10:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.426+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x560dfe39d050 0x560dfe39f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.426+0000 7ffbb1ffb700 10 client.15144 ms_handle_connect on v2:172.21.15.66:6834/3463909305 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.426+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x560dfe39d050 0x560dfe39f200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be8cf6d0b70f1c10 server_cookie=7b9697f8c94e1ffe in_seq=0 out_seq=0 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/590911578 <== mds.4 v2:172.21.15.66:6834/3463909305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560dfe2b41f0 con 0x560dfe39d050 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbb1ffb700 10 client.15144 handle_command_reply: tid=0 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 client.15144 shutdown 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/590911578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 msgr2=0x560dfe38dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38dee0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0a5490 tx=0x7ffbbc0ad800 comp rx=0 tx=0).stop 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/590911578 shutdown_connections 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x560dfe39d050 0x560dfe39f200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38dee0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 0x560dfe38e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/590911578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 0x560dfe38e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbb1ffb700 0 client.15144 ms_handle_reset on v2:172.21.15.66:6834/3463909305 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/590911578 >> 172.21.15.39:0/590911578 conn(0x560dfe206df0 msgr2=0x560dfe382ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/590911578 shutdown_connections 2024-05-04T03:10:47.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/590911578 wait complete. 2024-05-04T03:10:47.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.427+0000 7ffbdf67eb80 20 client.15144 trim_cache size 0 max 16384 2024-05-04T03:10:47.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.449+0000 7ffbdf67eb80 1 Processor -- start 2024-05-04T03:10:47.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.449+0000 7ffbdf67eb80 1 -- start start 2024-05-04T03:10:47.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.449+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.449+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe39d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.449+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe389da0 0x560dfe39a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.449+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dfe39d540 con 0x560dfe383ef0 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.449+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560dfe39dc10 con 0x560dfe389da0 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.449+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560dfe39e2e0 con 0x560dfe384840 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43108/0 (socket says 172.21.15.39:43108) 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 -- 172.21.15.39:0/842477283 learned_addr learned my addr 172.21.15.39:0/842477283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe39d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbcad9d700 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe389da0 0x560dfe39a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 -- 172.21.15.39:0/842477283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 msgr2=0x560dfe39d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe39d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 -- 172.21.15.39:0/842477283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe389da0 msgr2=0x560dfe39a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe389da0 0x560dfe39a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 -- 172.21.15.39:0/842477283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dfe3912a0 con 0x560dfe383ef0 2024-05-04T03:10:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbcad9d700 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe389da0 0x560dfe39a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbca59c700 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38d220 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0a8ed0 tx=0x7ffbbc0a9b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7a69aafd78616 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbb3fff700 1 -- 172.21.15.39:0/842477283 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0c2030 con 0x560dfe383ef0 2024-05-04T03:10:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbb3fff700 1 -- 172.21.15.39:0/842477283 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbbc0c9030 con 0x560dfe383ef0 2024-05-04T03:10:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.450+0000 7ffbb3fff700 1 -- 172.21.15.39:0/842477283 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0a8740 con 0x560dfe383ef0 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/842477283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 msgr2=0x560dfe38d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38d220 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0a8ed0 tx=0x7ffbbc0a9b10 comp rx=0 tx=0).stop 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/842477283 shutdown_connections 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe38d220 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe389da0 0x560dfe39a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/842477283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe384840 0x560dfe39d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/842477283 >> 172.21.15.39:0/842477283 conn(0x560dfe206df0 msgr2=0x560dfe378a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/842477283 shutdown_connections 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.451+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/842477283 wait complete. 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 1 -- start start 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 1 Processor -- start 2024-05-04T03:10:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe3a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 0x560dfe3a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 0x560dfe3a2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dfe39f080 con 0x560dfe383ef0 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560dfe39e9b0 con 0x560dfe384840 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560dfe39f750 con 0x560dfe389da0 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbdf67eb80 10 client.0 _ll_register_callbacks cb 0x560dfe23e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe3a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe3a8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43116/0 (socket says 172.21.15.39:43116) 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbca59c700 1 -- 172.21.15.39:0/2903399064 learned_addr learned my addr 172.21.15.39:0/2903399064 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 0x560dfe3a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbcad9d700 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 0x560dfe3a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbca59c700 1 -- 172.21.15.39:0/2903399064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 msgr2=0x560dfe3a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbca59c700 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 0x560dfe3a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbca59c700 1 -- 172.21.15.39:0/2903399064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 msgr2=0x560dfe3a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbca59c700 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 0x560dfe3a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.452+0000 7ffbca59c700 1 -- 172.21.15.39:0/2903399064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dfe3a2930 con 0x560dfe383ef0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbca59c700 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe3a8a70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc005950 tx=0x7ffbbc0a5400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9af793477f7ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/2903399064 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0ca490 con 0x560dfe383ef0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/2903399064 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ffbbc0b1de0 con 0x560dfe383ef0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2903399064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbbc0c9030 con 0x560dfe383ef0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/2903399064 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc0b9050 con 0x560dfe383ef0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 10 client.15150 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2903399064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560dfe3a2930 con 0x560dfe383ef0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/2903399064 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbbc0c9030 con 0x560dfe383ef0 2024-05-04T03:10:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/2903399064 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffbbc0bc020 con 0x560dfe383ef0 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 10 client.15150 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 20 client.15150 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x560dfe40add0 0x560dfe40c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 4 client.15150 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.453+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2903399064 --> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] -- command(tid 0: {"prefix": "status"}) -- 0x560dfe2ae290 con 0x560dfe40add0 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.454+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x560dfe40add0 0x560dfe40c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.454+0000 7ffbc9d9b700 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x560dfe40add0 0x560dfe40c240 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=270942b305cf8c15 server_cookie=2ee985df0f0282f in_seq=0 out_seq=0 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.454+0000 7ffbb1ffb700 10 client.15150 ms_handle_connect on v2:172.21.15.66:6834/3463909305 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbb1ffb700 1 -- 172.21.15.39:0/2903399064 <== mds.4 v2:172.21.15.66:6834/3463909305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x560dfe2ae290 con 0x560dfe40add0 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbb1ffb700 10 client.15150 handle_command_reply: tid=0 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 client.15150 shutdown 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2903399064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 msgr2=0x560dfe3a8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe3a8a70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc005950 tx=0x7ffbbc0a5400 comp rx=0 tx=0).stop 2024-05-04T03:10:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2903399064 shutdown_connections 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x560dfe40add0 0x560dfe40c240 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dfe383ef0 0x560dfe3a8a70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560dfe389da0 0x560dfe3a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 --2- 172.21.15.39:0/2903399064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dfe384840 0x560dfe3a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2903399064 >> 172.21.15.39:0/2903399064 conn(0x560dfe206df0 msgr2=0x560dfe379220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbb1ffb700 0 client.15150 ms_handle_reset on v2:172.21.15.66:6834/3463909305 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2903399064 shutdown_connections 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 1 -- 172.21.15.39:0/2903399064 wait complete. 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.455+0000 7ffbdf67eb80 20 client.15150 trim_cache size 0 max 16384 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24455, 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 170.289786413, 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 228.98373508 2024-05-04T03:10:47.462 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ffbb80b2900 msgr2=0x7ffbb80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ffbb80b2900 0x7ffbb80b4e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005fd0 tx=0x7ffbb4005db0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc1034c0 msgr2=0x7ffbcc19bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc1034c0 0x7ffbcc19bbd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00d950 tx=0x7ffbbc00dc60 comp rx=0 tx=0).stop 2024-05-04T03:10:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/1549541507 shutdown_connections 2024-05-04T03:10:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ffbb80b2900 0x7ffbb80b4e10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbcc1034c0 0x7ffbcc19bbd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc108740 0x7ffbcc19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 --2- 172.21.15.39:0/1549541507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbcc107df0 0x7ffbcc19c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:47.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/1549541507 >> 172.21.15.39:0/1549541507 conn(0x7ffbcc0fedf0 msgr2=0x7ffbcc1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:47.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/1549541507 shutdown_connections 2024-05-04T03:10:47.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:47.456+0000 7ffbd0c6a700 1 -- 172.21.15.39:0/1549541507 wait complete. 2024-05-04T03:10:48.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-04T03:10:48.597 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:48.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:48 smithi039 ceph-mon[29715]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:48.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:48 smithi066 ceph-mon[33264]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:48.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:48 smithi167 ceph-mon[33614]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 -- 172.21.15.39:0/2575813643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c109870 msgr2=0x7f9c4c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 --2- 172.21.15.39:0/2575813643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c109870 0x7f9c4c1111d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400b0d0 tx=0x7f9c4400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 -- 172.21.15.39:0/2575813643 shutdown_connections 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 --2- 172.21.15.39:0/2575813643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c109870 0x7f9c4c1111d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 --2- 172.21.15.39:0/2575813643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c1045f0 0x7f9c4c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 --2- 172.21.15.39:0/2575813643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c4c108f20 0x7f9c4c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 -- 172.21.15.39:0/2575813643 >> 172.21.15.39:0/2575813643 conn(0x7f9c4c100040 msgr2=0x7f9c4c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 -- 172.21.15.39:0/2575813643 shutdown_connections 2024-05-04T03:10:50.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.350+0000 7f9c52357700 1 -- 172.21.15.39:0/2575813643 wait complete. 2024-05-04T03:10:50.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.351+0000 7f9c52357700 1 Processor -- start 2024-05-04T03:10:50.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.351+0000 7f9c52357700 1 -- start start 2024-05-04T03:10:50.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c52357700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c1045f0 0x7f9c4c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c52357700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c108f20 0x7f9c4c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c52357700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c4c109870 0x7f9c4c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c52357700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1143d0 con 0x7f9c4c1045f0 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c52357700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c114250 con 0x7f9c4c109870 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c52357700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c114550 con 0x7f9c4c108f20 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c4bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c1045f0 0x7f9c4c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c4bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c1045f0 0x7f9c4c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43136/0 (socket says 172.21.15.39:43136) 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c4bfff700 1 -- 172.21.15.39:0/214201544 learned_addr learned my addr 172.21.15.39:0/214201544 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c508f4700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c4c109870 0x7f9c4c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.352+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c108f20 0x7f9c4c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c4bfff700 1 -- 172.21.15.39:0/214201544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c108f20 msgr2=0x7f9c4c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c108f20 0x7f9c4c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c4bfff700 1 -- 172.21.15.39:0/214201544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c4c109870 msgr2=0x7f9c4c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c4c109870 0x7f9c4c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c4bfff700 1 -- 172.21.15.39:0/214201544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c19fc70 con 0x7f9c4c1045f0 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c508f4700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c4c109870 0x7f9c4c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:10:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c1045f0 0x7f9c4c19ada0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b530 tx=0x7f9c3c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:50.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c497fa700 1 -- 172.21.15.39:0/214201544 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c015020 con 0x7f9c4c1045f0 2024-05-04T03:10:50.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c52357700 1 -- 172.21.15.39:0/214201544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c19ff60 con 0x7f9c4c1045f0 2024-05-04T03:10:50.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c497fa700 1 -- 172.21.15.39:0/214201544 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c3c011cc0 con 0x7f9c4c1045f0 2024-05-04T03:10:50.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c497fa700 1 -- 172.21.15.39:0/214201544 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c010c80 con 0x7f9c4c1045f0 2024-05-04T03:10:50.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.353+0000 7f9c52357700 1 -- 172.21.15.39:0/214201544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c1a7880 con 0x7f9c4c1045f0 2024-05-04T03:10:50.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.356+0000 7f9c497fa700 1 -- 172.21.15.39:0/214201544 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9c3c018020 con 0x7f9c4c1045f0 2024-05-04T03:10:50.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.357+0000 7f9c497fa700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9c380b2880 0x7f9c380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.357+0000 7f9c60d6bb80 1 Processor -- start 2024-05-04T03:10:50.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.357+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9c380b2880 0x7f9c380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.357+0000 7f9c60d6bb80 1 -- start start 2024-05-04T03:10:50.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.357+0000 7f9c497fa700 1 -- 172.21.15.39:0/214201544 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9c3c0e1900 con 0x7f9c4c1045f0 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.357+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec7401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.357+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 0x55b8ec73a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.357+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec65c810 con 0x55b8ec73a840 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec5fb770 con 0x55b8ec73fda0 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec6587d0 con 0x55b8ec739ef0 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9c380b2880 0x7f9c380b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c107d40 tx=0x7f9c3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43146/0 (socket says 172.21.15.39:43146) 2024-05-04T03:10:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 -- 172.21.15.39:0/2801944565 learned_addr learned my addr 172.21.15.39:0/2801944565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec7401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 0x55b8ec73a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 -- 172.21.15.39:0/2801944565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 msgr2=0x55b8ec73a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 0x55b8ec73a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 -- 172.21.15.39:0/2801944565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 msgr2=0x55b8ec7401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec7401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 -- 172.21.15.39:0/2801944565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ec5a9880 con 0x55b8ec73a840 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec7401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.358+0000 7f9c508f4700 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7469f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009c00 tx=0x7f9c440095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48193698be3b2222 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.359+0000 7f9c427fc700 1 -- 172.21.15.39:0/2801944565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44003c00 con 0x55b8ec73a840 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.359+0000 7f9c427fc700 1 -- 172.21.15.39:0/2801944565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c44003da0 con 0x55b8ec73a840 2024-05-04T03:10:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.359+0000 7f9c427fc700 1 -- 172.21.15.39:0/2801944565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44007980 con 0x55b8ec73a840 2024-05-04T03:10:50.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/2801944565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 msgr2=0x55b8ec7469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7469f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009c00 tx=0x7f9c440095d0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/2801944565 shutdown_connections 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7469f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec7401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/2801944565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 0x55b8ec73a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/2801944565 >> 172.21.15.39:0/2801944565 conn(0x55b8ec5bcdf0 msgr2=0x55b8ec734bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/2801944565 shutdown_connections 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/2801944565 wait complete. 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 -- start start 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.360+0000 7f9c60d6bb80 1 Processor -- start 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec739ef0 0x55b8ec743e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73a840 0x55b8ec744390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 0x55b8ec7448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec747170 con 0x55b8ec739ef0 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec5c2be0 con 0x55b8ec73a840 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec7472f0 con 0x55b8ec73fda0 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c60d6bb80 10 client.0 _ll_register_callbacks cb 0x55b8ec5f48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec739ef0 0x55b8ec743e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec739ef0 0x55b8ec743e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43162/0 (socket says 172.21.15.39:43162) 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4bfff700 1 -- 172.21.15.39:0/3216476700 learned_addr learned my addr 172.21.15.39:0/3216476700 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c508f4700 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 0x55b8ec7448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73a840 0x55b8ec744390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4bfff700 1 -- 172.21.15.39:0/3216476700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 msgr2=0x55b8ec7448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 0x55b8ec7448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4bfff700 1 -- 172.21.15.39:0/3216476700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73a840 msgr2=0x55b8ec744390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:50.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73a840 0x55b8ec744390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4bfff700 1 -- 172.21.15.39:0/3216476700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ec5f8640 con 0x55b8ec739ef0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.361+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73a840 0x55b8ec744390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec739ef0 0x55b8ec743e50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0a8bd0 tx=0x7f9c3c0a8f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c125f22a3fed7fa server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/3216476700 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c018520 con 0x55b8ec739ef0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/3216476700 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9c3c0a9130 con 0x55b8ec739ef0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3216476700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c44003da0 con 0x55b8ec739ef0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/3216476700 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c3c0a9330 con 0x55b8ec739ef0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c60d6bb80 10 client.15162 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3216476700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8ec751da0 con 0x55b8ec739ef0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/3216476700 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c44003da0 con 0x55b8ec739ef0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.362+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/3216476700 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9c3c0a4840 con 0x55b8ec739ef0 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c60d6bb80 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:50.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c60d6bb80 10 client.15162 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-04T03:10:50.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c60d6bb80 20 client.15162 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:50.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55b8ec7543f0 0x55b8ec755860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c60d6bb80 4 client.15162 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:50.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3216476700 --> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8ec66a1f0 con 0x55b8ec7543f0 2024-05-04T03:10:50.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55b8ec7543f0 0x55b8ec755860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55b8ec7543f0 0x55b8ec755860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39c6922ed0350d26 server_cookie=262216537747d90d in_seq=0 out_seq=0 2024-05-04T03:10:50.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.363+0000 7f9c2b7fe700 10 client.15162 ms_handle_connect on v2:172.21.15.167:6834/2699315372 2024-05-04T03:10:50.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.364+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/3216476700 <== mds.3 v2:172.21.15.167:6834/2699315372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b8ec66a1f0 con 0x55b8ec7543f0 2024-05-04T03:10:50.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.364+0000 7f9c2b7fe700 10 client.15162 handle_command_reply: tid=0 2024-05-04T03:10:50.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.364+0000 7f9c60d6bb80 1 client.15162 shutdown 2024-05-04T03:10:50.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3216476700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec739ef0 msgr2=0x55b8ec743e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec739ef0 0x55b8ec743e50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0a8bd0 tx=0x7f9c3c0a8f50 comp rx=0 tx=0).stop 2024-05-04T03:10:50.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3216476700 shutdown_connections 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55b8ec7543f0 0x55b8ec755860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec739ef0 0x55b8ec743e50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 0x55b8ec7448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3216476700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73a840 0x55b8ec744390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3216476700 >> 172.21.15.39:0/3216476700 conn(0x55b8ec5bcdf0 msgr2=0x55b8ec738ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c2b7fe700 0 client.15162 ms_handle_reset on v2:172.21.15.167:6834/2699315372 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3216476700 shutdown_connections 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3216476700 wait complete. 2024-05-04T03:10:50.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.365+0000 7f9c60d6bb80 20 client.15162 trim_cache size 0 max 16384 2024-05-04T03:10:50.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c60d6bb80 1 Processor -- start 2024-05-04T03:10:50.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c60d6bb80 1 -- start start 2024-05-04T03:10:50.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 0x55b8ec734d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec750590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec72f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec753ba0 con 0x55b8ec73a840 2024-05-04T03:10:50.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec754270 con 0x55b8ec73fda0 2024-05-04T03:10:50.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec754940 con 0x55b8ec739ef0 2024-05-04T03:10:50.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.386+0000 7f9c4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec750590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec750590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43166/0 (socket says 172.21.15.39:43166) 2024-05-04T03:10:50.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4b7fe700 1 -- 172.21.15.39:0/3608339498 learned_addr learned my addr 172.21.15.39:0/3608339498 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:50.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 0x55b8ec734d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4b7fe700 1 -- 172.21.15.39:0/3608339498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 msgr2=0x55b8ec734d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c508f4700 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec72f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 0x55b8ec734d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4b7fe700 1 -- 172.21.15.39:0/3608339498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 msgr2=0x55b8ec72f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec72f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4b7fe700 1 -- 172.21.15.39:0/3608339498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ec7472a0 con 0x55b8ec73a840 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec750590 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005950 tx=0x7f9c34013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5659d846efd1d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c41ffb700 1 -- 172.21.15.39:0/3608339498 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3402d070 con 0x55b8ec73a840 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c41ffb700 1 -- 172.21.15.39:0/3608339498 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c34028960 con 0x55b8ec73a840 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.387+0000 7f9c41ffb700 1 -- 172.21.15.39:0/3608339498 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3401f9a0 con 0x55b8ec73a840 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3608339498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 msgr2=0x55b8ec750590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec750590 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005950 tx=0x7f9c34013000 comp rx=0 tx=0).stop 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3608339498 shutdown_connections 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec73fda0 0x55b8ec72f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec739ef0 0x55b8ec734d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/3608339498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec750590 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3608339498 >> 172.21.15.39:0/3608339498 conn(0x55b8ec5bcdf0 msgr2=0x55b8ec738ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3608339498 shutdown_connections 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/3608339498 wait complete. 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.388+0000 7f9c60d6bb80 1 -- start start 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c60d6bb80 1 Processor -- start 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec739ef0 0x55b8ec75f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c60d6bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 0x55b8ec740f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec7556e0 con 0x55b8ec73a840 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec755010 con 0x55b8ec739ef0 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c60d6bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec755db0 con 0x55b8ec73fda0 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c60d6bb80 10 client.0 _ll_register_callbacks cb 0x55b8ec5f48d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43168/0 (socket says 172.21.15.39:43168) 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4b7fe700 1 -- 172.21.15.39:0/1901960336 learned_addr learned my addr 172.21.15.39:0/1901960336 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec739ef0 0x55b8ec75f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c508f4700 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 0x55b8ec740f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4b7fe700 1 -- 172.21.15.39:0/1901960336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 msgr2=0x55b8ec740f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 0x55b8ec740f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4b7fe700 1 -- 172.21.15.39:0/1901960336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec739ef0 msgr2=0x55b8ec75f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec739ef0 0x55b8ec75f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.389+0000 7f9c4b7fe700 1 -- 172.21.15.39:0/1901960336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ec741670 con 0x55b8ec73a840 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c4b7fe700 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7409e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400ba80 tx=0x7f9c34010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67440e9dad28844 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/1901960336 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34036020 con 0x55b8ec73a840 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/1901960336 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9c340284d0 con 0x55b8ec73a840 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/1901960336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c34028960 con 0x55b8ec73a840 2024-05-04T03:10:50.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/1901960336 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c3402d040 con 0x55b8ec73a840 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c60d6bb80 10 client.15168 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/1901960336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8ec7bf860 con 0x55b8ec73a840 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.390+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/1901960336 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c34028960 con 0x55b8ec73a840 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/1901960336 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9c34028ce0 con 0x55b8ec73a840 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c60d6bb80 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c60d6bb80 10 client.15168 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c60d6bb80 20 client.15168 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55b8ec7c1750 0x55b8ec7c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c60d6bb80 4 client.15168 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/1901960336 --> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] -- command(tid 0: {"prefix": "status"}) -- 0x55b8ec664290 con 0x55b8ec7c1750 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55b8ec7c1750 0x55b8ec7c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c4bfff700 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55b8ec7c1750 0x55b8ec7c2bc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=453f58f16ed497f0 server_cookie=7d1d7c96c149113b in_seq=0 out_seq=0 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.391+0000 7f9c2b7fe700 10 client.15168 ms_handle_connect on v2:172.21.15.167:6834/2699315372 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c2b7fe700 1 -- 172.21.15.39:0/1901960336 <== mds.3 v2:172.21.15.167:6834/2699315372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b8ec664290 con 0x55b8ec7c1750 2024-05-04T03:10:50.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c2b7fe700 10 client.15168 handle_command_reply: tid=0 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 client.15168 shutdown 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/1901960336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 msgr2=0x55b8ec7409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7409e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400ba80 tx=0x7f9c34010b50 comp rx=0 tx=0).stop 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/1901960336 shutdown_connections 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8ec73fda0 0x55b8ec740f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b8ec739ef0 0x55b8ec75f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x55b8ec7c1750 0x55b8ec7c2bc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c2b7fe700 0 client.15168 ms_handle_reset on v2:172.21.15.167:6834/2699315372 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 --2- 172.21.15.39:0/1901960336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b8ec73a840 0x55b8ec7409e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/1901960336 >> 172.21.15.39:0/1901960336 conn(0x55b8ec5bcdf0 msgr2=0x55b8ec72f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/1901960336 shutdown_connections 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 1 -- 172.21.15.39:0/1901960336 wait complete. 2024-05-04T03:10:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.392+0000 7f9c60d6bb80 20 client.15168 trim_cache size 0 max 16384 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24466, 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 174.24334683500001, 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:50.401 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:50.402 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 229.13311574599999 2024-05-04T03:10:50.402 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 -- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9c380b2880 msgr2=0x7f9c380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9c380b2880 0x7f9c380b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c107d40 tx=0x7f9c3400b410 comp rx=0 tx=0).stop 2024-05-04T03:10:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 -- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c1045f0 msgr2=0x7f9c4c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c1045f0 0x7f9c4c19ada0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00b530 tx=0x7f9c3c00b840 comp rx=0 tx=0).stop 2024-05-04T03:10:50.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 -- 172.21.15.39:0/214201544 shutdown_connections 2024-05-04T03:10:50.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9c380b2880 0x7f9c380b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c4c1045f0 0x7f9c4c19ada0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c4c109870 0x7f9c4c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 --2- 172.21.15.39:0/214201544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4c108f20 0x7f9c4c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:50.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 -- 172.21.15.39:0/214201544 >> 172.21.15.39:0/214201544 conn(0x7f9c4c100040 msgr2=0x7f9c4c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:50.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 -- 172.21.15.39:0/214201544 shutdown_connections 2024-05-04T03:10:50.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:50.393+0000 7f9c52357700 1 -- 172.21.15.39:0/214201544 wait complete. 2024-05-04T03:10:50.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:50 smithi039 ceph-mon[29715]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:50.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:50 smithi066 ceph-mon[33264]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:50.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:50 smithi167 ceph-mon[33614]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:10:51.133 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-04T03:10:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:52.560 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:52 smithi039 ceph-mon[29715]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:52.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:52 smithi066 ceph-mon[33264]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:52.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:52 smithi167 ceph-mon[33614]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 -- 172.21.15.39:0/2442766667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564102790 msgr2=0x7f0564102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 --2- 172.21.15.39:0/2442766667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564102790 0x7f0564102ba0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0554009990 tx=0x7f0554009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 -- 172.21.15.39:0/2442766667 shutdown_connections 2024-05-04T03:10:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 --2- 172.21.15.39:0/2442766667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05641030e0 0x7f0564111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 --2- 172.21.15.39:0/2442766667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0564109300 0x7f0564109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 --2- 172.21.15.39:0/2442766667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564102790 0x7f0564102ba0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 -- 172.21.15.39:0/2442766667 >> 172.21.15.39:0/2442766667 conn(0x7f05640fe520 msgr2=0x7f0564100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:53.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 -- 172.21.15.39:0/2442766667 shutdown_connections 2024-05-04T03:10:53.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.168+0000 7f056c169700 1 -- 172.21.15.39:0/2442766667 wait complete. 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056c169700 1 Processor -- start 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056c169700 1 -- start start 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056c169700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0564102790 0x7f056419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056c169700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05641030e0 0x7f056419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056c169700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564109300 0x7f056419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056c169700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0564114450 con 0x7f0564109300 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056c169700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05641142d0 con 0x7f0564102790 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056c169700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05641145d0 con 0x7f05641030e0 2024-05-04T03:10:53.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056a706700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564109300 0x7f056419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056a706700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564109300 0x7f056419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43186/0 (socket says 172.21.15.39:43186) 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056a706700 1 -- 172.21.15.39:0/855898023 learned_addr learned my addr 172.21.15.39:0/855898023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f0569f05700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0564102790 0x7f056419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f0569704700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05641030e0 0x7f056419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.169+0000 7f056a706700 1 -- 172.21.15.39:0/855898023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05641030e0 msgr2=0x7f056419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f056a706700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05641030e0 0x7f056419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f056a706700 1 -- 172.21.15.39:0/855898023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0564102790 msgr2=0x7f056419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f056a706700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0564102790 0x7f056419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f056a706700 1 -- 172.21.15.39:0/855898023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056419fcd0 con 0x7f0564109300 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f056a706700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564109300 0x7f056419f630 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f055c007ee0 tx=0x7f055c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:53.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f055affd700 1 -- 172.21.15.39:0/855898023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c004bb0 con 0x7f0564109300 2024-05-04T03:10:53.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f055affd700 1 -- 172.21.15.39:0/855898023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f055c004d50 con 0x7f0564109300 2024-05-04T03:10:53.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f056c169700 1 -- 172.21.15.39:0/855898023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056419ffc0 con 0x7f0564109300 2024-05-04T03:10:53.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f055affd700 1 -- 172.21.15.39:0/855898023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0058b0 con 0x7f0564109300 2024-05-04T03:10:53.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.170+0000 7f056c169700 1 -- 172.21.15.39:0/855898023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05641a78e0 con 0x7f0564109300 2024-05-04T03:10:53.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.173+0000 7f055affd700 1 -- 172.21.15.39:0/855898023 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f055c01a020 con 0x7f0564109300 2024-05-04T03:10:53.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.173+0000 7f055affd700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f05500b2880 0x7f05500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f0569f05700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f05500b2880 0x7f05500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f055affd700 1 -- 172.21.15.39:0/855898023 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f055c0e1860 con 0x7f0564109300 2024-05-04T03:10:53.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f057ab7db80 1 Processor -- start 2024-05-04T03:10:53.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f057ab7db80 1 -- start start 2024-05-04T03:10:53.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f0569f05700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f05500b2880 0x7f05500b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f056406ca70 tx=0x7f0560007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:53.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01bef0 0x5589fe01c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01c840 0x5589fe0289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f057ab7db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5589fdf3e810 con 0x5589fe01c840 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f057ab7db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5589fdedd770 con 0x5589fe021da0 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f057ab7db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589fdf3a7d0 con 0x5589fe01bef0 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f056a706700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01c840 0x5589fe0289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f056a706700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01c840 0x5589fe0289f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43200/0 (socket says 172.21.15.39:43200) 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f056a706700 1 -- 172.21.15.39:0/1755727983 learned_addr learned my addr 172.21.15.39:0/1755727983 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.174+0000 7f0569f05700 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f0569704700 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01bef0 0x5589fe01c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f056a706700 1 -- 172.21.15.39:0/1755727983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01bef0 msgr2=0x5589fe01c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f056a706700 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01bef0 0x5589fe01c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f056a706700 1 -- 172.21.15.39:0/1755727983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 msgr2=0x5589fe0221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f056a706700 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f056a706700 1 -- 172.21.15.39:0/1755727983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589fde8b880 con 0x5589fe01c840 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f0569f05700 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:10:53.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f056a706700 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01c840 0x5589fe0289f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f055c0a6100 tx=0x7f055c0a88b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6282103487e539f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f054ffff700 1 -- 172.21.15.39:0/1755727983 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0a9590 con 0x5589fe01c840 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f054ffff700 1 -- 172.21.15.39:0/1755727983 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f055c0a8e90 con 0x5589fe01c840 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.175+0000 7f054ffff700 1 -- 172.21.15.39:0/1755727983 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0a9350 con 0x5589fe01c840 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 -- 172.21.15.39:0/1755727983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01c840 msgr2=0x5589fe0289f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01c840 0x5589fe0289f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f055c0a6100 tx=0x7f055c0a88b0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 -- 172.21.15.39:0/1755727983 shutdown_connections 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01c840 0x5589fe0289f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 --2- 172.21.15.39:0/1755727983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01bef0 0x5589fe01c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 -- 172.21.15.39:0/1755727983 >> 172.21.15.39:0/1755727983 conn(0x5589fde9edf0 msgr2=0x5589fe016bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 -- 172.21.15.39:0/1755727983 shutdown_connections 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.176+0000 7f057ab7db80 1 -- 172.21.15.39:0/1755727983 wait complete. 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 1 -- start start 2024-05-04T03:10:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 1 Processor -- start 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe025e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5589fe029170 con 0x5589fe01bef0 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5589fdea4be0 con 0x5589fe021da0 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589fe0292f0 con 0x5589fe01c840 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f057ab7db80 10 client.0 _ll_register_callbacks cb 0x5589fded68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f0569f05700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe025e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f0569f05700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe025e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43210/0 (socket says 172.21.15.39:43210) 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f0569f05700 1 -- 172.21.15.39:0/3632272491 learned_addr learned my addr 172.21.15.39:0/3632272491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.177+0000 7f056a706700 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f0569704700 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f0569f05700 1 -- 172.21.15.39:0/3632272491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 msgr2=0x5589fe0263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f0569f05700 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0263b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f0569f05700 1 -- 172.21.15.39:0/3632272491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 msgr2=0x5589fe0268f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f0569f05700 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0268f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f0569f05700 1 -- 172.21.15.39:0/3632272491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589fdeda640 con 0x5589fe01bef0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f0569f05700 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe025e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0560010bf0 tx=0x7f05600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cbdf4aea35597f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f054d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f054d7fa700 1 -- 172.21.15.39:0/3632272491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560024070 con 0x5589fe01bef0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f054d7fa700 1 -- 172.21.15.39:0/3632272491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0560003b10 con 0x5589fe01bef0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f057ab7db80 1 -- 172.21.15.39:0/3632272491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f055c0a8e90 con 0x5589fe01bef0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f054d7fa700 1 -- 172.21.15.39:0/3632272491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0560030050 con 0x5589fe01bef0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f057ab7db80 10 client.15180 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.178+0000 7f057ab7db80 1 -- 172.21.15.39:0/3632272491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589fe033df0 con 0x5589fe01bef0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f054d7fa700 1 -- 172.21.15.39:0/3632272491 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f055c0a8e90 con 0x5589fe01bef0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f054d7fa700 1 -- 172.21.15.39:0/3632272491 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0560020410 con 0x5589fe01bef0 2024-05-04T03:10:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f057ab7db80 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f057ab7db80 10 client.15180 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f057ab7db80 20 client.15180 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x5589fe0364b0 0x5589fe037920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f057ab7db80 4 client.15180 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f057ab7db80 1 -- 172.21.15.39:0/3632272491 --> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589fdf4c1f0 con 0x5589fe0364b0 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.179+0000 7f0569704700 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x5589fe0364b0 0x5589fe037920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.180+0000 7f054d7fa700 10 client.15180 ms_handle_connect on v2:172.21.15.167:6836/2270685849 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.180+0000 7f0569704700 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x5589fe0364b0 0x5589fe037920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=578c77002f81080c server_cookie=e9990d94d834ac2c in_seq=0 out_seq=0 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f054d7fa700 1 -- 172.21.15.39:0/3632272491 <== mds.2 v2:172.21.15.167:6836/2270685849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589fdf4c1f0 con 0x5589fe0364b0 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f054d7fa700 10 client.15180 handle_command_reply: tid=0 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 client.15180 shutdown 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 -- 172.21.15.39:0/3632272491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 msgr2=0x5589fe025e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe025e70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0560010bf0 tx=0x7f05600047c0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 -- 172.21.15.39:0/3632272491 shutdown_connections 2024-05-04T03:10:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x5589fe0364b0 0x5589fe037920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe025e70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe021da0 0x5589fe0268f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3632272491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f054d7fa700 0 client.15180 ms_handle_reset on v2:172.21.15.167:6836/2270685849 2024-05-04T03:10:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 -- 172.21.15.39:0/3632272491 >> 172.21.15.39:0/3632272491 conn(0x5589fde9edf0 msgr2=0x5589fe01aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 -- 172.21.15.39:0/3632272491 shutdown_connections 2024-05-04T03:10:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 1 -- 172.21.15.39:0/3632272491 wait complete. 2024-05-04T03:10:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.181+0000 7f057ab7db80 20 client.15180 trim_cache size 0 max 16384 2024-05-04T03:10:53.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f057ab7db80 1 Processor -- start 2024-05-04T03:10:53.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f057ab7db80 1 -- start start 2024-05-04T03:10:53.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01bef0 0x5589fe0325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0114c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe021da0 0x5589fe023990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f057ab7db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5589fe035c60 con 0x5589fe021da0 2024-05-04T03:10:53.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f057ab7db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5589fe036330 con 0x5589fe01bef0 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f057ab7db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589fe036a00 con 0x5589fe01c840 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f056a706700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe021da0 0x5589fe023990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f056a706700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe021da0 0x5589fe023990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43224/0 (socket says 172.21.15.39:43224) 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f056a706700 1 -- 172.21.15.39:0/3377935250 learned_addr learned my addr 172.21.15.39:0/3377935250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f0569704700 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f0569f05700 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01bef0 0x5589fe0325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.203+0000 7f056a706700 1 -- 172.21.15.39:0/3377935250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 msgr2=0x5589fe0114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f056a706700 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0114c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f056a706700 1 -- 172.21.15.39:0/3377935250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01bef0 msgr2=0x5589fe0325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f056a706700 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01bef0 0x5589fe0325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f056a706700 1 -- 172.21.15.39:0/3377935250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589fe0243e0 con 0x5589fe021da0 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f0569704700 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f0569f05700 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01bef0 0x5589fe0325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f056a706700 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe021da0 0x5589fe023990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f055c014000 tx=0x7f055c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01b1241e22e20f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:53.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f054f7fe700 1 -- 172.21.15.39:0/3377935250 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0c8070 con 0x5589fe021da0 2024-05-04T03:10:53.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f054f7fe700 1 -- 172.21.15.39:0/3377935250 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f055c0ad240 con 0x5589fe021da0 2024-05-04T03:10:53.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.204+0000 7f054f7fe700 1 -- 172.21.15.39:0/3377935250 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c06c070 con 0x5589fe021da0 2024-05-04T03:10:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 -- 172.21.15.39:0/3377935250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe021da0 msgr2=0x5589fe023990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe021da0 0x5589fe023990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f055c014000 tx=0x7f055c00b040 comp rx=0 tx=0).stop 2024-05-04T03:10:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 -- 172.21.15.39:0/3377935250 shutdown_connections 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe021da0 0x5589fe023990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01bef0 0x5589fe0325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3377935250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe01c840 0x5589fe0114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 -- 172.21.15.39:0/3377935250 >> 172.21.15.39:0/3377935250 conn(0x5589fde9edf0 msgr2=0x5589fe016a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 -- 172.21.15.39:0/3377935250 shutdown_connections 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 -- 172.21.15.39:0/3377935250 wait complete. 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 -- start start 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.205+0000 7f057ab7db80 1 Processor -- start 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe0279e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01c840 0x5589fe027f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f057ab7db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe021da0 0x5589fe028460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f057ab7db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5589fe0377a0 con 0x5589fe01bef0 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f057ab7db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5589fe0370d0 con 0x5589fe01c840 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f057ab7db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5589fe037e70 con 0x5589fe021da0 2024-05-04T03:10:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe0279e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f057ab7db80 10 client.0 _ll_register_callbacks cb 0x5589fded68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe0279e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43236/0 (socket says 172.21.15.39:43236) 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 -- 172.21.15.39:0/3475813040 learned_addr learned my addr 172.21.15.39:0/3475813040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569704700 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01c840 0x5589fe027f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f056a706700 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe021da0 0x5589fe028460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 -- 172.21.15.39:0/3475813040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe021da0 msgr2=0x5589fe028460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe021da0 0x5589fe028460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 -- 172.21.15.39:0/3475813040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01c840 msgr2=0x5589fe027f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01c840 0x5589fe027f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 -- 172.21.15.39:0/3475813040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589fe03f500 con 0x5589fe01bef0 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f0569f05700 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe0279e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f05600079b0 tx=0x7f05600206c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1658b00a57b89b1c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.206+0000 7f054d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f054d7fa700 1 -- 172.21.15.39:0/3475813040 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560024040 con 0x5589fe01bef0 2024-05-04T03:10:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f054d7fa700 1 -- 172.21.15.39:0/3475813040 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0560003b10 con 0x5589fe01bef0 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 1 -- 172.21.15.39:0/3475813040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589fdf46290 con 0x5589fe01bef0 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f054d7fa700 1 -- 172.21.15.39:0/3475813040 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0560020c10 con 0x5589fe01bef0 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 10 client.15186 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 1 -- 172.21.15.39:0/3475813040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589fe0a1cf0 con 0x5589fe01bef0 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f054d7fa700 1 -- 172.21.15.39:0/3475813040 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5589fdf46290 con 0x5589fe01bef0 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f054d7fa700 1 -- 172.21.15.39:0/3475813040 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0560011440 con 0x5589fe01bef0 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 10 client.15186 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 20 client.15186 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x5589fe0a4120 0x5589fe0a5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 4 client.15186 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.207+0000 7f057ab7db80 1 -- 172.21.15.39:0/3475813040 --> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] -- command(tid 0: {"prefix": "status"}) -- 0x5589fddbe200 con 0x5589fe0a4120 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.208+0000 7f0569704700 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x5589fe0a4120 0x5589fe0a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:53.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.208+0000 7f054d7fa700 10 client.15186 ms_handle_connect on v2:172.21.15.167:6836/2270685849 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.208+0000 7f0569704700 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x5589fe0a4120 0x5589fe0a5590 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d538b30d2606d3f5 server_cookie=c76008e5c6535fd6 in_seq=0 out_seq=0 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f054d7fa700 1 -- 172.21.15.39:0/3475813040 <== mds.2 v2:172.21.15.167:6836/2270685849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5589fddbe200 con 0x5589fe0a4120 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f054d7fa700 10 client.15186 handle_command_reply: tid=0 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 client.15186 shutdown 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 -- 172.21.15.39:0/3475813040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 msgr2=0x5589fe0279e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe0279e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f05600079b0 tx=0x7f05600206c0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 -- 172.21.15.39:0/3475813040 shutdown_connections 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x5589fe0a4120 0x5589fe0a5590 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5589fe01bef0 0x5589fe0279e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5589fe021da0 0x5589fe028460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f054d7fa700 0 client.15186 ms_handle_reset on v2:172.21.15.167:6836/2270685849 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 --2- 172.21.15.39:0/3475813040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5589fe01c840 0x5589fe027f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 -- 172.21.15.39:0/3475813040 >> 172.21.15.39:0/3475813040 conn(0x5589fde9edf0 msgr2=0x5589fe016a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 -- 172.21.15.39:0/3475813040 shutdown_connections 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 1 -- 172.21.15.39:0/3475813040 wait complete. 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.209+0000 7f057ab7db80 20 client.15186 trim_cache size 0 max 16384 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24472, 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:53.215 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 178.07631915499999, 2024-05-04T03:10:53.216 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:53.216 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:53.216 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:53.216 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 229.178078513 2024-05-04T03:10:53.216 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:53.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 -- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f05500b2880 msgr2=0x7f05500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f05500b2880 0x7f05500b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f056406ca70 tx=0x7f0560007400 comp rx=0 tx=0).stop 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 -- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564109300 msgr2=0x7f056419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564109300 0x7f056419f630 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f055c007ee0 tx=0x7f055c00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 -- 172.21.15.39:0/855898023 shutdown_connections 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f05500b2880 0x7f05500b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0564109300 0x7f056419f630 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0564102790 0x7f056419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 --2- 172.21.15.39:0/855898023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05641030e0 0x7f056419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 -- 172.21.15.39:0/855898023 >> 172.21.15.39:0/855898023 conn(0x7f05640fe520 msgr2=0x7f0564107210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 -- 172.21.15.39:0/855898023 shutdown_connections 2024-05-04T03:10:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:53.210+0000 7f056c169700 1 -- 172.21.15.39:0/855898023 wait complete. 2024-05-04T03:10:53.914 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-04T03:10:54.163 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:54.541 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:54 smithi039 ceph-mon[29715]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:54.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:54 smithi167 ceph-mon[33614]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:54.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:54 smithi066 ceph-mon[33264]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:10:55.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.893+0000 7f58008f8700 1 -- 172.21.15.39:0/3399923410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 msgr2=0x7f57fc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.893+0000 7f58008f8700 1 --2- 172.21.15.39:0/3399923410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 0x7f57fc109320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57e40099c0 tx=0x7f57e4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.893+0000 7f58008f8700 1 -- 172.21.15.39:0/3399923410 shutdown_connections 2024-05-04T03:10:55.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.893+0000 7f58008f8700 1 --2- 172.21.15.39:0/3399923410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57fc109860 0x7f57fc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.894+0000 7f58008f8700 1 --2- 172.21.15.39:0/3399923410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57fc1045e0 0x7f57fc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.894+0000 7f58008f8700 1 --2- 172.21.15.39:0/3399923410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 0x7f57fc109320 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.894+0000 7f58008f8700 1 -- 172.21.15.39:0/3399923410 >> 172.21.15.39:0/3399923410 conn(0x7f57fc0ffed0 msgr2=0x7f57fc102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:55.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.894+0000 7f58008f8700 1 -- 172.21.15.39:0/3399923410 shutdown_connections 2024-05-04T03:10:55.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.894+0000 7f58008f8700 1 -- 172.21.15.39:0/3399923410 wait complete. 2024-05-04T03:10:55.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.894+0000 7f58008f8700 1 Processor -- start 2024-05-04T03:10:55.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f58008f8700 1 -- start start 2024-05-04T03:10:55.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f58008f8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57fc1045e0 0x7f57fc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f58008f8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 0x7f57fc19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f58008f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57fc109860 0x7f57fc19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f58008f8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc114320 con 0x7f57fc108f10 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f58008f8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1141a0 con 0x7f57fc1045e0 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f58008f8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc1144a0 con 0x7f57fc109860 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57f9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 0x7f57fc19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57f9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 0x7f57fc19b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59336/0 (socket says 172.21.15.39:59336) 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57f9d9b700 1 -- 172.21.15.39:0/681624680 learned_addr learned my addr 172.21.15.39:0/681624680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57fa59c700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57fc1045e0 0x7f57fc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57fad9d700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57fc109860 0x7f57fc19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57f9d9b700 1 -- 172.21.15.39:0/681624680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57fc109860 msgr2=0x7f57fc19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57fc109860 0x7f57fc19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57f9d9b700 1 -- 172.21.15.39:0/681624680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57fc1045e0 msgr2=0x7f57fc19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57fc1045e0 0x7f57fc19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.895+0000 7f57f9d9b700 1 -- 172.21.15.39:0/681624680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc19fc00 con 0x7f57fc108f10 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.896+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 0x7f57fc19b2d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57e400b580 tx=0x7f57e4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.896+0000 7f57f37fe700 1 -- 172.21.15.39:0/681624680 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4027070 con 0x7f57fc108f10 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.896+0000 7f57f37fe700 1 -- 172.21.15.39:0/681624680 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57e400fc00 con 0x7f57fc108f10 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.896+0000 7f57f37fe700 1 -- 172.21.15.39:0/681624680 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4022cb0 con 0x7f57fc108f10 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.896+0000 7f58008f8700 1 -- 172.21.15.39:0/681624680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc19fe90 con 0x7f57fc108f10 2024-05-04T03:10:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.896+0000 7f58008f8700 1 -- 172.21.15.39:0/681624680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc1a7810 con 0x7f57fc108f10 2024-05-04T03:10:55.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.899+0000 7f57f37fe700 1 -- 172.21.15.39:0/681624680 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f57e402a070 con 0x7f57fc108f10 2024-05-04T03:10:55.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.899+0000 7f57f37fe700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f57e80b2900 0x7f57e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f57fa59c700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f57e80b2900 0x7f57e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f57f37fe700 1 -- 172.21.15.39:0/681624680 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f57e40e2f90 con 0x7f57fc108f10 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f580f30cb80 1 Processor -- start 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f580f30cb80 1 -- start start 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 0x55749cd7b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd74ef0 0x55749cd75300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f57fa59c700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f57e80b2900 0x7f57e80b4e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009550 tx=0x7f57f40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f580f30cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55749cc97810 con 0x55749cd74ef0 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f580f30cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55749cc36770 con 0x55749cd7ada0 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f580f30cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55749cc937d0 con 0x55749cd75840 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f57f9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd74ef0 0x55749cd75300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f57f9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd74ef0 0x55749cd75300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59338/0 (socket says 172.21.15.39:59338) 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.900+0000 7f57f9d9b700 1 -- 172.21.15.39:0/4140951570 learned_addr learned my addr 172.21.15.39:0/4140951570 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57fa59c700 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 0x55749cd7b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57fad9d700 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f9d9b700 1 -- 172.21.15.39:0/4140951570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 msgr2=0x55749cd819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd819f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f9d9b700 1 -- 172.21.15.39:0/4140951570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 msgr2=0x55749cd7b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 0x55749cd7b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f9d9b700 1 -- 172.21.15.39:0/4140951570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749cbe4880 con 0x55749cd74ef0 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd74ef0 0x55749cd75300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f57e40aa350 tx=0x7f57e40aa090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa173e010f9eb60 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f0ff9700 1 -- 172.21.15.39:0/4140951570 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40aa650 con 0x55749cd74ef0 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f0ff9700 1 -- 172.21.15.39:0/4140951570 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57e40aea00 con 0x55749cd74ef0 2024-05-04T03:10:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.901+0000 7f57f0ff9700 1 -- 172.21.15.39:0/4140951570 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40aab10 con 0x55749cd74ef0 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 -- 172.21.15.39:0/4140951570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd74ef0 msgr2=0x55749cd75300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd74ef0 0x55749cd75300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f57e40aa350 tx=0x7f57e40aa090 comp rx=0 tx=0).stop 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 -- 172.21.15.39:0/4140951570 shutdown_connections 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 0x55749cd7b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 --2- 172.21.15.39:0/4140951570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd74ef0 0x55749cd75300 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 -- 172.21.15.39:0/4140951570 >> 172.21.15.39:0/4140951570 conn(0x55749cbf7df0 msgr2=0x55749cd6fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 -- 172.21.15.39:0/4140951570 shutdown_connections 2024-05-04T03:10:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.902+0000 7f580f30cb80 1 -- 172.21.15.39:0/4140951570 wait complete. 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 1 -- start start 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 1 Processor -- start 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 0x55749cd7ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd7f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd7f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55749cd81f30 con 0x55749cd7ada0 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55749cbfdbe0 con 0x55749cd74ef0 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55749cd820b0 con 0x55749cd75840 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f580f30cb80 10 client.0 _ll_register_callbacks cb 0x55749cc2f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd7f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd7f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59350/0 (socket says 172.21.15.39:59350) 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f57fad9d700 1 -- 172.21.15.39:0/2001927517 learned_addr learned my addr 172.21.15.39:0/2001927517 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f57fa59c700 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 0x55749cd7ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.903+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd7f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57fad9d700 1 -- 172.21.15.39:0/2001927517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 msgr2=0x55749cd7f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57fad9d700 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd7f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57fad9d700 1 -- 172.21.15.39:0/2001927517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 msgr2=0x55749cd7ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57fad9d700 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 0x55749cd7ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57fad9d700 1 -- 172.21.15.39:0/2001927517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749cc33640 con 0x55749cd7ada0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57fad9d700 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd7f8f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f57ec003930 tx=0x7f57ec003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5863775a72d530ec server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57e1ffb700 1 -- 172.21.15.39:0/2001927517 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00bd70 con 0x55749cd7ada0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57e1ffb700 1 -- 172.21.15.39:0/2001927517 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f57ec01b440 con 0x55749cd7ada0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f580f30cb80 1 -- 172.21.15.39:0/2001927517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e40aea00 con 0x55749cd7ada0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57e1ffb700 1 -- 172.21.15.39:0/2001927517 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f57ec004830 con 0x55749cd7ada0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f580f30cb80 10 client.15198 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f580f30cb80 1 -- 172.21.15.39:0/2001927517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55749cd8caf0 con 0x55749cd7ada0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.904+0000 7f57e1ffb700 1 -- 172.21.15.39:0/2001927517 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57e40aea00 con 0x55749cd7ada0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.905+0000 7f57e1ffb700 1 -- 172.21.15.39:0/2001927517 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f57ec019020 con 0x55749cd7ada0 2024-05-04T03:10:55.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.905+0000 7f580f30cb80 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:55.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.905+0000 7f580f30cb80 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:10:55.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.905+0000 7f580f30cb80 20 client.15198 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:55.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.905+0000 7f580f30cb80 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55749cd8f1b0 0x55749cd90620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.905+0000 7f580f30cb80 4 client.15198 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:55.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.905+0000 7f580f30cb80 1 -- 172.21.15.39:0/2001927517 --> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55749cca51f0 con 0x55749cd8f1b0 2024-05-04T03:10:55.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.905+0000 7f57fa59c700 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55749cd8f1b0 0x55749cd90620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.906+0000 7f57e1ffb700 10 client.15198 ms_handle_connect on v2:172.21.15.167:6838/975309166 2024-05-04T03:10:55.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.906+0000 7f57fa59c700 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55749cd8f1b0 0x55749cd90620 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fea91f3a27e1cdd server_cookie=369d72c9db4147b6 in_seq=0 out_seq=0 2024-05-04T03:10:55.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.906+0000 7f57e1ffb700 1 -- 172.21.15.39:0/2001927517 <== mds.1 v2:172.21.15.167:6838/975309166 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55749cca51f0 con 0x55749cd8f1b0 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.906+0000 7f57e1ffb700 10 client.15198 handle_command_reply: tid=0 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.906+0000 7f580f30cb80 1 client.15198 shutdown 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 -- 172.21.15.39:0/2001927517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 msgr2=0x55749cd7f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd7f8f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f57ec003930 tx=0x7f57ec003c40 comp rx=0 tx=0).stop 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 -- 172.21.15.39:0/2001927517 shutdown_connections 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55749cd8f1b0 0x55749cd90620 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd7f8f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 0x55749cd7ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 --2- 172.21.15.39:0/2001927517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd7f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f57e1ffb700 0 client.15198 ms_handle_reset on v2:172.21.15.167:6838/975309166 2024-05-04T03:10:55.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 -- 172.21.15.39:0/2001927517 >> 172.21.15.39:0/2001927517 conn(0x55749cbf7df0 msgr2=0x55749cd73ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:55.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 -- 172.21.15.39:0/2001927517 shutdown_connections 2024-05-04T03:10:55.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 1 -- 172.21.15.39:0/2001927517 wait complete. 2024-05-04T03:10:55.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.907+0000 7f580f30cb80 20 client.15198 trim_cache size 0 max 16384 2024-05-04T03:10:55.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.928+0000 7f580f30cb80 1 Processor -- start 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.928+0000 7f580f30cb80 1 -- start start 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.928+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd74ef0 0x55749cd8b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd75840 0x55749cd8e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 0x55749cd7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f580f30cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55749cd8e960 con 0x55749cd75840 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f580f30cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55749cd8f030 con 0x55749cd7ada0 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f580f30cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55749cd8f700 con 0x55749cd74ef0 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd75840 0x55749cd8e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd75840 0x55749cd8e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59360/0 (socket says 172.21.15.39:59360) 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 -- 172.21.15.39:0/4257766902 learned_addr learned my addr 172.21.15.39:0/4257766902 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57fad9d700 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 0x55749cd7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57fa59c700 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd74ef0 0x55749cd8b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 -- 172.21.15.39:0/4257766902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd74ef0 msgr2=0x55749cd8b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd74ef0 0x55749cd8b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 -- 172.21.15.39:0/4257766902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 msgr2=0x55749cd7c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 0x55749cd7c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 -- 172.21.15.39:0/4257766902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749cd6c470 con 0x55749cd75840 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd75840 0x55749cd8e320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57e40a66d0 tx=0x7f57e40a9fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daea6d7a3dccc5f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57e3fff700 1 -- 172.21.15.39:0/4257766902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40c3020 con 0x55749cd75840 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57e3fff700 1 -- 172.21.15.39:0/4257766902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57e40aa650 con 0x55749cd75840 2024-05-04T03:10:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.929+0000 7f57e3fff700 1 -- 172.21.15.39:0/4257766902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40aab10 con 0x55749cd75840 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 -- 172.21.15.39:0/4257766902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd75840 msgr2=0x55749cd8e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd75840 0x55749cd8e320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57e40a66d0 tx=0x7f57e40a9fa0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 -- 172.21.15.39:0/4257766902 shutdown_connections 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd7ada0 0x55749cd7c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd74ef0 0x55749cd8b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 --2- 172.21.15.39:0/4257766902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd75840 0x55749cd8e320 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 -- 172.21.15.39:0/4257766902 >> 172.21.15.39:0/4257766902 conn(0x55749cbf7df0 msgr2=0x55749cd69a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 -- 172.21.15.39:0/4257766902 shutdown_connections 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.930+0000 7f580f30cb80 1 -- 172.21.15.39:0/4257766902 wait complete. 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 1 -- start start 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 1 Processor -- start 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 0x55749cd7e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd7eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd98a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55749cd904a0 con 0x55749cd7ada0 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55749cd8fdd0 con 0x55749cd74ef0 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55749cd90b70 con 0x55749cd75840 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f580f30cb80 10 client.0 _ll_register_callbacks cb 0x55749cc2f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd98a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd98a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59368/0 (socket says 172.21.15.39:59368) 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f57fad9d700 1 -- 172.21.15.39:0/1658320993 learned_addr learned my addr 172.21.15.39:0/1658320993 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f57fa59c700 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 0x55749cd7e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.931+0000 7f57f9d9b700 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd7eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57fad9d700 1 -- 172.21.15.39:0/1658320993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 msgr2=0x55749cd7eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57fad9d700 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd7eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57fad9d700 1 -- 172.21.15.39:0/1658320993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 msgr2=0x55749cd7e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57fad9d700 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 0x55749cd7e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57fad9d700 1 -- 172.21.15.39:0/1658320993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749cd98fd0 con 0x55749cd7ada0 2024-05-04T03:10:55.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57fad9d700 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd98a90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005740 tx=0x7f57ec004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7a5f57263eec server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57e1ffb700 1 -- 172.21.15.39:0/1658320993 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00b950 con 0x55749cd7ada0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57e1ffb700 1 -- 172.21.15.39:0/1658320993 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f57ec004570 con 0x55749cd7ada0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f580f30cb80 1 -- 172.21.15.39:0/1658320993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e40aa650 con 0x55749cd7ada0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57e1ffb700 1 -- 172.21.15.39:0/1658320993 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f57ec00bd70 con 0x55749cd7ada0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f580f30cb80 10 client.15204 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f580f30cb80 1 -- 172.21.15.39:0/1658320993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55749cd9a1a0 con 0x55749cd7ada0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57e1ffb700 1 -- 172.21.15.39:0/1658320993 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57e40aa650 con 0x55749cd7ada0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.932+0000 7f57e1ffb700 1 -- 172.21.15.39:0/1658320993 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f57ec019050 con 0x55749cd7ada0 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f580f30cb80 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f580f30cb80 10 client.15204 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f580f30cb80 20 client.15204 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:55.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f580f30cb80 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55749cdfc5d0 0x55749cdfda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f580f30cb80 4 client.15204 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f580f30cb80 1 -- 172.21.15.39:0/1658320993 --> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] -- command(tid 0: {"prefix": "status"}) -- 0x55749cc9f290 con 0x55749cdfc5d0 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f57fa59c700 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55749cdfc5d0 0x55749cdfda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f57e1ffb700 10 client.15204 ms_handle_connect on v2:172.21.15.167:6838/975309166 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.933+0000 7f57fa59c700 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55749cdfc5d0 0x55749cdfda40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebff8bfffc87848 server_cookie=2cf827047bb3463a in_seq=0 out_seq=0 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f57e1ffb700 1 -- 172.21.15.39:0/1658320993 <== mds.1 v2:172.21.15.167:6838/975309166 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55749cc9f290 con 0x55749cdfc5d0 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f57e1ffb700 10 client.15204 handle_command_reply: tid=0 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 client.15204 shutdown 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 -- 172.21.15.39:0/1658320993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 msgr2=0x55749cd98a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd98a90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005740 tx=0x7f57ec004f80 comp rx=0 tx=0).stop 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 -- 172.21.15.39:0/1658320993 shutdown_connections 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55749cdfc5d0 0x55749cdfda40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55749cd7ada0 0x55749cd98a90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55749cd74ef0 0x55749cd7e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f57e1ffb700 0 client.15204 ms_handle_reset on v2:172.21.15.167:6838/975309166 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 --2- 172.21.15.39:0/1658320993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55749cd75840 0x55749cd7eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 -- 172.21.15.39:0/1658320993 >> 172.21.15.39:0/1658320993 conn(0x55749cbf7df0 msgr2=0x55749cd73ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 -- 172.21.15.39:0/1658320993 shutdown_connections 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 1 -- 172.21.15.39:0/1658320993 wait complete. 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.934+0000 7f580f30cb80 20 client.15204 trim_cache size 0 max 16384 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24490, 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:55.940 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 181.81000116000001, 2024-05-04T03:10:55.941 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:55.941 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:55.941 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:55.941 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 226.27314721900001 2024-05-04T03:10:55.941 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:55.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 -- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f57e80b2900 msgr2=0x7f57e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f57e80b2900 0x7f57e80b4e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009550 tx=0x7f57f40091b0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 -- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 msgr2=0x7f57fc19b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 0x7f57fc19b2d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f57e400b580 tx=0x7f57e4004790 comp rx=0 tx=0).stop 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 -- 172.21.15.39:0/681624680 shutdown_connections 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f57e80b2900 0x7f57e80b4e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57fc109860 0x7f57fc19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f57fc1045e0 0x7f57fc19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 --2- 172.21.15.39:0/681624680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57fc108f10 0x7f57fc19b2d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 -- 172.21.15.39:0/681624680 >> 172.21.15.39:0/681624680 conn(0x7f57fc0ffed0 msgr2=0x7f57fc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 -- 172.21.15.39:0/681624680 shutdown_connections 2024-05-04T03:10:55.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:55.935+0000 7f58008f8700 1 -- 172.21.15.39:0/681624680 wait complete. 2024-05-04T03:10:56.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-04T03:10:56.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:56 smithi066 ceph-mon[33264]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:56.865 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:56.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:56 smithi167 ceph-mon[33614]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:56.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:56 smithi039 ceph-mon[29715]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:10:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 -- 172.21.15.39:0/694508757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440105e40 msgr2=0x7fe44010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 --2- 172.21.15.39:0/694508757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440105e40 0x7fe44010e390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe428009990 tx=0x7fe428009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 -- 172.21.15.39:0/694508757 shutdown_connections 2024-05-04T03:10:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 --2- 172.21.15.39:0/694508757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe44010e920 0x7fe440110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 --2- 172.21.15.39:0/694508757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4401054f0 0x7fe440105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 --2- 172.21.15.39:0/694508757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440105e40 0x7fe44010e390 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 -- 172.21.15.39:0/694508757 >> 172.21.15.39:0/694508757 conn(0x7fe440078e00 msgr2=0x7fe440079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 -- 172.21.15.39:0/694508757 shutdown_connections 2024-05-04T03:10:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.644+0000 7fe446658700 1 -- 172.21.15.39:0/694508757 wait complete. 2024-05-04T03:10:58.648 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:10:58 smithi039 ceph-mon[29715]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:58.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.645+0000 7fe446658700 1 Processor -- start 2024-05-04T03:10:58.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.645+0000 7fe446658700 1 -- start start 2024-05-04T03:10:58.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.645+0000 7fe446658700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4401054f0 0x7fe44019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.645+0000 7fe446658700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440105e40 0x7fe44019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe446658700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44010e920 0x7fe44019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe446658700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401143b0 con 0x7fe4401054f0 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe446658700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe440114230 con 0x7fe44010e920 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe446658700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe440114530 con 0x7fe440105e40 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4401054f0 0x7fe44019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4401054f0 0x7fe44019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59398/0 (socket says 172.21.15.39:59398) 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 -- 172.21.15.39:0/3412451159 learned_addr learned my addr 172.21.15.39:0/3412451159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe444bf5700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44010e920 0x7fe44019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440105e40 0x7fe44019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 -- 172.21.15.39:0/3412451159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440105e40 msgr2=0x7fe44019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440105e40 0x7fe44019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 -- 172.21.15.39:0/3412451159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44010e920 msgr2=0x7fe44019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44010e920 0x7fe44019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 -- 172.21.15.39:0/3412451159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44019fc30 con 0x7fe4401054f0 2024-05-04T03:10:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe444bf5700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44010e920 0x7fe44019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4401054f0 0x7fe44019ad60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe43000d980 tx=0x7fe43000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3412451159 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430016070 con 0x7fe4401054f0 2024-05-04T03:10:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.646+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3412451159 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe430004020 con 0x7fe4401054f0 2024-05-04T03:10:58.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.647+0000 7fe446658700 1 -- 172.21.15.39:0/3412451159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44019ff20 con 0x7fe4401054f0 2024-05-04T03:10:58.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.647+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3412451159 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430004bb0 con 0x7fe4401054f0 2024-05-04T03:10:58.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.647+0000 7fe446658700 1 -- 172.21.15.39:0/3412451159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe440103110 con 0x7fe4401054f0 2024-05-04T03:10:58.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.650+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3412451159 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe430005020 con 0x7fe4401054f0 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.650+0000 7fe43d7fa700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe42c0b2880 0x7fe42c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.650+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe42c0b2880 0x7fe42c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.650+0000 7fe45506cb80 1 Processor -- start 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.650+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3412451159 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe4300e20b0 con 0x7fe4401054f0 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe45506cb80 1 -- start start 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bbe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bcadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe45506cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564462ae08f0 con 0x564462bc40d0 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe45506cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564462a45d50 con 0x564462bbe1c0 2024-05-04T03:10:58.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe45506cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564462adc8b0 con 0x564462bbeb10 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe42c0b2880 0x7fe42c0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe428006010 tx=0x7fe42800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59412/0 (socket says 172.21.15.39:59412) 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43ffff700 1 -- 172.21.15.39:0/615872489 learned_addr learned my addr 172.21.15.39:0/615872489 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe444bf5700 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bcadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bbe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43ffff700 1 -- 172.21.15.39:0/615872489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 msgr2=0x564462bcadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43ffff700 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bcadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43ffff700 1 -- 172.21.15.39:0/615872489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 msgr2=0x564462bbe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:58.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43ffff700 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bbe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43ffff700 1 -- 172.21.15.39:0/615872489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564462a2d880 con 0x564462bc40d0 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe444bf5700 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bcadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.651+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bbe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.652+0000 7fe43ffff700 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc44e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe4300a9670 tx=0x7fe430004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620ef51bffc1daec server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.652+0000 7fe4367fc700 1 -- 172.21.15.39:0/615872489 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300a96e0 con 0x564462bc40d0 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.652+0000 7fe4367fc700 1 -- 172.21.15.39:0/615872489 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4300ada90 con 0x564462bc40d0 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.652+0000 7fe4367fc700 1 -- 172.21.15.39:0/615872489 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43000e350 con 0x564462bc40d0 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 -- 172.21.15.39:0/615872489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 msgr2=0x564462bc44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc44e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe4300a9670 tx=0x7fe430004550 comp rx=0 tx=0).stop 2024-05-04T03:10:58.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 -- 172.21.15.39:0/615872489 shutdown_connections 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc44e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bcadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 --2- 172.21.15.39:0/615872489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bbe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 -- 172.21.15.39:0/615872489 >> 172.21.15.39:0/615872489 conn(0x564462a40df0 msgr2=0x564462bb9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 -- 172.21.15.39:0/615872489 shutdown_connections 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 -- 172.21.15.39:0/615872489 wait complete. 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 -- start start 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.653+0000 7fe45506cb80 1 Processor -- start 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bc82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bc8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe45506cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564462bcb3e0 con 0x564462bc40d0 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe45506cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564462a99e80 con 0x564462bbe1c0 2024-05-04T03:10:58.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe45506cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564462bcb560 con 0x564462bbeb10 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe45506cb80 10 client.0 _ll_register_callbacks cb 0x564462a78900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe43f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bc8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59418/0 (socket says 172.21.15.39:59418) 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 -- 172.21.15.39:0/3031429085 learned_addr learned my addr 172.21.15.39:0/3031429085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bc82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 -- 172.21.15.39:0/3031429085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 msgr2=0x564462bc8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bc8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 -- 172.21.15.39:0/3031429085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 msgr2=0x564462bc82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bc82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 -- 172.21.15.39:0/3031429085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564462a97710 con 0x564462bc40d0 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe444bf5700 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc8d50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe43800ac60 tx=0x7fe43800ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9374c3afdfe603ac server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:58.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.654+0000 7fe41f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe41f7fe700 1 -- 172.21.15.39:0/3031429085 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43801b060 con 0x564462bc40d0 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe41f7fe700 1 -- 172.21.15.39:0/3031429085 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe438007bb0 con 0x564462bc40d0 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 1 -- 172.21.15.39:0/3031429085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4300ada90 con 0x564462bc40d0 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe41f7fe700 1 -- 172.21.15.39:0/3031429085 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe438007db0 con 0x564462bc40d0 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 10 client.15216 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 1 -- 172.21.15.39:0/3031429085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564462bd6080 con 0x564462bc40d0 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe41f7fe700 1 -- 172.21.15.39:0/3031429085 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4300ada90 con 0x564462bc40d0 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe41f7fe700 1 -- 172.21.15.39:0/3031429085 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe438015030 con 0x564462bc40d0 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 20 client.15216 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564462bd8680 0x564462bd9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 4 client.15216 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.655+0000 7fe45506cb80 1 -- 172.21.15.39:0/3031429085 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564462aee2d0 con 0x564462bd8680 2024-05-04T03:10:58.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.656+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564462bd8680 0x564462bd9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.656+0000 7fe41f7fe700 10 client.15216 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.656+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564462bd8680 0x564462bd9af0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e43b27be3e87322 server_cookie=c4742d17bcab694b in_seq=0 out_seq=0 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe41f7fe700 1 -- 172.21.15.39:0/3031429085 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564462aee2d0 con 0x564462bd8680 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe41f7fe700 10 client.15216 handle_command_reply: tid=0 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 client.15216 shutdown 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 -- 172.21.15.39:0/3031429085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 msgr2=0x564462bc8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc8d50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe43800ac60 tx=0x7fe43800ac90 comp rx=0 tx=0).stop 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 -- 172.21.15.39:0/3031429085 shutdown_connections 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564462bd8680 0x564462bd9af0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc8d50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbe1c0 0x564462bc82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3031429085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbeb10 0x564462bc8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe45506cb80 1 -- 172.21.15.39:0/3031429085 >> 172.21.15.39:0/3031429085 conn(0x564462a40df0 msgr2=0x564462bbd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:58.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.657+0000 7fe41f7fe700 0 client.15216 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:10:58.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.658+0000 7fe45506cb80 1 -- 172.21.15.39:0/3031429085 shutdown_connections 2024-05-04T03:10:58.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.658+0000 7fe45506cb80 1 -- 172.21.15.39:0/3031429085 wait complete. 2024-05-04T03:10:58.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.658+0000 7fe45506cb80 20 client.15216 trim_cache size 0 max 16384 2024-05-04T03:10:58.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe45506cb80 1 Processor -- start 2024-05-04T03:10:58.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe45506cb80 1 -- start start 2024-05-04T03:10:58.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 0x564462bc7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbeb10 0x564462bc8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe45506cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564462bd7e30 con 0x564462bc40d0 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe45506cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564462bd8500 con 0x564462bbeb10 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe45506cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564462bd8bd0 con 0x564462bbe1c0 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.679+0000 7fe444bf5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe444bf5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59428/0 (socket says 172.21.15.39:59428) 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe444bf5700 1 -- 172.21.15.39:0/3647882242 learned_addr learned my addr 172.21.15.39:0/3647882242 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 0x564462bc7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbeb10 0x564462bc8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe444bf5700 1 -- 172.21.15.39:0/3647882242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 msgr2=0x564462bc7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe444bf5700 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 0x564462bc7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe444bf5700 1 -- 172.21.15.39:0/3647882242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbeb10 msgr2=0x564462bc8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe444bf5700 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbeb10 0x564462bc8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe444bf5700 1 -- 172.21.15.39:0/3647882242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564462bba8a0 con 0x564462bc40d0 2024-05-04T03:10:58.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe444bf5700 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc29f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe438009440 tx=0x7fe4380093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5e2f8c918f6348 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:58.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe435ffb700 1 -- 172.21.15.39:0/3647882242 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438007bb0 con 0x564462bc40d0 2024-05-04T03:10:58.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe435ffb700 1 -- 172.21.15.39:0/3647882242 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe438027070 con 0x564462bc40d0 2024-05-04T03:10:58.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.680+0000 7fe435ffb700 1 -- 172.21.15.39:0/3647882242 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380227d0 con 0x564462bc40d0 2024-05-04T03:10:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 -- 172.21.15.39:0/3647882242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 msgr2=0x564462bc29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc29f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe438009440 tx=0x7fe4380093d0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 -- 172.21.15.39:0/3647882242 shutdown_connections 2024-05-04T03:10:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bc40d0 0x564462bc29f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 0x564462bc7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 --2- 172.21.15.39:0/3647882242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bbeb10 0x564462bc8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 -- 172.21.15.39:0/3647882242 >> 172.21.15.39:0/3647882242 conn(0x564462a40df0 msgr2=0x564462bbd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 -- 172.21.15.39:0/3647882242 shutdown_connections 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.681+0000 7fe45506cb80 1 -- 172.21.15.39:0/3647882242 wait complete. 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 1 -- start start 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 1 Processor -- start 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 0x564462bdc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bbeb10 0x564462bdc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bc40d0 0x564462c41ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564462bd9970 con 0x564462bbeb10 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564462bd92a0 con 0x564462bc40d0 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564462bda040 con 0x564462bbe1c0 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe45506cb80 10 client.0 _ll_register_callbacks cb 0x564462a78900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:10:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe43f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bbeb10 0x564462bdc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe43f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bbeb10 0x564462bdc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59438/0 (socket says 172.21.15.39:59438) 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe43f7fe700 1 -- 172.21.15.39:0/2042201850 learned_addr learned my addr 172.21.15.39:0/2042201850 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.682+0000 7fe444bf5700 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bc40d0 0x564462c41ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe43ffff700 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 0x564462bdc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe43f7fe700 1 -- 172.21.15.39:0/2042201850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 msgr2=0x564462bdc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 0x564462bdc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe43f7fe700 1 -- 172.21.15.39:0/2042201850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bc40d0 msgr2=0x564462c41ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bc40d0 0x564462c41ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe43f7fe700 1 -- 172.21.15.39:0/2042201850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564462c42020 con 0x564462bbeb10 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe43f7fe700 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bbeb10 0x564462bdc6f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe428000c00 tx=0x7fe428009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc46c633e3fb674 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe41f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe41f7fe700 1 -- 172.21.15.39:0/2042201850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428034060 con 0x564462bbeb10 2024-05-04T03:10:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe41f7fe700 1 -- 172.21.15.39:0/2042201850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe42800fcb0 con 0x564462bbeb10 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe45506cb80 1 -- 172.21.15.39:0/2042201850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe438027070 con 0x564462bbeb10 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe41f7fe700 1 -- 172.21.15.39:0/2042201850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe42802f460 con 0x564462bbeb10 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe45506cb80 10 client.15222 fetch_fsmap learned FSMap version 27 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe45506cb80 1 -- 172.21.15.39:0/2042201850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564462c43170 con 0x564462bbeb10 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.683+0000 7fe41f7fe700 1 -- 172.21.15.39:0/2042201850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe438027070 con 0x564462bbeb10 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe41f7fe700 1 -- 172.21.15.39:0/2042201850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4280235d0 con 0x564462bbeb10 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe45506cb80 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe45506cb80 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe45506cb80 20 client.15222 populate_metadata read hostname 'smithi039' 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe45506cb80 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564462c45650 0x564462c46ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe45506cb80 4 client.15222 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe45506cb80 1 -- 172.21.15.39:0/2042201850 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "status"}) -- 0x564462ae8370 con 0x564462c45650 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe43ffff700 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564462c45650 0x564462c46ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe41f7fe700 10 client.15222 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.684+0000 7fe43ffff700 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564462c45650 0x564462c46ac0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32759210bfeff35 server_cookie=9899ec5be073ceab in_seq=0 out_seq=0 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe41f7fe700 1 -- 172.21.15.39:0/2042201850 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x564462ae8370 con 0x564462c45650 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe41f7fe700 10 client.15222 handle_command_reply: tid=0 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 client.15222 shutdown 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 -- 172.21.15.39:0/2042201850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bbeb10 msgr2=0x564462bdc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bbeb10 0x564462bdc6f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe428000c00 tx=0x7fe428009310 comp rx=0 tx=0).stop 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 -- 172.21.15.39:0/2042201850 shutdown_connections 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564462c45650 0x564462c46ac0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564462bc40d0 0x564462c41ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564462bbe1c0 0x564462bdc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 --2- 172.21.15.39:0/2042201850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564462bbeb10 0x564462bdc6f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 -- 172.21.15.39:0/2042201850 >> 172.21.15.39:0/2042201850 conn(0x564462a40df0 msgr2=0x564462bc9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe41f7fe700 0 client.15222 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:10:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 -- 172.21.15.39:0/2042201850 shutdown_connections 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 1 -- 172.21.15.39:0/2042201850 wait complete. 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.685+0000 7fe45506cb80 20 client.15222 trim_cache size 0 max 16384 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24506, 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:10:58.693 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:10:58.694 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 198.618411334, 2024-05-04T03:10:58.694 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 27, 2024-05-04T03:10:58.694 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:10:58.694 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-05-04T03:10:58.694 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 217.470874235 2024-05-04T03:10:58.694 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 -- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe42c0b2880 msgr2=0x7fe42c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe42c0b2880 0x7fe42c0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe428006010 tx=0x7fe42800b540 comp rx=0 tx=0).stop 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 -- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4401054f0 msgr2=0x7fe44019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4401054f0 0x7fe44019ad60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe43000d980 tx=0x7fe43000dd40 comp rx=0 tx=0).stop 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 -- 172.21.15.39:0/3412451159 shutdown_connections 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe42c0b2880 0x7fe42c0b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4401054f0 0x7fe44019ad60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe44010e920 0x7fe44019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 --2- 172.21.15.39:0/3412451159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440105e40 0x7fe44019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 -- 172.21.15.39:0/3412451159 >> 172.21.15.39:0/3412451159 conn(0x7fe440078e00 msgr2=0x7fe440100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 -- 172.21.15.39:0/3412451159 shutdown_connections 2024-05-04T03:10:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:10:58.686+0000 7fe446658700 1 -- 172.21.15.39:0/3412451159 wait complete. 2024-05-04T03:10:58.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:10:58 smithi066 ceph-mon[33264]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:58.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:10:58 smithi167 ceph-mon[33614]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:10:59.390 INFO:teuthology.run_tasks:Running task sequential... 2024-05-04T03:10:59.400 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-04T03:10:59.400 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-04T03:10:59.400 INFO:teuthology.task.print:Enabling mgr modules 2024-05-04T03:10:59.400 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-04T03:10:59.457 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-04T03:10:59.457 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-04T03:10:59.458 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-04T03:10:59.458 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-04T03:10:59.458 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-04T03:10:59.458 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-04T03:10:59.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:10:59.458 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-05-04T03:10:59.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:10:59.476 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-05-04T03:10:59.544 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2024-05-04T03:10:59.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:10:59.546 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-04T03:10:59.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-04T03:10:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:00 smithi066 ceph-mon[33264]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:11:00.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:00 smithi167 ceph-mon[33614]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:11:00.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:00 smithi039 ceph-mon[29715]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-04T03:11:01.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 -- 172.21.15.39:0/3476022296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed10108740 msgr2=0x7fed101100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 --2- 172.21.15.39:0/3476022296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed10108740 0x7fed101100a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed0800b0d0 tx=0x7fed0800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 -- 172.21.15.39:0/3476022296 shutdown_connections 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 --2- 172.21.15.39:0/3476022296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed10108740 0x7fed101100a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 --2- 172.21.15.39:0/3476022296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed101034c0 0x7fed101038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 --2- 172.21.15.39:0/3476022296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed10107df0 0x7fed10108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 -- 172.21.15.39:0/3476022296 >> 172.21.15.39:0/3476022296 conn(0x7fed100fedf0 msgr2=0x7fed10101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 -- 172.21.15.39:0/3476022296 shutdown_connections 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.614+0000 7fed17b80700 1 -- 172.21.15.39:0/3476022296 wait complete. 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed17b80700 1 Processor -- start 2024-05-04T03:11:01.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed17b80700 1 -- start start 2024-05-04T03:11:01.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed17b80700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed101034c0 0x7fed1019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:01.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed17b80700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed10107df0 0x7fed1019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:01.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed17b80700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed10108740 0x7fed10199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:01.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed17b80700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed10113270 con 0x7fed101034c0 2024-05-04T03:11:01.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed17b80700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fed101130f0 con 0x7fed10108740 2024-05-04T03:11:01.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed17b80700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed101133f0 con 0x7fed10107df0 2024-05-04T03:11:01.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1591c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed101034c0 0x7fed1019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:01.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1591c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed101034c0 0x7fed1019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59442/0 (socket says 172.21.15.39:59442) 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1591c700 1 -- 172.21.15.39:0/4088331850 learned_addr learned my addr 172.21.15.39:0/4088331850 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1511b700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed10107df0 0x7fed1019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1611d700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed10108740 0x7fed10199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1591c700 1 -- 172.21.15.39:0/4088331850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed10107df0 msgr2=0x7fed1019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1591c700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed10107df0 0x7fed1019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1591c700 1 -- 172.21.15.39:0/4088331850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed10108740 msgr2=0x7fed10199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1591c700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed10108740 0x7fed10199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.615+0000 7fed1591c700 1 -- 172.21.15.39:0/4088331850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1019a100 con 0x7fed101034c0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.616+0000 7fed1591c700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed101034c0 0x7fed1019f830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00d950 tx=0x7fed0c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.616+0000 7fed06ffd700 1 -- 172.21.15.39:0/4088331850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c016070 con 0x7fed101034c0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.616+0000 7fed06ffd700 1 -- 172.21.15.39:0/4088331850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed0c010460 con 0x7fed101034c0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.616+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1019a3f0 con 0x7fed101034c0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.616+0000 7fed06ffd700 1 -- 172.21.15.39:0/4088331850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00f620 con 0x7fed101034c0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.616+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed101a66c0 con 0x7fed101034c0 2024-05-04T03:11:01.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.617+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed100512c0 con 0x7fed101034c0 2024-05-04T03:11:01.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.618+0000 7fed06ffd700 1 -- 172.21.15.39:0/4088331850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fed0c019070 con 0x7fed101034c0 2024-05-04T03:11:01.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.619+0000 7fed06ffd700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fecfc0b2880 0x7fecfc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:01.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.619+0000 7fed1511b700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fecfc0b2880 0x7fecfc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:01.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.619+0000 7fed06ffd700 1 -- 172.21.15.39:0/4088331850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fed0c0e1e10 con 0x7fed101034c0 2024-05-04T03:11:01.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.620+0000 7fed1511b700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fecfc0b2880 0x7fecfc0b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed00005950 tx=0x7fed000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:01.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.623+0000 7fed06ffd700 1 -- 172.21.15.39:0/4088331850 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fed0c0a9b40 con 0x7fed101034c0 2024-05-04T03:11:01.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.881+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fed1002e5f0 con 0x7fed101034c0 2024-05-04T03:11:01.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.882+0000 7fed06ffd700 1 -- 172.21.15.39:0/4088331850 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fed0c0a9440 con 0x7fed101034c0 2024-05-04T03:11:01.884 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-04T03:11:01.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fecfc0b2880 msgr2=0x7fecfc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:01.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fecfc0b2880 0x7fecfc0b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed00005950 tx=0x7fed000058e0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed101034c0 msgr2=0x7fed1019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:01.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed101034c0 0x7fed1019f830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00d950 tx=0x7fed0c00dd10 comp rx=0 tx=0).stop 2024-05-04T03:11:01.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 shutdown_connections 2024-05-04T03:11:01.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fecfc0b2880 0x7fecfc0b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed101034c0 0x7fed1019f830 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed10108740 0x7fed10199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 --2- 172.21.15.39:0/4088331850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed10107df0 0x7fed1019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 >> 172.21.15.39:0/4088331850 conn(0x7fed100fedf0 msgr2=0x7fed101096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:01.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.884+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 shutdown_connections 2024-05-04T03:11:01.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:01.885+0000 7fed17b80700 1 -- 172.21.15.39:0/4088331850 wait complete. 2024-05-04T03:11:02.560 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-04T03:11:02.560 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-04T03:11:02.560 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-05-04T03:11:02.560 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-04T03:11:02.560 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-04T03:11:02.561 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-04T03:11:02.561 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-04T03:11:02.561 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-04T03:11:02.561 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-04T03:11:02.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:02.561 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-05-04T03:11:02.578 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T03:11:02.578 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T03:11:02.578 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T03:11:02.578 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-05-04T03:11:02.578 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42013sec preferred_lft 42013sec 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link noprefixroute 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-05-04T03:11:02.579 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-04T03:11:02.580 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-05-04T03:11:02.580 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-05-04T03:11:02.580 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-04T03:11:02.580 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:11:02.580 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-04T03:11:02.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-05-04T03:11:02.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-05-04T03:11:02.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-05-04T03:11:02.581 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-04T03:11:02.581 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-04T03:11:02.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:11:02.748 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:02 smithi039 ceph-mon[29715]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:11:02.748 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:02 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4088331850' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-04T03:11:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:11:02.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:02.769 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-04T03:11:02.793 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-05-04T03:11:02.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:02.796 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-05-04T03:11:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:02 smithi066 ceph-mon[33264]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:11:02.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:02 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4088331850' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-04T03:11:02.870 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-05-04T03:11:02.871 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-04T03:11:02.871 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-04T03:11:02.871 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-04T03:11:02.871 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-04T03:11:02.871 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:11:02.871 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-04T03:11:02.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-04T03:11:02.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-04T03:11:02.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-04T03:11:02.872 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-04T03:11:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:02 smithi167 ceph-mon[33614]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:11:02.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:02 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4088331850' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-04T03:11:02.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:11:03.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:11:03.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:03.151 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-05-04T03:11:03.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:03.167 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-05-04T03:11:03.222 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:11:03.222 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-04T03:11:03.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-04T03:11:03.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-04T03:11:03.223 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-04T03:11:03.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:11:03.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:11:03.341 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-04T03:11:03.341 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:11:03.341 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-04T03:11:03.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-04T03:11:03.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-04T03:11:03.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-04T03:11:03.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-04T03:11:03.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-04T03:11:03.342 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-04T03:11:03.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:11:03.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:11:03.466 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:11:03.466 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-05-04T03:11:03.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-05-04T03:11:03.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-05-04T03:11:03.466 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-05-04T03:11:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:11:03.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:11:03.546 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-04T03:11:03.546 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T03:11:03.546 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /tmp/cephfs 2024-05-04T03:11:03.561 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-04T03:11:03.561 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T03:11:03.561 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /tmp/cephfs 2024-05-04T03:11:03.612 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-05-04T03:11:03.677 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T03:11:03.732 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-05-04T03:11:03.732 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-05-04T03:11:03.732 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-05-04T03:11:03.732 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-05-04T03:11:03.732 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-05-04T03:11:03.733 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/42f716cb05c7508d089da719628e5df9f9fa2d386c55714bc0e56432e10c9816/merged 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/820d79538b097a07afce23d87bb442f347f3fd2da6e9e6a2e1a9c7816bf9eb92/merged 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/7ebb0a51266cd4ae6b2a2acd858509db62f01d1516522e8a654989cb4c58a5fb/merged 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/bab05c5cc0b1d365daea9dccdc3d6bc59d8fac2d4c954c07e4f1c53e40373ed4/merged 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/1809b4979cf35b1d1893ea7f0562a3ad6a0d165a124393ca848962ebdfa72db4/merged 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/b126bdfa8ba8016fabcbeac7cf91093a121cff284735b9203a1e2e799de81528/merged 2024-05-04T03:11:03.734 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c41bc43416b885fcf43b0909a26e42b4b37eb609951a161febce6af35b266ef2/merged 2024-05-04T03:11:03.735 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a5cd632f5e8d79c50f7f787a46844200fb51e6a8fd7304c98baddfbeb8314205/merged 2024-05-04T03:11:03.735 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/59ada88eaaa44c5e7f2f48f8697337962acb1d409e45b1fb67de8a996ea94451/merged 2024-05-04T03:11:03.735 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c3d1de78362cd0c4b9a70310efc1fa02dfac0816a9dd8f2d58ff2694a7685a76/merged 2024-05-04T03:11:03.735 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-05-04T03:11:03.735 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-04T03:11:03.735 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-04T03:11:03.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:03.736 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-05-04T03:11:03.791 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-04T03:11:03.791 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/e7a5b914-81ab-44ab-942c-66f330eaa990 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-05-04T03:11:03.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-05-04T03:11:03.859 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-05-04T03:11:03.916 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-05-04T03:11:03.917 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/42f716cb05c7508d089da719628e5df9f9fa2d386c55714bc0e56432e10c9816/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/820d79538b097a07afce23d87bb442f347f3fd2da6e9e6a2e1a9c7816bf9eb92/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/7ebb0a51266cd4ae6b2a2acd858509db62f01d1516522e8a654989cb4c58a5fb/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/bab05c5cc0b1d365daea9dccdc3d6bc59d8fac2d4c954c07e4f1c53e40373ed4/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/1809b4979cf35b1d1893ea7f0562a3ad6a0d165a124393ca848962ebdfa72db4/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/b126bdfa8ba8016fabcbeac7cf91093a121cff284735b9203a1e2e799de81528/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c41bc43416b885fcf43b0909a26e42b4b37eb609951a161febce6af35b266ef2/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a5cd632f5e8d79c50f7f787a46844200fb51e6a8fd7304c98baddfbeb8314205/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/59ada88eaaa44c5e7f2f48f8697337962acb1d409e45b1fb67de8a996ea94451/merged 2024-05-04T03:11:03.918 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c3d1de78362cd0c4b9a70310efc1fa02dfac0816a9dd8f2d58ff2694a7685a76/merged 2024-05-04T03:11:03.919 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-05-04T03:11:03.919 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-04T03:11:03.919 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-04T03:11:03.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:03.919 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-05-04T03:11:03.949 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:2024-05-04T03:11:03.947+0000 7f3513f94480 -1 init, newargv = 0x7f3508004b60 newargc=17 2024-05-04T03:11:03.949 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[85071]: starting ceph client 2024-05-04T03:11:03.981 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[85071]: starting fuse 2024-05-04T03:11:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:04 smithi066 ceph-mon[33264]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:11:04.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:04 smithi167 ceph-mon[33614]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:11:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:04 smithi039 ceph-mon[29715]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:11:05.065 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-05-04T03:11:05.097 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-05-04T03:11:05.156 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysusers.service 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-05-04T03:11:05.157 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/42f716cb05c7508d089da719628e5df9f9fa2d386c55714bc0e56432e10c9816/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/820d79538b097a07afce23d87bb442f347f3fd2da6e9e6a2e1a9c7816bf9eb92/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/7ebb0a51266cd4ae6b2a2acd858509db62f01d1516522e8a654989cb4c58a5fb/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/bab05c5cc0b1d365daea9dccdc3d6bc59d8fac2d4c954c07e4f1c53e40373ed4/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/1809b4979cf35b1d1893ea7f0562a3ad6a0d165a124393ca848962ebdfa72db4/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/b126bdfa8ba8016fabcbeac7cf91093a121cff284735b9203a1e2e799de81528/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c41bc43416b885fcf43b0909a26e42b4b37eb609951a161febce6af35b266ef2/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a5cd632f5e8d79c50f7f787a46844200fb51e6a8fd7304c98baddfbeb8314205/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/59ada88eaaa44c5e7f2f48f8697337962acb1d409e45b1fb67de8a996ea94451/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c3d1de78362cd0c4b9a70310efc1fa02dfac0816a9dd8f2d58ff2694a7685a76/merged 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-04T03:11:05.158 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-04T03:11:05.159 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs 2024-05-04T03:11:05.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:05.159 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-05-04T03:11:05.217 INFO:teuthology.orchestra.run.smithi039.stdout:115 2024-05-04T03:11:05.217 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-05-04T03:11:05.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo stdin-killer -- python3 -c ' 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> import glob 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> import re 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> import os 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> import subprocess 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> def _find_admin_socket(client_name): 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> files = glob.glob(asok_path) 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> mountpoint = "/tmp/cephfs" 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> # Given a non-glob path, it better be there 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> if "*" not in asok_path: 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> assert(len(files) == 1) 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> return files[0] 2024-05-04T03:11:05.218 DEBUG:teuthology.orchestra.run.smithi039:> 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> for f in files: 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> contents = proc_f.read() 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> if mountpoint in contents: 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> return f 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> print(_find_admin_socket("client.0")) 2024-05-04T03:11:05.219 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-05-04T03:11:05.345 INFO:teuthology.orchestra.run.smithi039.stdout:/var/run/ceph/ceph-client.0.85071.asok 2024-05-04T03:11:05.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:11:05.356 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.85071.asok 2024-05-04T03:11:05.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:05.356 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.85071.asok status 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "metadata": { 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_sha1": "bd4e00a39d766328b458e4c953f4d722fed96a31", 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "entity_id": "0", 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": "smithi039", 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "mount_point": "/tmp/cephfs", 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "pid": "85071", 2024-05-04T03:11:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "root": "/volumes/qa/sv_0/e7a5b914-81ab-44ab-942c-66f330eaa990" 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_count": 0, 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_pinned_count": 0, 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 15234, 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "inst": { 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": { 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "client", 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "num": 15234 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3010807843 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:11:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3010807843 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "inst_str": "client.15234 192.168.0.1:0/3010807843", 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "addr_str": "192.168.0.1:0/3010807843", 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "inode_count": 5, 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_epoch": 27, 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch": 80, 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch_barrier": 0, 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklisted": false, 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs" 2024-05-04T03:11:05.506 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:11:05.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:05.514 DEBUG:teuthology.orchestra.run.smithi039:> stat --file-system '--printf=%T 2024-05-04T03:11:05.514 DEBUG:teuthology.orchestra.run.smithi039:> ' -- /tmp/cephfs 2024-05-04T03:11:05.530 INFO:teuthology.orchestra.run.smithi039.stdout:fuseblk 2024-05-04T03:11:05.530 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-05-04T03:11:05.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:11:05.530 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /tmp/cephfs 2024-05-04T03:11:05.597 INFO:teuthology.run_tasks:Running task exec... 2024-05-04T03:11:05.608 INFO:teuthology.task.exec:Executing custom commands... 2024-05-04T03:11:05.608 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T03:11:05.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-04T03:11:05.658 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:05 smithi039 ceph-mon[29715]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:11:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 -- 172.21.15.39:0/4178011324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 msgr2=0x7f19e4103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 --2- 172.21.15.39:0/4178011324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 0x7f19e4103840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009900 tx=0x7f19d402a3f0 comp rx=0 tx=0).stop 2024-05-04T03:11:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 -- 172.21.15.39:0/4178011324 shutdown_connections 2024-05-04T03:11:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 --2- 172.21.15.39:0/4178011324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e4108a80 0x7f19e410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:05.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 --2- 172.21.15.39:0/4178011324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e4103d80 0x7f19e4104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 --2- 172.21.15.39:0/4178011324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 0x7f19e4103840 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 -- 172.21.15.39:0/4178011324 >> 172.21.15.39:0/4178011324 conn(0x7f19e40fdd30 msgr2=0x7f19e41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 -- 172.21.15.39:0/4178011324 shutdown_connections 2024-05-04T03:11:05.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.749+0000 7f19e91d6640 1 -- 172.21.15.39:0/4178011324 wait complete. 2024-05-04T03:11:05.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e91d6640 1 Processor -- start 2024-05-04T03:11:05.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e91d6640 1 -- start start 2024-05-04T03:11:05.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e91d6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 0x7f19e4075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:05.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e91d6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e4103d80 0x7f19e419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e91d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e4108a80 0x7f19e41a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e91d6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410e4d0 con 0x7f19e4102b40 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e91d6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410e350 con 0x7f19e4108a80 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e91d6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410e650 con 0x7f19e4103d80 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 0x7f19e4075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 0x7f19e4075990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57972/0 (socket says 172.21.15.39:57972) 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e2d9d640 1 -- 172.21.15.39:0/3351523576 learned_addr learned my addr 172.21.15.39:0/3351523576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e359e640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e4108a80 0x7f19e41a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.750+0000 7f19e259c640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e4103d80 0x7f19e419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e2d9d640 1 -- 172.21.15.39:0/3351523576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e4103d80 msgr2=0x7f19e419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e2d9d640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e4103d80 0x7f19e419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:05.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e2d9d640 1 -- 172.21.15.39:0/3351523576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e4108a80 msgr2=0x7f19e41a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e2d9d640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e4108a80 0x7f19e41a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e2d9d640 1 -- 172.21.15.39:0/3351523576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e41a1d50 con 0x7f19e4102b40 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e259c640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e4103d80 0x7f19e419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e2d9d640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 0x7f19e4075990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f19d4005760 tx=0x7f19d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19dbfff640 1 -- 172.21.15.39:0/3351523576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d403f030 con 0x7f19e4102b40 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e41a1fe0 con 0x7f19e4102b40 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19dbfff640 1 -- 172.21.15.39:0/3351523576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19d4003a80 con 0x7f19e4102b40 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19dbfff640 1 -- 172.21.15.39:0/3351523576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4035070 con 0x7f19e4102b40 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.751+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41a2440 con 0x7f19e4102b40 2024-05-04T03:11:05.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.752+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e4107890 con 0x7f19e4102b40 2024-05-04T03:11:05.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.754+0000 7f19dbfff640 1 -- 172.21.15.39:0/3351523576 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f19d4042030 con 0x7f19e4102b40 2024-05-04T03:11:05.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.754+0000 7f19dbfff640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19d00b27e0 0x7f19d00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:05.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.754+0000 7f19e259c640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19d00b27e0 0x7f19d00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:05.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.754+0000 7f19dbfff640 1 -- 172.21.15.39:0/3351523576 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f19d41026d0 con 0x7f19e4102b40 2024-05-04T03:11:05.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.755+0000 7f19e259c640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19d00b27e0 0x7f19d00b4cf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19cc005fd0 tx=0x7f19cc005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.759+0000 7f19dbfff640 1 -- 172.21.15.39:0/3351523576 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19d40ca400 con 0x7f19e4102b40 2024-05-04T03:11:05.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:05 smithi066 ceph-mon[33264]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:11:05.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:05 smithi167 ceph-mon[33614]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:11:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:05.991+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f19e402dc30 con 0x7f19e4102b40 2024-05-04T03:11:06.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.520+0000 7f19dbfff640 1 -- 172.21.15.39:0/3351523576 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f19d40c9d00 con 0x7f19e4102b40 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19d00b27e0 msgr2=0x7f19d00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19d00b27e0 0x7f19d00b4cf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19cc005fd0 tx=0x7f19cc005f40 comp rx=0 tx=0).stop 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 msgr2=0x7f19e4075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 0x7f19e4075990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f19d4005760 tx=0x7f19d4004270 comp rx=0 tx=0).stop 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 shutdown_connections 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19d00b27e0 0x7f19d00b4cf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19e4108a80 0x7f19e41a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19e4103d80 0x7f19e419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 --2- 172.21.15.39:0/3351523576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19e4102b40 0x7f19e4075990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 >> 172.21.15.39:0/3351523576 conn(0x7f19e40fdd30 msgr2=0x7f19e4104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 shutdown_connections 2024-05-04T03:11:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.524+0000 7f19e91d6640 1 -- 172.21.15.39:0/3351523576 wait complete. 2024-05-04T03:11:06.538 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 -- 172.21.15.39:0/2534662781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600102b50 msgr2=0x7f1600103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 --2- 172.21.15.39:0/2534662781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600102b50 0x7f1600103850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009900 tx=0x7f15fc02a410 comp rx=0 tx=0).stop 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 -- 172.21.15.39:0/2534662781 shutdown_connections 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 --2- 172.21.15.39:0/2534662781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1600108a90 0x7f160010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 --2- 172.21.15.39:0/2534662781 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1600103d90 0x7f16001041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 --2- 172.21.15.39:0/2534662781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600102b50 0x7f1600103850 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 -- 172.21.15.39:0/2534662781 >> 172.21.15.39:0/2534662781 conn(0x7f16000fdd40 msgr2=0x7f16001001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 -- 172.21.15.39:0/2534662781 shutdown_connections 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.697+0000 7f16087b2640 1 -- 172.21.15.39:0/2534662781 wait complete. 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f16087b2640 1 Processor -- start 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f16087b2640 1 -- start start 2024-05-04T03:11:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f16087b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1600102b50 0x7f16001991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f16087b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600103d90 0x7f1600199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f16087b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1600108a90 0x7f16001a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f16087b2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f160010e500 con 0x7f1600103d90 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f16087b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f160010e380 con 0x7f1600102b50 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f16087b2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f160010e680 con 0x7f1600108a90 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f1605d4d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600103d90 0x7f1600199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f1605d4d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600103d90 0x7f1600199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57998/0 (socket says 172.21.15.39:57998) 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f1605d4d640 1 -- 172.21.15.39:0/3826930323 learned_addr learned my addr 172.21.15.39:0/3826930323 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.698+0000 7f160654e640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1600102b50 0x7f16001991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f1605d4d640 1 -- 172.21.15.39:0/3826930323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1600108a90 msgr2=0x7f16001a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f1605d4d640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1600108a90 0x7f16001a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f1605d4d640 1 -- 172.21.15.39:0/3826930323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1600102b50 msgr2=0x7f16001991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f1605d4d640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1600102b50 0x7f16001991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f1605d4d640 1 -- 172.21.15.39:0/3826930323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16001a0d00 con 0x7f1600103d90 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f1605d4d640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600103d90 0x7f1600199700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f15f000d950 tx=0x7f15f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f15f77fe640 1 -- 172.21.15.39:0/3826930323 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0017070 con 0x7f1600103d90 2024-05-04T03:11:06.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f15f77fe640 1 -- 172.21.15.39:0/3826930323 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15f0009c30 con 0x7f1600103d90 2024-05-04T03:11:06.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f15f77fe640 1 -- 172.21.15.39:0/3826930323 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f000f620 con 0x7f1600103d90 2024-05-04T03:11:06.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16001a0ff0 con 0x7f1600103d90 2024-05-04T03:11:06.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.699+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16001a14b0 con 0x7f1600103d90 2024-05-04T03:11:06.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.700+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16001078a0 con 0x7f1600103d90 2024-05-04T03:11:06.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.702+0000 7f15f77fe640 1 -- 172.21.15.39:0/3826930323 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f15f0005030 con 0x7f1600103d90 2024-05-04T03:11:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.703+0000 7f15f77fe640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f15ec0b2860 0x7f15ec0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.703+0000 7f160654e640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f15ec0b2860 0x7f15ec0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.703+0000 7f15f77fe640 1 -- 172.21.15.39:0/3826930323 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f15f00e1db0 con 0x7f1600103d90 2024-05-04T03:11:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.704+0000 7f160654e640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f15ec0b2860 0x7f15ec0b4d70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005760 tx=0x7f15fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:06.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.707+0000 7f15f77fe640 1 -- 172.21.15.39:0/3826930323 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15f00a9b60 con 0x7f1600103d90 2024-05-04T03:11:06.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:06 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3351523576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-04T03:11:06.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:06 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3351523576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-04T03:11:06.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.927+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1600063270 con 0x7f15ec0b2860 2024-05-04T03:11:06.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:06 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3351523576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-04T03:11:06.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.946+0000 7f15f77fe640 1 -- 172.21.15.39:0/3826930323 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f1600063270 con 0x7f15ec0b2860 2024-05-04T03:11:06.947 INFO:teuthology.orchestra.run.smithi039.stdout:{} 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f15ec0b2860 msgr2=0x7f15ec0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f15ec0b2860 0x7f15ec0b4d70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005760 tx=0x7f15fc000bc0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600103d90 msgr2=0x7f1600199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600103d90 0x7f1600199700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f15f000d950 tx=0x7f15f000de10 comp rx=0 tx=0).stop 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 shutdown_connections 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f15ec0b2860 0x7f15ec0b4d70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1600108a90 0x7f16001a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600103d90 0x7f1600199700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 --2- 172.21.15.39:0/3826930323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1600102b50 0x7f16001991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 >> 172.21.15.39:0/3826930323 conn(0x7f16000fdd40 msgr2=0x7f1600104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 shutdown_connections 2024-05-04T03:11:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:06.949+0000 7f16087b2640 1 -- 172.21.15.39:0/3826930323 wait complete. 2024-05-04T03:11:06.961 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-04T03:11:06.971 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-04T03:11:07.001 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T03:11:07.001 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-04T03:11:07.004 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-04T03:11:07.036 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-04T03:11:07.037 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:11:07.191 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:11:06 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:11:06.932+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:11:07.191 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:11:06 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:11:06.932+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:11:07.191 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:11:06 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:11:06.932+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:11:07.191 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:11:06 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:11:06.932+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:11:07.191 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:11:06 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x[29943]: 2024-05-04T03:11:06.932+0000 7fc030d80700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:11:07.287 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:07.671 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:07 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3351523576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-04T03:11:07.671 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:07 smithi039 ceph-mon[29715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:07.671 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:07 smithi039 ceph-mon[29715]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:07.671 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:07 smithi039 ceph-mon[29715]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-04T03:11:07.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:07 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3351523576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-04T03:11:07.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:07 smithi066 ceph-mon[33264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:07.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:07 smithi066 ceph-mon[33264]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:07.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:07 smithi066 ceph-mon[33264]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-04T03:11:07.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:07 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3351523576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-04T03:11:07.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:07 smithi167 ceph-mon[33614]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:07.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:07 smithi167 ceph-mon[33614]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:07.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:07 smithi167 ceph-mon[33614]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-04T03:11:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:08 smithi039 ceph-mon[29715]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-04T03:11:08.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:08 smithi066 ceph-mon[33264]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-04T03:11:08.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:08 smithi167 ceph-mon[33614]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-04T03:11:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 -- 172.21.15.39:0/132569509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe280108f00 msgr2=0x7fe280109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:09.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 --2- 172.21.15.39:0/132569509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe280108f00 0x7fe280109310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe2740099c0 tx=0x7fe274009cd0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 -- 172.21.15.39:0/132569509 shutdown_connections 2024-05-04T03:11:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 --2- 172.21.15.39:0/132569509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe280109850 0x7fe2801111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 --2- 172.21.15.39:0/132569509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2801045d0 0x7fe2801049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 --2- 172.21.15.39:0/132569509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe280108f00 0x7fe280109310 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 -- 172.21.15.39:0/132569509 >> 172.21.15.39:0/132569509 conn(0x7fe2800ffea0 msgr2=0x7fe2801022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 -- 172.21.15.39:0/132569509 shutdown_connections 2024-05-04T03:11:09.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.087+0000 7fe285c87700 1 -- 172.21.15.39:0/132569509 wait complete. 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.088+0000 7fe285c87700 1 Processor -- start 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.088+0000 7fe285c87700 1 -- start start 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe285c87700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2801045d0 0x7fe28019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe285c87700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe280108f00 0x7fe28019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe285c87700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe280109850 0x7fe28019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe285c87700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe280114380 con 0x7fe2801045d0 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe285c87700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe280114200 con 0x7fe280109850 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe285c87700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe280114500 con 0x7fe280108f00 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2801045d0 0x7fe28019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2801045d0 0x7fe28019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58030/0 (socket says 172.21.15.39:58030) 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 -- 172.21.15.39:0/2206647747 learned_addr learned my addr 172.21.15.39:0/2206647747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27effd700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe280108f00 0x7fe28019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27ffff700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe280109850 0x7fe28019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 -- 172.21.15.39:0/2206647747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe280108f00 msgr2=0x7fe28019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe280108f00 0x7fe28019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 -- 172.21.15.39:0/2206647747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe280109850 msgr2=0x7fe28019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe280109850 0x7fe28019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 -- 172.21.15.39:0/2206647747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28019fc60 con 0x7fe2801045d0 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.089+0000 7fe27f7fe700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2801045d0 0x7fe28019ad90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe27000b530 tx=0x7fe27000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.090+0000 7fe27cff9700 1 -- 172.21.15.39:0/2206647747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270015020 con 0x7fe2801045d0 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.090+0000 7fe27cff9700 1 -- 172.21.15.39:0/2206647747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe270011cc0 con 0x7fe2801045d0 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.090+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28019ff50 con 0x7fe2801045d0 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.090+0000 7fe27cff9700 1 -- 172.21.15.39:0/2206647747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270010c80 con 0x7fe2801045d0 2024-05-04T03:11:09.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.090+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2801a7870 con 0x7fe2801045d0 2024-05-04T03:11:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.091+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2800512c0 con 0x7fe2801045d0 2024-05-04T03:11:09.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.092+0000 7fe27cff9700 1 -- 172.21.15.39:0/2206647747 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fe270018020 con 0x7fe2801045d0 2024-05-04T03:11:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.093+0000 7fe27cff9700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe2680b2910 0x7fe2680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.093+0000 7fe27cff9700 1 -- 172.21.15.39:0/2206647747 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe2700e1f00 con 0x7fe2801045d0 2024-05-04T03:11:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.093+0000 7fe27effd700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe2680b2910 0x7fe2680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.094+0000 7fe27effd700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe2680b2910 0x7fe2680b4e20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe274009990 tx=0x7fe274005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.097+0000 7fe27cff9700 1 -- 172.21.15.39:0/2206647747 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe27001f020 con 0x7fe2801045d0 2024-05-04T03:11:09.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.304+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe28019c0f0 con 0x7fe2680b2910 2024-05-04T03:11:09.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.313+0000 7fe27cff9700 1 -- 172.21.15.39:0/2206647747 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259048 (secure 0 0 0) 0x7fe274067400 con 0x7fe2680b2910 2024-05-04T03:11:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe2680b2910 msgr2=0x7fe2680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe2680b2910 0x7fe2680b4e20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe274009990 tx=0x7fe274005fb0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2801045d0 msgr2=0x7fe28019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2801045d0 0x7fe28019ad90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe27000b530 tx=0x7fe27000b840 comp rx=0 tx=0).stop 2024-05-04T03:11:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 shutdown_connections 2024-05-04T03:11:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe2680b2910 0x7fe2680b4e20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2801045d0 0x7fe28019ad90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe280109850 0x7fe28019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 --2- 172.21.15.39:0/2206647747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe280108f00 0x7fe28019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 >> 172.21.15.39:0/2206647747 conn(0x7fe2800ffea0 msgr2=0x7fe2801022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 shutdown_connections 2024-05-04T03:11:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:09.316+0000 7fe285c87700 1 -- 172.21.15.39:0/2206647747 wait complete. 2024-05-04T03:11:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-04T03:11:09.616 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:09 smithi039 ceph-mon[29715]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-04T03:11:09.616 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:09 smithi039 ceph-mon[29715]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:09.817 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:09 smithi066 ceph-mon[33264]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-04T03:11:09.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:09 smithi066 ceph-mon[33264]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:09.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:09 smithi167 ceph-mon[33614]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-04T03:11:09.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:09 smithi167 ceph-mon[33614]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:10.023 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-05-04T03:11:09.052768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660928,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2011,"num_read_kb":1109,"num_write":395,"num_write_kb":2307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":824,"ondisk_log_size":824,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376600,"kb_used_data":10808,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565224,"statfs":{"total":1151940427776,"available":1151554789376,"internally_reserved":0,"allocated":11067392,"data_stored":7086276,"data_compressed":109203,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2776,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":65,"num_write":19,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003435"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998775+0000","last_change":"2024-05-04T03:07:37.070347+0000","last_active":"2024-05-04T03:09:44.998775+0000","last_peered":"2024-05-04T03:09:44.998775+0000","last_clean":"2024-05-04T03:09:44.998775+0000","last_became_active":"2024-05-04T03:07:37.069997+0000","last_became_peered":"2024-05-04T03:07:37.069997+0000","last_unstale":"2024-05-04T03:09:44.998775+0000","last_undegraded":"2024-05-04T03:09:44.998775+0000","last_fullsized":"2024-05-04T03:09:44.998775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:24:11.362228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010264+0000","last_change":"2024-05-04T03:07:37.071598+0000","last_active":"2024-05-04T03:09:45.010264+0000","last_peered":"2024-05-04T03:09:45.010264+0000","last_clean":"2024-05-04T03:09:45.010264+0000","last_became_active":"2024-05-04T03:07:37.071059+0000","last_became_peered":"2024-05-04T03:07:37.071059+0000","last_unstale":"2024-05-04T03:09:45.010264+0000","last_undegraded":"2024-05-04T03:09:45.010264+0000","last_fullsized":"2024-05-04T03:09:45.010264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:33:33.929246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017072+0000","last_change":"2024-05-04T03:07:37.069951+0000","last_active":"2024-05-04T03:09:45.017072+0000","last_peered":"2024-05-04T03:09:45.017072+0000","last_clean":"2024-05-04T03:09:45.017072+0000","last_became_active":"2024-05-04T03:07:37.069558+0000","last_became_peered":"2024-05-04T03:07:37.069558+0000","last_unstale":"2024-05-04T03:09:45.017072+0000","last_undegraded":"2024-05-04T03:09:45.017072+0000","last_fullsized":"2024-05-04T03:09:45.017072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:37:52.596455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.001811+0000","last_change":"2024-05-04T03:07:37.068820+0000","last_active":"2024-05-04T03:09:45.001811+0000","last_peered":"2024-05-04T03:09:45.001811+0000","last_clean":"2024-05-04T03:09:45.001811+0000","last_became_active":"2024-05-04T03:07:37.068323+0000","last_became_peered":"2024-05-04T03:07:37.068323+0000","last_unstale":"2024-05-04T03:09:45.001811+0000","last_undegraded":"2024-05-04T03:09:45.001811+0000","last_fullsized":"2024-05-04T03:09:45.001811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:52:54.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.981848+0000","last_change":"2024-05-04T03:07:37.061982+0000","last_active":"2024-05-04T03:09:44.981848+0000","last_peered":"2024-05-04T03:09:44.981848+0000","last_clean":"2024-05-04T03:09:44.981848+0000","last_became_active":"2024-05-04T03:07:37.061621+0000","last_became_peered":"2024-05-04T03:07:37.061621+0000","last_unstale":"2024-05-04T03:09:44.981848+0000","last_undegraded":"2024-05-04T03:09:44.981848+0000","last_fullsized":"2024-05-04T03:09:44.981848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:17:47.421011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003074+0000","last_change":"2024-05-04T03:07:37.058641+0000","last_active":"2024-05-04T03:09:45.003074+0000","last_peered":"2024-05-04T03:09:45.003074+0000","last_clean":"2024-05-04T03:09:45.003074+0000","last_became_active":"2024-05-04T03:07:37.058235+0000","last_became_peered":"2024-05-04T03:07:37.058235+0000","last_unstale":"2024-05-04T03:09:45.003074+0000","last_undegraded":"2024-05-04T03:09:45.003074+0000","last_fullsized":"2024-05-04T03:09:45.003074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:17:45.441777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998905+0000","last_change":"2024-05-04T03:07:37.065959+0000","last_active":"2024-05-04T03:09:44.998905+0000","last_peered":"2024-05-04T03:09:44.998905+0000","last_clean":"2024-05-04T03:09:44.998905+0000","last_became_active":"2024-05-04T03:07:37.065508+0000","last_became_peered":"2024-05-04T03:07:37.065508+0000","last_unstale":"2024-05-04T03:09:44.998905+0000","last_undegraded":"2024-05-04T03:09:44.998905+0000","last_fullsized":"2024-05-04T03:09:44.998905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:17.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994908+0000","last_change":"2024-05-04T03:07:37.065620+0000","last_active":"2024-05-04T03:09:44.994908+0000","last_peered":"2024-05-04T03:09:44.994908+0000","last_clean":"2024-05-04T03:09:44.994908+0000","last_became_active":"2024-05-04T03:07:37.065261+0000","last_became_peered":"2024-05-04T03:07:37.065261+0000","last_unstale":"2024-05-04T03:09:44.994908+0000","last_undegraded":"2024-05-04T03:09:44.994908+0000","last_fullsized":"2024-05-04T03:09:44.994908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:27.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017270+0000","last_change":"2024-05-04T03:07:37.072204+0000","last_active":"2024-05-04T03:09:45.017270+0000","last_peered":"2024-05-04T03:09:45.017270+0000","last_clean":"2024-05-04T03:09:45.017270+0000","last_became_active":"2024-05-04T03:07:37.070622+0000","last_became_peered":"2024-05-04T03:07:37.070622+0000","last_unstale":"2024-05-04T03:09:45.017270+0000","last_undegraded":"2024-05-04T03:09:45.017270+0000","last_fullsized":"2024-05-04T03:09:45.017270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:55:25.931369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003159+0000","last_change":"2024-05-04T03:08:16.253125+0000","last_active":"2024-05-04T03:09:45.003159+0000","last_peered":"2024-05-04T03:09:45.003159+0000","last_clean":"2024-05-04T03:09:45.003159+0000","last_became_active":"2024-05-04T03:08:16.252682+0000","last_became_peered":"2024-05-04T03:08:16.252682+0000","last_unstale":"2024-05-04T03:09:45.003159+0000","last_undegraded":"2024-05-04T03:09:45.003159+0000","last_fullsized":"2024-05-04T03:09:45.003159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:05.933491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.015463+0000","last_change":"2024-05-04T03:07:37.071614+0000","last_active":"2024-05-04T03:09:45.015463+0000","last_peered":"2024-05-04T03:09:45.015463+0000","last_clean":"2024-05-04T03:09:45.015463+0000","last_became_active":"2024-05-04T03:07:37.070927+0000","last_became_peered":"2024-05-04T03:07:37.070927+0000","last_unstale":"2024-05-04T03:09:45.015463+0000","last_undegraded":"2024-05-04T03:09:45.015463+0000","last_fullsized":"2024-05-04T03:09:45.015463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:30.432959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005280+0000","last_change":"2024-05-04T03:07:37.069305+0000","last_active":"2024-05-04T03:09:45.005280+0000","last_peered":"2024-05-04T03:09:45.005280+0000","last_clean":"2024-05-04T03:09:45.005280+0000","last_became_active":"2024-05-04T03:07:37.068804+0000","last_became_peered":"2024-05-04T03:07:37.068804+0000","last_unstale":"2024-05-04T03:09:45.005280+0000","last_undegraded":"2024-05-04T03:09:45.005280+0000","last_fullsized":"2024-05-04T03:09:45.005280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:43.693161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.006446+0000","last_change":"2024-05-04T03:07:37.062848+0000","last_active":"2024-05-04T03:09:45.006446+0000","last_peered":"2024-05-04T03:09:45.006446+0000","last_clean":"2024-05-04T03:09:45.006446+0000","last_became_active":"2024-05-04T03:07:37.062548+0000","last_became_peered":"2024-05-04T03:07:37.062548+0000","last_unstale":"2024-05-04T03:09:45.006446+0000","last_undegraded":"2024-05-04T03:09:45.006446+0000","last_fullsized":"2024-05-04T03:09:45.006446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010679+0000","last_change":"2024-05-04T03:07:37.062598+0000","last_active":"2024-05-04T03:09:45.010679+0000","last_peered":"2024-05-04T03:09:45.010679+0000","last_clean":"2024-05-04T03:09:45.010679+0000","last_became_active":"2024-05-04T03:07:37.061711+0000","last_became_peered":"2024-05-04T03:07:37.061711+0000","last_unstale":"2024-05-04T03:09:45.010679+0000","last_undegraded":"2024-05-04T03:09:45.010679+0000","last_fullsized":"2024-05-04T03:09:45.010679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:29:03.153680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007786+0000","last_change":"2024-05-04T03:07:37.067889+0000","last_active":"2024-05-04T03:09:45.007786+0000","last_peered":"2024-05-04T03:09:45.007786+0000","last_clean":"2024-05-04T03:09:45.007786+0000","last_became_active":"2024-05-04T03:07:37.066768+0000","last_became_peered":"2024-05-04T03:07:37.066768+0000","last_unstale":"2024-05-04T03:09:45.007786+0000","last_undegraded":"2024-05-04T03:09:45.007786+0000","last_fullsized":"2024-05-04T03:09:45.007786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:01:18.136379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017415+0000","last_change":"2024-05-04T03:07:37.070438+0000","last_active":"2024-05-04T03:09:45.017415+0000","last_peered":"2024-05-04T03:09:45.017415+0000","last_clean":"2024-05-04T03:09:45.017415+0000","last_became_active":"2024-05-04T03:07:37.070007+0000","last_became_peered":"2024-05-04T03:07:37.070007+0000","last_unstale":"2024-05-04T03:09:45.017415+0000","last_undegraded":"2024-05-04T03:09:45.017415+0000","last_fullsized":"2024-05-04T03:09:45.017415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:22.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999016+0000","last_change":"2024-05-04T03:07:37.070020+0000","last_active":"2024-05-04T03:09:44.999016+0000","last_peered":"2024-05-04T03:09:44.999016+0000","last_clean":"2024-05-04T03:09:44.999016+0000","last_became_active":"2024-05-04T03:07:37.069629+0000","last_became_peered":"2024-05-04T03:07:37.069629+0000","last_unstale":"2024-05-04T03:09:44.999016+0000","last_undegraded":"2024-05-04T03:09:44.999016+0000","last_fullsized":"2024-05-04T03:09:44.999016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:28.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.011076+0000","last_change":"2024-05-04T03:07:37.068651+0000","last_active":"2024-05-04T03:09:45.011076+0000","last_peered":"2024-05-04T03:09:45.011076+0000","last_clean":"2024-05-04T03:09:45.011076+0000","last_became_active":"2024-05-04T03:07:37.067135+0000","last_became_peered":"2024-05-04T03:07:37.067135+0000","last_unstale":"2024-05-04T03:09:45.011076+0000","last_undegraded":"2024-05-04T03:09:45.011076+0000","last_fullsized":"2024-05-04T03:09:45.011076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:00:59.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994607+0000","last_change":"2024-05-04T03:07:37.055419+0000","last_active":"2024-05-04T03:09:44.994607+0000","last_peered":"2024-05-04T03:09:44.994607+0000","last_clean":"2024-05-04T03:09:44.994607+0000","last_became_active":"2024-05-04T03:07:37.055024+0000","last_became_peered":"2024-05-04T03:07:37.055024+0000","last_unstale":"2024-05-04T03:09:44.994607+0000","last_undegraded":"2024-05-04T03:09:44.994607+0000","last_fullsized":"2024-05-04T03:09:44.994607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:55:05.506505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":232,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:07.481589+0000","last_change":"2024-05-04T03:07:34.019975+0000","last_active":"2024-05-04T03:11:07.481589+0000","last_peered":"2024-05-04T03:11:07.481589+0000","last_clean":"2024-05-04T03:11:07.481589+0000","last_became_active":"2024-05-04T03:07:34.019532+0000","last_became_peered":"2024-05-04T03:07:34.019532+0000","last_unstale":"2024-05-04T03:11:07.481589+0000","last_undegraded":"2024-05-04T03:11:07.481589+0000","last_fullsized":"2024-05-04T03:11:07.481589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:11:03.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.990313+0000","last_change":"2024-05-04T03:07:37.064725+0000","last_active":"2024-05-04T03:09:44.990313+0000","last_peered":"2024-05-04T03:09:44.990313+0000","last_clean":"2024-05-04T03:09:44.990313+0000","last_became_active":"2024-05-04T03:07:37.064385+0000","last_became_peered":"2024-05-04T03:07:37.064385+0000","last_unstale":"2024-05-04T03:09:44.990313+0000","last_undegraded":"2024-05-04T03:09:44.990313+0000","last_fullsized":"2024-05-04T03:09:44.990313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:22:12.024328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244201+0000","last_change":"2024-05-04T03:07:34.157179+0000","last_active":"2024-05-04T03:08:16.244201+0000","last_peered":"2024-05-04T03:08:16.244201+0000","last_clean":"2024-05-04T03:08:16.244201+0000","last_became_active":"2024-05-04T03:07:34.156790+0000","last_became_peered":"2024-05-04T03:07:34.156790+0000","last_unstale":"2024-05-04T03:08:16.244201+0000","last_undegraded":"2024-05-04T03:08:16.244201+0000","last_fullsized":"2024-05-04T03:08:16.244201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:32.066410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017111+0000","last_change":"2024-05-04T03:07:37.070481+0000","last_active":"2024-05-04T03:09:45.017111+0000","last_peered":"2024-05-04T03:09:45.017111+0000","last_clean":"2024-05-04T03:09:45.017111+0000","last_became_active":"2024-05-04T03:07:37.069903+0000","last_became_peered":"2024-05-04T03:07:37.069903+0000","last_unstale":"2024-05-04T03:09:45.017111+0000","last_undegraded":"2024-05-04T03:09:45.017111+0000","last_fullsized":"2024-05-04T03:09:45.017111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:45:40.764716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245782+0000","last_change":"2024-05-04T03:07:34.023978+0000","last_active":"2024-05-04T03:08:16.245782+0000","last_peered":"2024-05-04T03:08:16.245782+0000","last_clean":"2024-05-04T03:08:16.245782+0000","last_became_active":"2024-05-04T03:07:34.023199+0000","last_became_peered":"2024-05-04T03:07:34.023199+0000","last_unstale":"2024-05-04T03:08:16.245782+0000","last_undegraded":"2024-05-04T03:08:16.245782+0000","last_fullsized":"2024-05-04T03:08:16.245782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:17:43.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017273+0000","last_change":"2024-05-04T03:07:37.072661+0000","last_active":"2024-05-04T03:09:45.017273+0000","last_peered":"2024-05-04T03:09:45.017273+0000","last_clean":"2024-05-04T03:09:45.017273+0000","last_became_active":"2024-05-04T03:07:37.071393+0000","last_became_peered":"2024-05-04T03:07:37.071393+0000","last_unstale":"2024-05-04T03:09:45.017273+0000","last_undegraded":"2024-05-04T03:09:45.017273+0000","last_fullsized":"2024-05-04T03:09:45.017273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:39:39.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246044+0000","last_change":"2024-05-04T03:07:34.018543+0000","last_active":"2024-05-04T03:08:16.246044+0000","last_peered":"2024-05-04T03:08:16.246044+0000","last_clean":"2024-05-04T03:08:16.246044+0000","last_became_active":"2024-05-04T03:07:34.018145+0000","last_became_peered":"2024-05-04T03:07:34.018145+0000","last_unstale":"2024-05-04T03:08:16.246044+0000","last_undegraded":"2024-05-04T03:08:16.246044+0000","last_fullsized":"2024-05-04T03:08:16.246044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:44.529492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007771+0000","last_change":"2024-05-04T03:07:37.070811+0000","last_active":"2024-05-04T03:09:45.007771+0000","last_peered":"2024-05-04T03:09:45.007771+0000","last_clean":"2024-05-04T03:09:45.007771+0000","last_became_active":"2024-05-04T03:07:37.070441+0000","last_became_peered":"2024-05-04T03:07:37.070441+0000","last_unstale":"2024-05-04T03:09:45.007771+0000","last_undegraded":"2024-05-04T03:09:45.007771+0000","last_fullsized":"2024-05-04T03:09:45.007771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:01:48.146339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":224,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:04.458834+0000","last_change":"2024-05-04T03:07:34.020508+0000","last_active":"2024-05-04T03:11:04.458834+0000","last_peered":"2024-05-04T03:11:04.458834+0000","last_clean":"2024-05-04T03:11:04.458834+0000","last_became_active":"2024-05-04T03:07:34.020076+0000","last_became_peered":"2024-05-04T03:07:34.020076+0000","last_unstale":"2024-05-04T03:11:04.458834+0000","last_undegraded":"2024-05-04T03:11:04.458834+0000","last_fullsized":"2024-05-04T03:11:04.458834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:27.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1099,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.988466+0000","last_change":"2024-05-04T03:07:37.060004+0000","last_active":"2024-05-04T03:09:44.988466+0000","last_peered":"2024-05-04T03:09:44.988466+0000","last_clean":"2024-05-04T03:09:44.988466+0000","last_became_active":"2024-05-04T03:07:37.059350+0000","last_became_peered":"2024-05-04T03:07:37.059350+0000","last_unstale":"2024-05-04T03:09:44.988466+0000","last_undegraded":"2024-05-04T03:09:44.988466+0000","last_fullsized":"2024-05-04T03:09:44.988466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:37:07.315935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248143+0000","last_change":"2024-05-04T03:07:34.020174+0000","last_active":"2024-05-04T03:08:16.248143+0000","last_peered":"2024-05-04T03:08:16.248143+0000","last_clean":"2024-05-04T03:08:16.248143+0000","last_became_active":"2024-05-04T03:07:34.019790+0000","last_became_peered":"2024-05-04T03:07:34.019790+0000","last_unstale":"2024-05-04T03:08:16.248143+0000","last_undegraded":"2024-05-04T03:08:16.248143+0000","last_fullsized":"2024-05-04T03:08:16.248143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:37:37.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.987213+0000","last_change":"2024-05-04T03:08:16.251277+0000","last_active":"2024-05-04T03:09:44.987213+0000","last_peered":"2024-05-04T03:09:44.987213+0000","last_clean":"2024-05-04T03:09:44.987213+0000","last_became_active":"2024-05-04T03:08:16.250836+0000","last_became_peered":"2024-05-04T03:08:16.250836+0000","last_unstale":"2024-05-04T03:09:44.987213+0000","last_undegraded":"2024-05-04T03:09:44.987213+0000","last_fullsized":"2024-05-04T03:09:44.987213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:59:47.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248454+0000","last_change":"2024-05-04T03:07:34.013188+0000","last_active":"2024-05-04T03:08:16.248454+0000","last_peered":"2024-05-04T03:08:16.248454+0000","last_clean":"2024-05-04T03:08:16.248454+0000","last_became_active":"2024-05-04T03:07:34.012759+0000","last_became_peered":"2024-05-04T03:07:34.012759+0000","last_unstale":"2024-05-04T03:08:16.248454+0000","last_undegraded":"2024-05-04T03:08:16.248454+0000","last_fullsized":"2024-05-04T03:08:16.248454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:54:29.621250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.004186+0000","last_change":"2024-05-04T03:07:37.056265+0000","last_active":"2024-05-04T03:09:45.004186+0000","last_peered":"2024-05-04T03:09:45.004186+0000","last_clean":"2024-05-04T03:09:45.004186+0000","last_became_active":"2024-05-04T03:07:37.055957+0000","last_became_peered":"2024-05-04T03:07:37.055957+0000","last_unstale":"2024-05-04T03:09:45.004186+0000","last_undegraded":"2024-05-04T03:09:45.004186+0000","last_fullsized":"2024-05-04T03:09:45.004186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:41.739034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247922+0000","last_change":"2024-05-04T03:07:34.020722+0000","last_active":"2024-05-04T03:08:16.247922+0000","last_peered":"2024-05-04T03:08:16.247922+0000","last_clean":"2024-05-04T03:08:16.247922+0000","last_became_active":"2024-05-04T03:07:34.020028+0000","last_became_peered":"2024-05-04T03:07:34.020028+0000","last_unstale":"2024-05-04T03:08:16.247922+0000","last_undegraded":"2024-05-04T03:08:16.247922+0000","last_fullsized":"2024-05-04T03:08:16.247922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:23:20.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998984+0000","last_change":"2024-05-04T03:07:37.059680+0000","last_active":"2024-05-04T03:09:44.998984+0000","last_peered":"2024-05-04T03:09:44.998984+0000","last_clean":"2024-05-04T03:09:44.998984+0000","last_became_active":"2024-05-04T03:07:37.059294+0000","last_became_peered":"2024-05-04T03:07:37.059294+0000","last_unstale":"2024-05-04T03:09:44.998984+0000","last_undegraded":"2024-05-04T03:09:44.998984+0000","last_fullsized":"2024-05-04T03:09:44.998984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:50:06.251706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":409,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:05.459731+0000","last_change":"2024-05-04T03:07:34.019055+0000","last_active":"2024-05-04T03:11:05.459731+0000","last_peered":"2024-05-04T03:11:05.459731+0000","last_clean":"2024-05-04T03:11:05.459731+0000","last_became_active":"2024-05-04T03:07:34.018638+0000","last_became_peered":"2024-05-04T03:07:34.018638+0000","last_unstale":"2024-05-04T03:11:05.459731+0000","last_undegraded":"2024-05-04T03:11:05.459731+0000","last_fullsized":"2024-05-04T03:11:05.459731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:58:01.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":376,"num_read_kb":376,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017350+0000","last_change":"2024-05-04T03:07:37.063037+0000","last_active":"2024-05-04T03:09:45.017350+0000","last_peered":"2024-05-04T03:09:45.017350+0000","last_clean":"2024-05-04T03:09:45.017350+0000","last_became_active":"2024-05-04T03:07:37.062350+0000","last_became_peered":"2024-05-04T03:07:37.062350+0000","last_unstale":"2024-05-04T03:09:45.017350+0000","last_undegraded":"2024-05-04T03:09:45.017350+0000","last_fullsized":"2024-05-04T03:09:45.017350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:41:15.898241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.374402+0000","last_change":"2024-05-04T03:08:16.373765+0000","last_active":"2024-05-04T03:08:16.374402+0000","last_peered":"2024-05-04T03:08:16.374402+0000","last_clean":"2024-05-04T03:08:16.374402+0000","last_became_active":"2024-05-04T03:08:16.368588+0000","last_became_peered":"2024-05-04T03:08:16.368588+0000","last_unstale":"2024-05-04T03:08:16.374402+0000","last_undegraded":"2024-05-04T03:08:16.374402+0000","last_fullsized":"2024-05-04T03:08:16.374402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:55:07.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007709+0000","last_change":"2024-05-04T03:07:37.071330+0000","last_active":"2024-05-04T03:09:45.007709+0000","last_peered":"2024-05-04T03:09:45.007709+0000","last_clean":"2024-05-04T03:09:45.007709+0000","last_became_active":"2024-05-04T03:07:37.070743+0000","last_became_peered":"2024-05-04T03:07:37.070743+0000","last_unstale":"2024-05-04T03:09:45.007709+0000","last_undegraded":"2024-05-04T03:09:45.007709+0000","last_fullsized":"2024-05-04T03:09:45.007709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:10:00.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248881+0000","last_change":"2024-05-04T03:07:34.157432+0000","last_active":"2024-05-04T03:08:16.248881+0000","last_peered":"2024-05-04T03:08:16.248881+0000","last_clean":"2024-05-04T03:08:16.248881+0000","last_became_active":"2024-05-04T03:07:34.156976+0000","last_became_peered":"2024-05-04T03:07:34.156976+0000","last_unstale":"2024-05-04T03:08:16.248881+0000","last_undegraded":"2024-05-04T03:08:16.248881+0000","last_fullsized":"2024-05-04T03:08:16.248881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:56.082287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017872+0000","last_change":"2024-05-04T03:07:37.064301+0000","last_active":"2024-05-04T03:09:45.017872+0000","last_peered":"2024-05-04T03:09:45.017872+0000","last_clean":"2024-05-04T03:09:45.017872+0000","last_became_active":"2024-05-04T03:07:37.063691+0000","last_became_peered":"2024-05-04T03:07:37.063691+0000","last_unstale":"2024-05-04T03:09:45.017872+0000","last_undegraded":"2024-05-04T03:09:45.017872+0000","last_fullsized":"2024-05-04T03:09:45.017872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:22:13.844703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":450,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:05.459458+0000","last_change":"2024-05-04T03:07:34.021595+0000","last_active":"2024-05-04T03:11:05.459458+0000","last_peered":"2024-05-04T03:11:05.459458+0000","last_clean":"2024-05-04T03:11:05.459458+0000","last_became_active":"2024-05-04T03:07:34.021126+0000","last_became_peered":"2024-05-04T03:07:34.021126+0000","last_unstale":"2024-05-04T03:11:05.459458+0000","last_undegraded":"2024-05-04T03:11:05.459458+0000","last_fullsized":"2024-05-04T03:11:05.459458+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:55.094800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61073,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":387,"num_read_kb":253,"num_write":38,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999219+0000","last_change":"2024-05-04T03:07:37.066365+0000","last_active":"2024-05-04T03:09:44.999219+0000","last_peered":"2024-05-04T03:09:44.999219+0000","last_clean":"2024-05-04T03:09:44.999219+0000","last_became_active":"2024-05-04T03:07:37.065851+0000","last_became_peered":"2024-05-04T03:07:37.065851+0000","last_unstale":"2024-05-04T03:09:44.999219+0000","last_undegraded":"2024-05-04T03:09:44.999219+0000","last_fullsized":"2024-05-04T03:09:44.999219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:03:43.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.518904+0000","last_change":"2024-05-04T03:08:16.262600+0000","last_active":"2024-05-04T03:08:16.518904+0000","last_peered":"2024-05-04T03:08:16.518904+0000","last_clean":"2024-05-04T03:08:16.518904+0000","last_became_active":"2024-05-04T03:08:16.250619+0000","last_became_peered":"2024-05-04T03:08:16.250619+0000","last_unstale":"2024-05-04T03:08:16.518904+0000","last_undegraded":"2024-05-04T03:08:16.518904+0000","last_fullsized":"2024-05-04T03:08:16.518904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:51:21.596467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002712+0000","last_change":"2024-05-04T03:07:37.055356+0000","last_active":"2024-05-04T03:09:45.002712+0000","last_peered":"2024-05-04T03:09:45.002712+0000","last_clean":"2024-05-04T03:09:45.002712+0000","last_became_active":"2024-05-04T03:07:37.055045+0000","last_became_peered":"2024-05-04T03:07:37.055045+0000","last_unstale":"2024-05-04T03:09:45.002712+0000","last_undegraded":"2024-05-04T03:09:45.002712+0000","last_fullsized":"2024-05-04T03:09:45.002712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:06.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247445+0000","last_change":"2024-05-04T03:07:34.021525+0000","last_active":"2024-05-04T03:08:16.247445+0000","last_peered":"2024-05-04T03:08:16.247445+0000","last_clean":"2024-05-04T03:08:16.247445+0000","last_became_active":"2024-05-04T03:07:34.021042+0000","last_became_peered":"2024-05-04T03:07:34.021042+0000","last_unstale":"2024-05-04T03:08:16.247445+0000","last_undegraded":"2024-05-04T03:08:16.247445+0000","last_fullsized":"2024-05-04T03:08:16.247445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:44.090176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.004521+0000","last_change":"2024-05-04T03:07:37.066471+0000","last_active":"2024-05-04T03:09:45.004521+0000","last_peered":"2024-05-04T03:09:45.004521+0000","last_clean":"2024-05-04T03:09:45.004521+0000","last_became_active":"2024-05-04T03:07:37.066059+0000","last_became_peered":"2024-05-04T03:07:37.066059+0000","last_unstale":"2024-05-04T03:09:45.004521+0000","last_undegraded":"2024-05-04T03:09:45.004521+0000","last_fullsized":"2024-05-04T03:09:45.004521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:16.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'8","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:06.464575+0000","last_change":"2024-05-04T03:07:34.021531+0000","last_active":"2024-05-04T03:11:06.464575+0000","last_peered":"2024-05-04T03:11:06.464575+0000","last_clean":"2024-05-04T03:11:06.464575+0000","last_became_active":"2024-05-04T03:07:34.021112+0000","last_became_peered":"2024-05-04T03:07:34.021112+0000","last_unstale":"2024-05-04T03:11:06.464575+0000","last_undegraded":"2024-05-04T03:11:06.464575+0000","last_fullsized":"2024-05-04T03:11:06.464575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:28.826426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1597,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996662+0000","last_change":"2024-05-04T03:07:37.067155+0000","last_active":"2024-05-04T03:09:44.996662+0000","last_peered":"2024-05-04T03:09:44.996662+0000","last_clean":"2024-05-04T03:09:44.996662+0000","last_became_active":"2024-05-04T03:07:37.066783+0000","last_became_peered":"2024-05-04T03:07:37.066783+0000","last_unstale":"2024-05-04T03:09:44.996662+0000","last_undegraded":"2024-05-04T03:09:44.996662+0000","last_fullsized":"2024-05-04T03:09:44.996662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:23.217930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245106+0000","last_change":"2024-05-04T03:07:34.158227+0000","last_active":"2024-05-04T03:08:16.245106+0000","last_peered":"2024-05-04T03:08:16.245106+0000","last_clean":"2024-05-04T03:08:16.245106+0000","last_became_active":"2024-05-04T03:07:34.156905+0000","last_became_peered":"2024-05-04T03:07:34.156905+0000","last_unstale":"2024-05-04T03:08:16.245106+0000","last_undegraded":"2024-05-04T03:08:16.245106+0000","last_fullsized":"2024-05-04T03:08:16.245106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:40:41.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.006787+0000","last_change":"2024-05-04T03:07:37.075559+0000","last_active":"2024-05-04T03:09:45.006787+0000","last_peered":"2024-05-04T03:09:45.006787+0000","last_clean":"2024-05-04T03:09:45.006787+0000","last_became_active":"2024-05-04T03:07:37.075220+0000","last_became_peered":"2024-05-04T03:07:37.075220+0000","last_unstale":"2024-05-04T03:09:45.006787+0000","last_undegraded":"2024-05-04T03:09:45.006787+0000","last_fullsized":"2024-05-04T03:09:45.006787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:51:23.942514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:52.084062+0000","last_change":"2024-05-04T03:07:34.020326+0000","last_active":"2024-05-04T03:09:52.084062+0000","last_peered":"2024-05-04T03:09:52.084062+0000","last_clean":"2024-05-04T03:09:52.084062+0000","last_became_active":"2024-05-04T03:07:34.019472+0000","last_became_peered":"2024-05-04T03:07:34.019472+0000","last_unstale":"2024-05-04T03:09:52.084062+0000","last_undegraded":"2024-05-04T03:09:52.084062+0000","last_fullsized":"2024-05-04T03:09:52.084062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:33:28.375901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002619+0000","last_change":"2024-05-04T03:07:37.066223+0000","last_active":"2024-05-04T03:09:45.002619+0000","last_peered":"2024-05-04T03:09:45.002619+0000","last_clean":"2024-05-04T03:09:45.002619+0000","last_became_active":"2024-05-04T03:07:37.065859+0000","last_became_peered":"2024-05-04T03:07:37.065859+0000","last_unstale":"2024-05-04T03:09:45.002619+0000","last_undegraded":"2024-05-04T03:09:45.002619+0000","last_fullsized":"2024-05-04T03:09:45.002619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:44.902986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245539+0000","last_change":"2024-05-04T03:07:34.158674+0000","last_active":"2024-05-04T03:08:16.245539+0000","last_peered":"2024-05-04T03:08:16.245539+0000","last_clean":"2024-05-04T03:08:16.245539+0000","last_became_active":"2024-05-04T03:07:34.157952+0000","last_became_peered":"2024-05-04T03:07:34.157952+0000","last_unstale":"2024-05-04T03:08:16.245539+0000","last_undegraded":"2024-05-04T03:08:16.245539+0000","last_fullsized":"2024-05-04T03:08:16.245539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:19:11.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005664+0000","last_change":"2024-05-04T03:07:37.053897+0000","last_active":"2024-05-04T03:09:45.005664+0000","last_peered":"2024-05-04T03:09:45.005664+0000","last_clean":"2024-05-04T03:09:45.005664+0000","last_became_active":"2024-05-04T03:07:37.053466+0000","last_became_peered":"2024-05-04T03:07:37.053466+0000","last_unstale":"2024-05-04T03:09:45.005664+0000","last_undegraded":"2024-05-04T03:09:45.005664+0000","last_fullsized":"2024-05-04T03:09:45.005664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:50.650805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":224,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:05.478212+0000","last_change":"2024-05-04T03:07:34.021494+0000","last_active":"2024-05-04T03:11:05.478212+0000","last_peered":"2024-05-04T03:11:05.478212+0000","last_clean":"2024-05-04T03:11:05.478212+0000","last_became_active":"2024-05-04T03:07:34.021113+0000","last_became_peered":"2024-05-04T03:07:34.021113+0000","last_unstale":"2024-05-04T03:11:05.478212+0000","last_undegraded":"2024-05-04T03:11:05.478212+0000","last_fullsized":"2024-05-04T03:11:05.478212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:02:09.514961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007408+0000","last_change":"2024-05-04T03:07:37.068199+0000","last_active":"2024-05-04T03:09:45.007408+0000","last_peered":"2024-05-04T03:09:45.007408+0000","last_clean":"2024-05-04T03:09:45.007408+0000","last_became_active":"2024-05-04T03:07:37.067828+0000","last_became_peered":"2024-05-04T03:07:37.067828+0000","last_unstale":"2024-05-04T03:09:45.007408+0000","last_undegraded":"2024-05-04T03:09:45.007408+0000","last_fullsized":"2024-05-04T03:09:45.007408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:28.882920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996821+0000","last_change":"2024-05-04T03:07:37.069753+0000","last_active":"2024-05-04T03:09:44.996821+0000","last_peered":"2024-05-04T03:09:44.996821+0000","last_clean":"2024-05-04T03:09:44.996821+0000","last_became_active":"2024-05-04T03:07:37.069338+0000","last_became_peered":"2024-05-04T03:07:37.069338+0000","last_unstale":"2024-05-04T03:09:44.996821+0000","last_undegraded":"2024-05-04T03:09:44.996821+0000","last_fullsized":"2024-05-04T03:09:44.996821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:37:11.355749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'7","reported_seq":224,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:05.462880+0000","last_change":"2024-05-04T03:07:34.157229+0000","last_active":"2024-05-04T03:11:05.462880+0000","last_peered":"2024-05-04T03:11:05.462880+0000","last_clean":"2024-05-04T03:11:05.462880+0000","last_became_active":"2024-05-04T03:07:34.156776+0000","last_became_peered":"2024-05-04T03:07:34.156776+0000","last_unstale":"2024-05-04T03:11:05.462880+0000","last_undegraded":"2024-05-04T03:11:05.462880+0000","last_fullsized":"2024-05-04T03:11:05.462880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:26.879754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010488+0000","last_change":"2024-05-04T03:07:37.067025+0000","last_active":"2024-05-04T03:09:45.010488+0000","last_peered":"2024-05-04T03:09:45.010488+0000","last_clean":"2024-05-04T03:09:45.010488+0000","last_became_active":"2024-05-04T03:07:37.066359+0000","last_became_peered":"2024-05-04T03:07:37.066359+0000","last_unstale":"2024-05-04T03:09:45.010488+0000","last_undegraded":"2024-05-04T03:09:45.010488+0000","last_fullsized":"2024-05-04T03:09:45.010488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:44:35.604859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367184+0000","last_change":"2024-05-04T03:07:34.018019+0000","last_active":"2024-05-04T03:08:16.367184+0000","last_peered":"2024-05-04T03:08:16.367184+0000","last_clean":"2024-05-04T03:08:16.367184+0000","last_became_active":"2024-05-04T03:07:34.017556+0000","last_became_peered":"2024-05-04T03:07:34.017556+0000","last_unstale":"2024-05-04T03:08:16.367184+0000","last_undegraded":"2024-05-04T03:08:16.367184+0000","last_fullsized":"2024-05-04T03:08:16.367184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:31:34.158164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:52.260319+0000","last_change":"2024-05-04T03:06:06.704755+0000","last_active":"2024-05-04T03:08:52.260319+0000","last_peered":"2024-05-04T03:08:52.260319+0000","last_clean":"2024-05-04T03:08:52.260319+0000","last_became_active":"2024-05-04T03:06:06.698747+0000","last_became_peered":"2024-05-04T03:06:06.698747+0000","last_unstale":"2024-05-04T03:08:52.260319+0000","last_undegraded":"2024-05-04T03:08:52.260319+0000","last_fullsized":"2024-05-04T03:08:52.260319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_clean_scrub_stamp":"2024-05-04T03:02:43.385377+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:04:50.012414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008902+0000","last_change":"2024-05-04T03:07:37.057043+0000","last_active":"2024-05-04T03:09:45.008902+0000","last_peered":"2024-05-04T03:09:45.008902+0000","last_clean":"2024-05-04T03:09:45.008902+0000","last_became_active":"2024-05-04T03:07:37.056725+0000","last_became_peered":"2024-05-04T03:07:37.056725+0000","last_unstale":"2024-05-04T03:09:45.008902+0000","last_undegraded":"2024-05-04T03:09:45.008902+0000","last_fullsized":"2024-05-04T03:09:45.008902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:13:28.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.377000+0000","last_change":"2024-05-04T03:08:16.376510+0000","last_active":"2024-05-04T03:08:16.377000+0000","last_peered":"2024-05-04T03:08:16.377000+0000","last_clean":"2024-05-04T03:08:16.377000+0000","last_became_active":"2024-05-04T03:08:16.368625+0000","last_became_peered":"2024-05-04T03:08:16.368625+0000","last_unstale":"2024-05-04T03:08:16.377000+0000","last_undegraded":"2024-05-04T03:08:16.377000+0000","last_fullsized":"2024-05-04T03:08:16.377000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:27:50.504487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245652+0000","last_change":"2024-05-04T03:07:34.157346+0000","last_active":"2024-05-04T03:08:16.245652+0000","last_peered":"2024-05-04T03:08:16.245652+0000","last_clean":"2024-05-04T03:08:16.245652+0000","last_became_active":"2024-05-04T03:07:34.156914+0000","last_became_peered":"2024-05-04T03:07:34.156914+0000","last_unstale":"2024-05-04T03:08:16.245652+0000","last_undegraded":"2024-05-04T03:08:16.245652+0000","last_fullsized":"2024-05-04T03:08:16.245652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:54:17.059644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.987131+0000","last_change":"2024-05-04T03:07:37.054373+0000","last_active":"2024-05-04T03:09:44.987131+0000","last_peered":"2024-05-04T03:09:44.987131+0000","last_clean":"2024-05-04T03:09:44.987131+0000","last_became_active":"2024-05-04T03:07:37.053764+0000","last_became_peered":"2024-05-04T03:07:37.053764+0000","last_unstale":"2024-05-04T03:09:44.987131+0000","last_undegraded":"2024-05-04T03:09:44.987131+0000","last_fullsized":"2024-05-04T03:09:44.987131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:13:47.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'18","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:04.104348+0000","last_change":"2024-05-04T03:07:34.020190+0000","last_active":"2024-05-04T03:11:04.104348+0000","last_peered":"2024-05-04T03:11:04.104348+0000","last_clean":"2024-05-04T03:11:04.104348+0000","last_became_active":"2024-05-04T03:07:34.019786+0000","last_became_peered":"2024-05-04T03:07:34.019786+0000","last_unstale":"2024-05-04T03:11:04.104348+0000","last_undegraded":"2024-05-04T03:11:04.104348+0000","last_fullsized":"2024-05-04T03:11:04.104348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:11.243414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999034+0000","last_change":"2024-05-04T03:07:37.055274+0000","last_active":"2024-05-04T03:09:44.999034+0000","last_peered":"2024-05-04T03:09:44.999034+0000","last_clean":"2024-05-04T03:09:44.999034+0000","last_became_active":"2024-05-04T03:07:37.054968+0000","last_became_peered":"2024-05-04T03:07:37.054968+0000","last_unstale":"2024-05-04T03:09:44.999034+0000","last_undegraded":"2024-05-04T03:09:44.999034+0000","last_fullsized":"2024-05-04T03:09:44.999034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.229266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367534+0000","last_change":"2024-05-04T03:07:34.132966+0000","last_active":"2024-05-04T03:08:16.367534+0000","last_peered":"2024-05-04T03:08:16.367534+0000","last_clean":"2024-05-04T03:08:16.367534+0000","last_became_active":"2024-05-04T03:07:34.132529+0000","last_became_peered":"2024-05-04T03:07:34.132529+0000","last_unstale":"2024-05-04T03:08:16.367534+0000","last_undegraded":"2024-05-04T03:08:16.367534+0000","last_fullsized":"2024-05-04T03:08:16.367534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:31:37.122097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008076+0000","last_change":"2024-05-04T03:08:16.368747+0000","last_active":"2024-05-04T03:09:45.008076+0000","last_peered":"2024-05-04T03:09:45.008076+0000","last_clean":"2024-05-04T03:09:45.008076+0000","last_became_active":"2024-05-04T03:08:16.368376+0000","last_became_peered":"2024-05-04T03:08:16.368376+0000","last_unstale":"2024-05-04T03:09:45.008076+0000","last_undegraded":"2024-05-04T03:09:45.008076+0000","last_fullsized":"2024-05-04T03:09:45.008076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:42:32.085650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242516+0000","last_change":"2024-05-04T03:07:34.157219+0000","last_active":"2024-05-04T03:08:16.242516+0000","last_peered":"2024-05-04T03:08:16.242516+0000","last_clean":"2024-05-04T03:08:16.242516+0000","last_became_active":"2024-05-04T03:07:34.156866+0000","last_became_peered":"2024-05-04T03:07:34.156866+0000","last_unstale":"2024-05-04T03:08:16.242516+0000","last_undegraded":"2024-05-04T03:08:16.242516+0000","last_fullsized":"2024-05-04T03:08:16.242516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:15:34.138825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.990376+0000","last_change":"2024-05-04T03:07:37.058733+0000","last_active":"2024-05-04T03:09:44.990376+0000","last_peered":"2024-05-04T03:09:44.990376+0000","last_clean":"2024-05-04T03:09:44.990376+0000","last_became_active":"2024-05-04T03:07:37.058396+0000","last_became_peered":"2024-05-04T03:07:37.058396+0000","last_unstale":"2024-05-04T03:09:44.990376+0000","last_undegraded":"2024-05-04T03:09:44.990376+0000","last_fullsized":"2024-05-04T03:09:44.990376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:35.563728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243578+0000","last_change":"2024-05-04T03:07:34.016458+0000","last_active":"2024-05-04T03:08:16.243578+0000","last_peered":"2024-05-04T03:08:16.243578+0000","last_clean":"2024-05-04T03:08:16.243578+0000","last_became_active":"2024-05-04T03:07:34.016038+0000","last_became_peered":"2024-05-04T03:07:34.016038+0000","last_unstale":"2024-05-04T03:08:16.243578+0000","last_undegraded":"2024-05-04T03:08:16.243578+0000","last_fullsized":"2024-05-04T03:08:16.243578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:39:25.156047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998594+0000","last_change":"2024-05-04T03:07:37.064024+0000","last_active":"2024-05-04T03:09:44.998594+0000","last_peered":"2024-05-04T03:09:44.998594+0000","last_clean":"2024-05-04T03:09:44.998594+0000","last_became_active":"2024-05-04T03:07:37.063642+0000","last_became_peered":"2024-05-04T03:07:37.063642+0000","last_unstale":"2024-05-04T03:09:44.998594+0000","last_undegraded":"2024-05-04T03:09:44.998594+0000","last_fullsized":"2024-05-04T03:09:44.998594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:59:58.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'4","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:10:26.534919+0000","last_change":"2024-05-04T03:08:16.266763+0000","last_active":"2024-05-04T03:10:26.534919+0000","last_peered":"2024-05-04T03:10:26.534919+0000","last_clean":"2024-05-04T03:10:26.534919+0000","last_became_active":"2024-05-04T03:08:16.251212+0000","last_became_peered":"2024-05-04T03:08:16.251212+0000","last_unstale":"2024-05-04T03:10:26.534919+0000","last_undegraded":"2024-05-04T03:10:26.534919+0000","last_fullsized":"2024-05-04T03:10:26.534919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:31:27.444610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998226+0000","last_change":"2024-05-04T03:07:37.063753+0000","last_active":"2024-05-04T03:09:44.998226+0000","last_peered":"2024-05-04T03:09:44.998226+0000","last_clean":"2024-05-04T03:09:44.998226+0000","last_became_active":"2024-05-04T03:07:37.063318+0000","last_became_peered":"2024-05-04T03:07:37.063318+0000","last_unstale":"2024-05-04T03:09:44.998226+0000","last_undegraded":"2024-05-04T03:09:44.998226+0000","last_fullsized":"2024-05-04T03:09:44.998226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:48.217136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245182+0000","last_change":"2024-05-04T03:07:34.022522+0000","last_active":"2024-05-04T03:08:16.245182+0000","last_peered":"2024-05-04T03:08:16.245182+0000","last_clean":"2024-05-04T03:08:16.245182+0000","last_became_active":"2024-05-04T03:07:34.021018+0000","last_became_peered":"2024-05-04T03:07:34.021018+0000","last_unstale":"2024-05-04T03:08:16.245182+0000","last_undegraded":"2024-05-04T03:08:16.245182+0000","last_fullsized":"2024-05-04T03:08:16.245182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:47:04.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.014755+0000","last_change":"2024-05-04T03:07:37.069930+0000","last_active":"2024-05-04T03:09:45.014755+0000","last_peered":"2024-05-04T03:09:45.014755+0000","last_clean":"2024-05-04T03:09:45.014755+0000","last_became_active":"2024-05-04T03:07:37.069580+0000","last_became_peered":"2024-05-04T03:07:37.069580+0000","last_unstale":"2024-05-04T03:09:45.014755+0000","last_undegraded":"2024-05-04T03:09:45.014755+0000","last_fullsized":"2024-05-04T03:09:45.014755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:15:33.815970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":225,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:06.541656+0000","last_change":"2024-05-04T03:07:34.157969+0000","last_active":"2024-05-04T03:11:06.541656+0000","last_peered":"2024-05-04T03:11:06.541656+0000","last_clean":"2024-05-04T03:11:06.541656+0000","last_became_active":"2024-05-04T03:07:34.157563+0000","last_became_peered":"2024-05-04T03:07:34.157563+0000","last_unstale":"2024-05-04T03:11:06.541656+0000","last_undegraded":"2024-05-04T03:11:06.541656+0000","last_fullsized":"2024-05-04T03:11:06.541656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:00:43.562109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.015438+0000","last_change":"2024-05-04T03:07:37.071279+0000","last_active":"2024-05-04T03:09:45.015438+0000","last_peered":"2024-05-04T03:09:45.015438+0000","last_clean":"2024-05-04T03:09:45.015438+0000","last_became_active":"2024-05-04T03:07:37.070449+0000","last_became_peered":"2024-05-04T03:07:37.070449+0000","last_unstale":"2024-05-04T03:09:45.015438+0000","last_undegraded":"2024-05-04T03:09:45.015438+0000","last_fullsized":"2024-05-04T03:09:45.015438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:13:22.535788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248601+0000","last_change":"2024-05-04T03:07:34.017010+0000","last_active":"2024-05-04T03:08:16.248601+0000","last_peered":"2024-05-04T03:08:16.248601+0000","last_clean":"2024-05-04T03:08:16.248601+0000","last_became_active":"2024-05-04T03:07:34.016521+0000","last_became_peered":"2024-05-04T03:07:34.016521+0000","last_unstale":"2024-05-04T03:08:16.248601+0000","last_undegraded":"2024-05-04T03:08:16.248601+0000","last_fullsized":"2024-05-04T03:08:16.248601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:36.684656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.000434+0000","last_change":"2024-05-04T03:07:37.066526+0000","last_active":"2024-05-04T03:09:45.000434+0000","last_peered":"2024-05-04T03:09:45.000434+0000","last_clean":"2024-05-04T03:09:45.000434+0000","last_became_active":"2024-05-04T03:07:37.065312+0000","last_became_peered":"2024-05-04T03:07:37.065312+0000","last_unstale":"2024-05-04T03:09:45.000434+0000","last_undegraded":"2024-05-04T03:09:45.000434+0000","last_fullsized":"2024-05-04T03:09:45.000434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:10.077295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248987+0000","last_change":"2024-05-04T03:07:34.016898+0000","last_active":"2024-05-04T03:08:16.248987+0000","last_peered":"2024-05-04T03:08:16.248987+0000","last_clean":"2024-05-04T03:08:16.248987+0000","last_became_active":"2024-05-04T03:07:34.016524+0000","last_became_peered":"2024-05-04T03:07:34.016524+0000","last_unstale":"2024-05-04T03:08:16.248987+0000","last_undegraded":"2024-05-04T03:08:16.248987+0000","last_fullsized":"2024-05-04T03:08:16.248987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:01.236207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.001328+0000","last_change":"2024-05-04T03:07:37.061501+0000","last_active":"2024-05-04T03:09:45.001328+0000","last_peered":"2024-05-04T03:09:45.001328+0000","last_clean":"2024-05-04T03:09:45.001328+0000","last_became_active":"2024-05-04T03:07:37.061077+0000","last_became_peered":"2024-05-04T03:07:37.061077+0000","last_unstale":"2024-05-04T03:09:45.001328+0000","last_undegraded":"2024-05-04T03:09:45.001328+0000","last_fullsized":"2024-05-04T03:09:45.001328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:32.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996375+0000","last_change":"2024-05-04T03:07:37.064612+0000","last_active":"2024-05-04T03:09:44.996375+0000","last_peered":"2024-05-04T03:09:44.996375+0000","last_clean":"2024-05-04T03:09:44.996375+0000","last_became_active":"2024-05-04T03:07:37.064303+0000","last_became_peered":"2024-05-04T03:07:37.064303+0000","last_unstale":"2024-05-04T03:09:44.996375+0000","last_undegraded":"2024-05-04T03:09:44.996375+0000","last_fullsized":"2024-05-04T03:09:44.996375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:47:07.811765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005934+0000","last_change":"2024-05-04T03:07:37.061577+0000","last_active":"2024-05-04T03:09:45.005934+0000","last_peered":"2024-05-04T03:09:45.005934+0000","last_clean":"2024-05-04T03:09:45.005934+0000","last_became_active":"2024-05-04T03:07:37.061214+0000","last_became_peered":"2024-05-04T03:07:37.061214+0000","last_unstale":"2024-05-04T03:09:45.005934+0000","last_undegraded":"2024-05-04T03:09:45.005934+0000","last_fullsized":"2024-05-04T03:09:45.005934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:52:11.733257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005073+0000","last_change":"2024-05-04T03:07:37.062906+0000","last_active":"2024-05-04T03:09:45.005073+0000","last_peered":"2024-05-04T03:09:45.005073+0000","last_clean":"2024-05-04T03:09:45.005073+0000","last_became_active":"2024-05-04T03:07:37.062510+0000","last_became_peered":"2024-05-04T03:07:37.062510+0000","last_unstale":"2024-05-04T03:09:45.005073+0000","last_undegraded":"2024-05-04T03:09:45.005073+0000","last_fullsized":"2024-05-04T03:09:45.005073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:57:58.910490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008959+0000","last_change":"2024-05-04T03:07:37.068367+0000","last_active":"2024-05-04T03:09:45.008959+0000","last_peered":"2024-05-04T03:09:45.008959+0000","last_clean":"2024-05-04T03:09:45.008959+0000","last_became_active":"2024-05-04T03:07:37.068038+0000","last_became_peered":"2024-05-04T03:07:37.068038+0000","last_unstale":"2024-05-04T03:09:45.008959+0000","last_undegraded":"2024-05-04T03:09:45.008959+0000","last_fullsized":"2024-05-04T03:09:45.008959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:11:19.195002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003363+0000","last_change":"2024-05-04T03:08:16.250759+0000","last_active":"2024-05-04T03:09:45.003363+0000","last_peered":"2024-05-04T03:09:45.003363+0000","last_clean":"2024-05-04T03:09:45.003363+0000","last_became_active":"2024-05-04T03:08:16.250379+0000","last_became_peered":"2024-05-04T03:08:16.250379+0000","last_unstale":"2024-05-04T03:09:45.003363+0000","last_undegraded":"2024-05-04T03:09:45.003363+0000","last_fullsized":"2024-05-04T03:09:45.003363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:58:09.419009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.991925+0000","last_change":"2024-05-04T03:07:37.064584+0000","last_active":"2024-05-04T03:09:44.991925+0000","last_peered":"2024-05-04T03:09:44.991925+0000","last_clean":"2024-05-04T03:09:44.991925+0000","last_became_active":"2024-05-04T03:07:37.064183+0000","last_became_peered":"2024-05-04T03:07:37.064183+0000","last_unstale":"2024-05-04T03:09:44.991925+0000","last_undegraded":"2024-05-04T03:09:44.991925+0000","last_fullsized":"2024-05-04T03:09:44.991925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:17.121081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.013789+0000","last_change":"2024-05-04T03:07:37.066742+0000","last_active":"2024-05-04T03:09:45.013789+0000","last_peered":"2024-05-04T03:09:45.013789+0000","last_clean":"2024-05-04T03:09:45.013789+0000","last_became_active":"2024-05-04T03:07:37.066126+0000","last_became_peered":"2024-05-04T03:07:37.066126+0000","last_unstale":"2024-05-04T03:09:45.013789+0000","last_undegraded":"2024-05-04T03:09:45.013789+0000","last_fullsized":"2024-05-04T03:09:45.013789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:57.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.997576+0000","last_change":"2024-05-04T03:07:37.068122+0000","last_active":"2024-05-04T03:09:44.997576+0000","last_peered":"2024-05-04T03:09:44.997576+0000","last_clean":"2024-05-04T03:09:44.997576+0000","last_became_active":"2024-05-04T03:07:37.066321+0000","last_became_peered":"2024-05-04T03:07:37.066321+0000","last_unstale":"2024-05-04T03:09:44.997576+0000","last_undegraded":"2024-05-04T03:09:44.997576+0000","last_fullsized":"2024-05-04T03:09:44.997576+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:25:09.321853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994963+0000","last_change":"2024-05-04T03:07:37.060322+0000","last_active":"2024-05-04T03:09:44.994963+0000","last_peered":"2024-05-04T03:09:44.994963+0000","last_clean":"2024-05-04T03:09:44.994963+0000","last_became_active":"2024-05-04T03:07:37.059944+0000","last_became_peered":"2024-05-04T03:07:37.059944+0000","last_unstale":"2024-05-04T03:09:44.994963+0000","last_undegraded":"2024-05-04T03:09:44.994963+0000","last_fullsized":"2024-05-04T03:09:44.994963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.105085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.993232+0000","last_change":"2024-05-04T03:07:37.056269+0000","last_active":"2024-05-04T03:09:44.993232+0000","last_peered":"2024-05-04T03:09:44.993232+0000","last_clean":"2024-05-04T03:09:44.993232+0000","last_became_active":"2024-05-04T03:07:37.055886+0000","last_became_peered":"2024-05-04T03:07:37.055886+0000","last_unstale":"2024-05-04T03:09:44.993232+0000","last_undegraded":"2024-05-04T03:09:44.993232+0000","last_fullsized":"2024-05-04T03:09:44.993232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:55:11.292577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017574+0000","last_change":"2024-05-04T03:07:37.071013+0000","last_active":"2024-05-04T03:09:45.017574+0000","last_peered":"2024-05-04T03:09:45.017574+0000","last_clean":"2024-05-04T03:09:45.017574+0000","last_became_active":"2024-05-04T03:07:37.069346+0000","last_became_peered":"2024-05-04T03:07:37.069346+0000","last_unstale":"2024-05-04T03:09:45.017574+0000","last_undegraded":"2024-05-04T03:09:45.017574+0000","last_fullsized":"2024-05-04T03:09:45.017574+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:19:11.898766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003394+0000","last_change":"2024-05-04T03:07:37.064962+0000","last_active":"2024-05-04T03:09:45.003394+0000","last_peered":"2024-05-04T03:09:45.003394+0000","last_clean":"2024-05-04T03:09:45.003394+0000","last_became_active":"2024-05-04T03:07:37.064637+0000","last_became_peered":"2024-05-04T03:07:37.064637+0000","last_unstale":"2024-05-04T03:09:45.003394+0000","last_undegraded":"2024-05-04T03:09:45.003394+0000","last_fullsized":"2024-05-04T03:09:45.003394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:10:10.659621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002661+0000","last_change":"2024-05-04T03:07:37.056721+0000","last_active":"2024-05-04T03:09:45.002661+0000","last_peered":"2024-05-04T03:09:45.002661+0000","last_clean":"2024-05-04T03:09:45.002661+0000","last_became_active":"2024-05-04T03:07:37.056353+0000","last_became_peered":"2024-05-04T03:07:37.056353+0000","last_unstale":"2024-05-04T03:09:45.002661+0000","last_undegraded":"2024-05-04T03:09:45.002661+0000","last_fullsized":"2024-05-04T03:09:45.002661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:30:47.476579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286351,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70326,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1935,"num_read_kb":1045,"num_write":270,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":219606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710779,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":426309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907007,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1078875,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":425694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266762,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":426072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462991,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":1020162,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626517,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":425888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":492579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":427227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":493599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":425236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986271,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":426281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822745,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":1018354,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:11:10.028 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:11:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 -- 172.21.15.39:0/228544927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 msgr2=0x7f08a81130e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 --2- 172.21.15.39:0/228544927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 0x7f08a81130e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b0a0 tx=0x7f08a000b3b0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 -- 172.21.15.39:0/228544927 shutdown_connections 2024-05-04T03:11:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 --2- 172.21.15.39:0/228544927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 0x7f08a81130e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 --2- 172.21.15.39:0/228544927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08a8103110 0x7f08a8103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 --2- 172.21.15.39:0/228544927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a8103a60 0x7f08a8110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 -- 172.21.15.39:0/228544927 >> 172.21.15.39:0/228544927 conn(0x7f08a8078e00 msgr2=0x7f08a8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 -- 172.21.15.39:0/228544927 shutdown_connections 2024-05-04T03:11:12.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.006+0000 7f08af882700 1 -- 172.21.15.39:0/228544927 wait complete. 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08af882700 1 Processor -- start 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08af882700 1 -- start start 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08af882700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08a8103110 0x7f08a819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08af882700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a8103a60 0x7f08a819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08af882700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 0x7f08a81a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08af882700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8116560 con 0x7f08a8110cc0 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08af882700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81163e0 con 0x7f08a8103a60 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08af882700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81166e0 con 0x7f08a8103110 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08ade1f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 0x7f08a81a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08ade1f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 0x7f08a81a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58058/0 (socket says 172.21.15.39:58058) 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08ade1f700 1 -- 172.21.15.39:0/3057691886 learned_addr learned my addr 172.21.15.39:0/3057691886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:12.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.007+0000 7f08ace1d700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a8103a60 0x7f08a819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08ad61e700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08a8103110 0x7f08a819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08ade1f700 1 -- 172.21.15.39:0/3057691886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08a8103110 msgr2=0x7f08a819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08ade1f700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08a8103110 0x7f08a819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08ade1f700 1 -- 172.21.15.39:0/3057691886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a8103a60 msgr2=0x7f08a819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08ade1f700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a8103a60 0x7f08a819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08ade1f700 1 -- 172.21.15.39:0/3057691886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a81a1da0 con 0x7f08a8110cc0 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08ade1f700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 0x7f08a81a1700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b830 tx=0x7f08a0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f089e7fc700 1 -- 172.21.15.39:0/3057691886 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0021070 con 0x7f08a8110cc0 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f089e7fc700 1 -- 172.21.15.39:0/3057691886 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08a0007b80 con 0x7f08a8110cc0 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f089e7fc700 1 -- 172.21.15.39:0/3057691886 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a00052a0 con 0x7f08a8110cc0 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a81a2030 con 0x7f08a8110cc0 2024-05-04T03:11:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.008+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a81a24c0 con 0x7f08a8110cc0 2024-05-04T03:11:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.010+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a80512c0 con 0x7f08a8110cc0 2024-05-04T03:11:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.011+0000 7f089e7fc700 1 -- 172.21.15.39:0/3057691886 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f08a0024070 con 0x7f08a8110cc0 2024-05-04T03:11:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.012+0000 7f089e7fc700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f08940b2990 0x7f08940b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:12.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.012+0000 7f08ad61e700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f08940b2990 0x7f08940b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:12.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.012+0000 7f089e7fc700 1 -- 172.21.15.39:0/3057691886 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f08a000f020 con 0x7f08a8110cc0 2024-05-04T03:11:12.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.012+0000 7f08ad61e700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f08940b2990 0x7f08940b4ea0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0898005950 tx=0x7f08980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.016+0000 7f089e7fc700 1 -- 172.21.15.39:0/3057691886 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08a002b020 con 0x7f08a8110cc0 2024-05-04T03:11:12.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.231+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08a8063c00 con 0x7f08940b2990 2024-05-04T03:11:12.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.239+0000 7f089e7fc700 1 -- 172.21.15.39:0/3057691886 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259048 (secure 0 0 0) 0x7f08a8063c00 con 0x7f08940b2990 2024-05-04T03:11:12.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f08940b2990 msgr2=0x7f08940b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f08940b2990 0x7f08940b4ea0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0898005950 tx=0x7f08980058e0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 msgr2=0x7f08a81a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 0x7f08a81a1700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b830 tx=0x7f08a0004d00 comp rx=0 tx=0).stop 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 shutdown_connections 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f08940b2990 0x7f08940b4ea0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08a8110cc0 0x7f08a81a1700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08a8103110 0x7f08a819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 --2- 172.21.15.39:0/3057691886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08a8103a60 0x7f08a819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.242+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 >> 172.21.15.39:0/3057691886 conn(0x7f08a8078e00 msgr2=0x7f08a81121a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.243+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 shutdown_connections 2024-05-04T03:11:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:12.243+0000 7f08af882700 1 -- 172.21.15.39:0/3057691886 wait complete. 2024-05-04T03:11:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-04T03:11:12.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:12 smithi167 ceph-mon[33614]: pgmap v398: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-04T03:11:12.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:12 smithi039 ceph-mon[29715]: pgmap v398: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-04T03:11:12.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:12 smithi066 ceph-mon[33264]: pgmap v398: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-04T03:11:12.970 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-05-04T03:11:11.053315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665633,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2038,"num_read_kb":1130,"num_write":397,"num_write_kb":2313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":826,"ondisk_log_size":826,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376604,"kb_used_data":10812,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565220,"statfs":{"total":1151940427776,"available":1151554785280,"internally_reserved":0,"allocated":11071488,"data_stored":7091372,"data_compressed":109203,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7481,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":76,"num_write":21,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003387"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998775+0000","last_change":"2024-05-04T03:07:37.070347+0000","last_active":"2024-05-04T03:09:44.998775+0000","last_peered":"2024-05-04T03:09:44.998775+0000","last_clean":"2024-05-04T03:09:44.998775+0000","last_became_active":"2024-05-04T03:07:37.069997+0000","last_became_peered":"2024-05-04T03:07:37.069997+0000","last_unstale":"2024-05-04T03:09:44.998775+0000","last_undegraded":"2024-05-04T03:09:44.998775+0000","last_fullsized":"2024-05-04T03:09:44.998775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:24:11.362228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010264+0000","last_change":"2024-05-04T03:07:37.071598+0000","last_active":"2024-05-04T03:09:45.010264+0000","last_peered":"2024-05-04T03:09:45.010264+0000","last_clean":"2024-05-04T03:09:45.010264+0000","last_became_active":"2024-05-04T03:07:37.071059+0000","last_became_peered":"2024-05-04T03:07:37.071059+0000","last_unstale":"2024-05-04T03:09:45.010264+0000","last_undegraded":"2024-05-04T03:09:45.010264+0000","last_fullsized":"2024-05-04T03:09:45.010264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:33:33.929246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017072+0000","last_change":"2024-05-04T03:07:37.069951+0000","last_active":"2024-05-04T03:09:45.017072+0000","last_peered":"2024-05-04T03:09:45.017072+0000","last_clean":"2024-05-04T03:09:45.017072+0000","last_became_active":"2024-05-04T03:07:37.069558+0000","last_became_peered":"2024-05-04T03:07:37.069558+0000","last_unstale":"2024-05-04T03:09:45.017072+0000","last_undegraded":"2024-05-04T03:09:45.017072+0000","last_fullsized":"2024-05-04T03:09:45.017072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:37:52.596455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.001811+0000","last_change":"2024-05-04T03:07:37.068820+0000","last_active":"2024-05-04T03:09:45.001811+0000","last_peered":"2024-05-04T03:09:45.001811+0000","last_clean":"2024-05-04T03:09:45.001811+0000","last_became_active":"2024-05-04T03:07:37.068323+0000","last_became_peered":"2024-05-04T03:07:37.068323+0000","last_unstale":"2024-05-04T03:09:45.001811+0000","last_undegraded":"2024-05-04T03:09:45.001811+0000","last_fullsized":"2024-05-04T03:09:45.001811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:52:54.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.981848+0000","last_change":"2024-05-04T03:07:37.061982+0000","last_active":"2024-05-04T03:09:44.981848+0000","last_peered":"2024-05-04T03:09:44.981848+0000","last_clean":"2024-05-04T03:09:44.981848+0000","last_became_active":"2024-05-04T03:07:37.061621+0000","last_became_peered":"2024-05-04T03:07:37.061621+0000","last_unstale":"2024-05-04T03:09:44.981848+0000","last_undegraded":"2024-05-04T03:09:44.981848+0000","last_fullsized":"2024-05-04T03:09:44.981848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:17:47.421011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003074+0000","last_change":"2024-05-04T03:07:37.058641+0000","last_active":"2024-05-04T03:09:45.003074+0000","last_peered":"2024-05-04T03:09:45.003074+0000","last_clean":"2024-05-04T03:09:45.003074+0000","last_became_active":"2024-05-04T03:07:37.058235+0000","last_became_peered":"2024-05-04T03:07:37.058235+0000","last_unstale":"2024-05-04T03:09:45.003074+0000","last_undegraded":"2024-05-04T03:09:45.003074+0000","last_fullsized":"2024-05-04T03:09:45.003074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:17:45.441777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998905+0000","last_change":"2024-05-04T03:07:37.065959+0000","last_active":"2024-05-04T03:09:44.998905+0000","last_peered":"2024-05-04T03:09:44.998905+0000","last_clean":"2024-05-04T03:09:44.998905+0000","last_became_active":"2024-05-04T03:07:37.065508+0000","last_became_peered":"2024-05-04T03:07:37.065508+0000","last_unstale":"2024-05-04T03:09:44.998905+0000","last_undegraded":"2024-05-04T03:09:44.998905+0000","last_fullsized":"2024-05-04T03:09:44.998905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:17.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994908+0000","last_change":"2024-05-04T03:07:37.065620+0000","last_active":"2024-05-04T03:09:44.994908+0000","last_peered":"2024-05-04T03:09:44.994908+0000","last_clean":"2024-05-04T03:09:44.994908+0000","last_became_active":"2024-05-04T03:07:37.065261+0000","last_became_peered":"2024-05-04T03:07:37.065261+0000","last_unstale":"2024-05-04T03:09:44.994908+0000","last_undegraded":"2024-05-04T03:09:44.994908+0000","last_fullsized":"2024-05-04T03:09:44.994908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:27.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017270+0000","last_change":"2024-05-04T03:07:37.072204+0000","last_active":"2024-05-04T03:09:45.017270+0000","last_peered":"2024-05-04T03:09:45.017270+0000","last_clean":"2024-05-04T03:09:45.017270+0000","last_became_active":"2024-05-04T03:07:37.070622+0000","last_became_peered":"2024-05-04T03:07:37.070622+0000","last_unstale":"2024-05-04T03:09:45.017270+0000","last_undegraded":"2024-05-04T03:09:45.017270+0000","last_fullsized":"2024-05-04T03:09:45.017270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:55:25.931369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003159+0000","last_change":"2024-05-04T03:08:16.253125+0000","last_active":"2024-05-04T03:09:45.003159+0000","last_peered":"2024-05-04T03:09:45.003159+0000","last_clean":"2024-05-04T03:09:45.003159+0000","last_became_active":"2024-05-04T03:08:16.252682+0000","last_became_peered":"2024-05-04T03:08:16.252682+0000","last_unstale":"2024-05-04T03:09:45.003159+0000","last_undegraded":"2024-05-04T03:09:45.003159+0000","last_fullsized":"2024-05-04T03:09:45.003159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:05.933491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.015463+0000","last_change":"2024-05-04T03:07:37.071614+0000","last_active":"2024-05-04T03:09:45.015463+0000","last_peered":"2024-05-04T03:09:45.015463+0000","last_clean":"2024-05-04T03:09:45.015463+0000","last_became_active":"2024-05-04T03:07:37.070927+0000","last_became_peered":"2024-05-04T03:07:37.070927+0000","last_unstale":"2024-05-04T03:09:45.015463+0000","last_undegraded":"2024-05-04T03:09:45.015463+0000","last_fullsized":"2024-05-04T03:09:45.015463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:30.432959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005280+0000","last_change":"2024-05-04T03:07:37.069305+0000","last_active":"2024-05-04T03:09:45.005280+0000","last_peered":"2024-05-04T03:09:45.005280+0000","last_clean":"2024-05-04T03:09:45.005280+0000","last_became_active":"2024-05-04T03:07:37.068804+0000","last_became_peered":"2024-05-04T03:07:37.068804+0000","last_unstale":"2024-05-04T03:09:45.005280+0000","last_undegraded":"2024-05-04T03:09:45.005280+0000","last_fullsized":"2024-05-04T03:09:45.005280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:43.693161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.006446+0000","last_change":"2024-05-04T03:07:37.062848+0000","last_active":"2024-05-04T03:09:45.006446+0000","last_peered":"2024-05-04T03:09:45.006446+0000","last_clean":"2024-05-04T03:09:45.006446+0000","last_became_active":"2024-05-04T03:07:37.062548+0000","last_became_peered":"2024-05-04T03:07:37.062548+0000","last_unstale":"2024-05-04T03:09:45.006446+0000","last_undegraded":"2024-05-04T03:09:45.006446+0000","last_fullsized":"2024-05-04T03:09:45.006446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010679+0000","last_change":"2024-05-04T03:07:37.062598+0000","last_active":"2024-05-04T03:09:45.010679+0000","last_peered":"2024-05-04T03:09:45.010679+0000","last_clean":"2024-05-04T03:09:45.010679+0000","last_became_active":"2024-05-04T03:07:37.061711+0000","last_became_peered":"2024-05-04T03:07:37.061711+0000","last_unstale":"2024-05-04T03:09:45.010679+0000","last_undegraded":"2024-05-04T03:09:45.010679+0000","last_fullsized":"2024-05-04T03:09:45.010679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:29:03.153680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007786+0000","last_change":"2024-05-04T03:07:37.067889+0000","last_active":"2024-05-04T03:09:45.007786+0000","last_peered":"2024-05-04T03:09:45.007786+0000","last_clean":"2024-05-04T03:09:45.007786+0000","last_became_active":"2024-05-04T03:07:37.066768+0000","last_became_peered":"2024-05-04T03:07:37.066768+0000","last_unstale":"2024-05-04T03:09:45.007786+0000","last_undegraded":"2024-05-04T03:09:45.007786+0000","last_fullsized":"2024-05-04T03:09:45.007786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:01:18.136379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017415+0000","last_change":"2024-05-04T03:07:37.070438+0000","last_active":"2024-05-04T03:09:45.017415+0000","last_peered":"2024-05-04T03:09:45.017415+0000","last_clean":"2024-05-04T03:09:45.017415+0000","last_became_active":"2024-05-04T03:07:37.070007+0000","last_became_peered":"2024-05-04T03:07:37.070007+0000","last_unstale":"2024-05-04T03:09:45.017415+0000","last_undegraded":"2024-05-04T03:09:45.017415+0000","last_fullsized":"2024-05-04T03:09:45.017415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:22.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999016+0000","last_change":"2024-05-04T03:07:37.070020+0000","last_active":"2024-05-04T03:09:44.999016+0000","last_peered":"2024-05-04T03:09:44.999016+0000","last_clean":"2024-05-04T03:09:44.999016+0000","last_became_active":"2024-05-04T03:07:37.069629+0000","last_became_peered":"2024-05-04T03:07:37.069629+0000","last_unstale":"2024-05-04T03:09:44.999016+0000","last_undegraded":"2024-05-04T03:09:44.999016+0000","last_fullsized":"2024-05-04T03:09:44.999016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:28.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.011076+0000","last_change":"2024-05-04T03:07:37.068651+0000","last_active":"2024-05-04T03:09:45.011076+0000","last_peered":"2024-05-04T03:09:45.011076+0000","last_clean":"2024-05-04T03:09:45.011076+0000","last_became_active":"2024-05-04T03:07:37.067135+0000","last_became_peered":"2024-05-04T03:07:37.067135+0000","last_unstale":"2024-05-04T03:09:45.011076+0000","last_undegraded":"2024-05-04T03:09:45.011076+0000","last_fullsized":"2024-05-04T03:09:45.011076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:00:59.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994607+0000","last_change":"2024-05-04T03:07:37.055419+0000","last_active":"2024-05-04T03:09:44.994607+0000","last_peered":"2024-05-04T03:09:44.994607+0000","last_clean":"2024-05-04T03:09:44.994607+0000","last_became_active":"2024-05-04T03:07:37.055024+0000","last_became_peered":"2024-05-04T03:07:37.055024+0000","last_unstale":"2024-05-04T03:09:44.994607+0000","last_undegraded":"2024-05-04T03:09:44.994607+0000","last_fullsized":"2024-05-04T03:09:44.994607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:55:05.506505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":232,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:07.481589+0000","last_change":"2024-05-04T03:07:34.019975+0000","last_active":"2024-05-04T03:11:07.481589+0000","last_peered":"2024-05-04T03:11:07.481589+0000","last_clean":"2024-05-04T03:11:07.481589+0000","last_became_active":"2024-05-04T03:07:34.019532+0000","last_became_peered":"2024-05-04T03:07:34.019532+0000","last_unstale":"2024-05-04T03:11:07.481589+0000","last_undegraded":"2024-05-04T03:11:07.481589+0000","last_fullsized":"2024-05-04T03:11:07.481589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:11:03.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.990313+0000","last_change":"2024-05-04T03:07:37.064725+0000","last_active":"2024-05-04T03:09:44.990313+0000","last_peered":"2024-05-04T03:09:44.990313+0000","last_clean":"2024-05-04T03:09:44.990313+0000","last_became_active":"2024-05-04T03:07:37.064385+0000","last_became_peered":"2024-05-04T03:07:37.064385+0000","last_unstale":"2024-05-04T03:09:44.990313+0000","last_undegraded":"2024-05-04T03:09:44.990313+0000","last_fullsized":"2024-05-04T03:09:44.990313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:22:12.024328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244201+0000","last_change":"2024-05-04T03:07:34.157179+0000","last_active":"2024-05-04T03:08:16.244201+0000","last_peered":"2024-05-04T03:08:16.244201+0000","last_clean":"2024-05-04T03:08:16.244201+0000","last_became_active":"2024-05-04T03:07:34.156790+0000","last_became_peered":"2024-05-04T03:07:34.156790+0000","last_unstale":"2024-05-04T03:08:16.244201+0000","last_undegraded":"2024-05-04T03:08:16.244201+0000","last_fullsized":"2024-05-04T03:08:16.244201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:32.066410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017111+0000","last_change":"2024-05-04T03:07:37.070481+0000","last_active":"2024-05-04T03:09:45.017111+0000","last_peered":"2024-05-04T03:09:45.017111+0000","last_clean":"2024-05-04T03:09:45.017111+0000","last_became_active":"2024-05-04T03:07:37.069903+0000","last_became_peered":"2024-05-04T03:07:37.069903+0000","last_unstale":"2024-05-04T03:09:45.017111+0000","last_undegraded":"2024-05-04T03:09:45.017111+0000","last_fullsized":"2024-05-04T03:09:45.017111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:45:40.764716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245782+0000","last_change":"2024-05-04T03:07:34.023978+0000","last_active":"2024-05-04T03:08:16.245782+0000","last_peered":"2024-05-04T03:08:16.245782+0000","last_clean":"2024-05-04T03:08:16.245782+0000","last_became_active":"2024-05-04T03:07:34.023199+0000","last_became_peered":"2024-05-04T03:07:34.023199+0000","last_unstale":"2024-05-04T03:08:16.245782+0000","last_undegraded":"2024-05-04T03:08:16.245782+0000","last_fullsized":"2024-05-04T03:08:16.245782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:17:43.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017273+0000","last_change":"2024-05-04T03:07:37.072661+0000","last_active":"2024-05-04T03:09:45.017273+0000","last_peered":"2024-05-04T03:09:45.017273+0000","last_clean":"2024-05-04T03:09:45.017273+0000","last_became_active":"2024-05-04T03:07:37.071393+0000","last_became_peered":"2024-05-04T03:07:37.071393+0000","last_unstale":"2024-05-04T03:09:45.017273+0000","last_undegraded":"2024-05-04T03:09:45.017273+0000","last_fullsized":"2024-05-04T03:09:45.017273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:39:39.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246044+0000","last_change":"2024-05-04T03:07:34.018543+0000","last_active":"2024-05-04T03:08:16.246044+0000","last_peered":"2024-05-04T03:08:16.246044+0000","last_clean":"2024-05-04T03:08:16.246044+0000","last_became_active":"2024-05-04T03:07:34.018145+0000","last_became_peered":"2024-05-04T03:07:34.018145+0000","last_unstale":"2024-05-04T03:08:16.246044+0000","last_undegraded":"2024-05-04T03:08:16.246044+0000","last_fullsized":"2024-05-04T03:08:16.246044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:44.529492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007771+0000","last_change":"2024-05-04T03:07:37.070811+0000","last_active":"2024-05-04T03:09:45.007771+0000","last_peered":"2024-05-04T03:09:45.007771+0000","last_clean":"2024-05-04T03:09:45.007771+0000","last_became_active":"2024-05-04T03:07:37.070441+0000","last_became_peered":"2024-05-04T03:07:37.070441+0000","last_unstale":"2024-05-04T03:09:45.007771+0000","last_undegraded":"2024-05-04T03:09:45.007771+0000","last_fullsized":"2024-05-04T03:09:45.007771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:01:48.146339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:09.469193+0000","last_change":"2024-05-04T03:07:34.020508+0000","last_active":"2024-05-04T03:11:09.469193+0000","last_peered":"2024-05-04T03:11:09.469193+0000","last_clean":"2024-05-04T03:11:09.469193+0000","last_became_active":"2024-05-04T03:07:34.020076+0000","last_became_peered":"2024-05-04T03:07:34.020076+0000","last_unstale":"2024-05-04T03:11:09.469193+0000","last_undegraded":"2024-05-04T03:11:09.469193+0000","last_fullsized":"2024-05-04T03:11:09.469193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:27.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1099,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.988466+0000","last_change":"2024-05-04T03:07:37.060004+0000","last_active":"2024-05-04T03:09:44.988466+0000","last_peered":"2024-05-04T03:09:44.988466+0000","last_clean":"2024-05-04T03:09:44.988466+0000","last_became_active":"2024-05-04T03:07:37.059350+0000","last_became_peered":"2024-05-04T03:07:37.059350+0000","last_unstale":"2024-05-04T03:09:44.988466+0000","last_undegraded":"2024-05-04T03:09:44.988466+0000","last_fullsized":"2024-05-04T03:09:44.988466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:37:07.315935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248143+0000","last_change":"2024-05-04T03:07:34.020174+0000","last_active":"2024-05-04T03:08:16.248143+0000","last_peered":"2024-05-04T03:08:16.248143+0000","last_clean":"2024-05-04T03:08:16.248143+0000","last_became_active":"2024-05-04T03:07:34.019790+0000","last_became_peered":"2024-05-04T03:07:34.019790+0000","last_unstale":"2024-05-04T03:08:16.248143+0000","last_undegraded":"2024-05-04T03:08:16.248143+0000","last_fullsized":"2024-05-04T03:08:16.248143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:37:37.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.987213+0000","last_change":"2024-05-04T03:08:16.251277+0000","last_active":"2024-05-04T03:09:44.987213+0000","last_peered":"2024-05-04T03:09:44.987213+0000","last_clean":"2024-05-04T03:09:44.987213+0000","last_became_active":"2024-05-04T03:08:16.250836+0000","last_became_peered":"2024-05-04T03:08:16.250836+0000","last_unstale":"2024-05-04T03:09:44.987213+0000","last_undegraded":"2024-05-04T03:09:44.987213+0000","last_fullsized":"2024-05-04T03:09:44.987213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:59:47.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248454+0000","last_change":"2024-05-04T03:07:34.013188+0000","last_active":"2024-05-04T03:08:16.248454+0000","last_peered":"2024-05-04T03:08:16.248454+0000","last_clean":"2024-05-04T03:08:16.248454+0000","last_became_active":"2024-05-04T03:07:34.012759+0000","last_became_peered":"2024-05-04T03:07:34.012759+0000","last_unstale":"2024-05-04T03:08:16.248454+0000","last_undegraded":"2024-05-04T03:08:16.248454+0000","last_fullsized":"2024-05-04T03:08:16.248454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:54:29.621250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.004186+0000","last_change":"2024-05-04T03:07:37.056265+0000","last_active":"2024-05-04T03:09:45.004186+0000","last_peered":"2024-05-04T03:09:45.004186+0000","last_clean":"2024-05-04T03:09:45.004186+0000","last_became_active":"2024-05-04T03:07:37.055957+0000","last_became_peered":"2024-05-04T03:07:37.055957+0000","last_unstale":"2024-05-04T03:09:45.004186+0000","last_undegraded":"2024-05-04T03:09:45.004186+0000","last_fullsized":"2024-05-04T03:09:45.004186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:41.739034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247922+0000","last_change":"2024-05-04T03:07:34.020722+0000","last_active":"2024-05-04T03:08:16.247922+0000","last_peered":"2024-05-04T03:08:16.247922+0000","last_clean":"2024-05-04T03:08:16.247922+0000","last_became_active":"2024-05-04T03:07:34.020028+0000","last_became_peered":"2024-05-04T03:07:34.020028+0000","last_unstale":"2024-05-04T03:08:16.247922+0000","last_undegraded":"2024-05-04T03:08:16.247922+0000","last_fullsized":"2024-05-04T03:08:16.247922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:23:20.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998984+0000","last_change":"2024-05-04T03:07:37.059680+0000","last_active":"2024-05-04T03:09:44.998984+0000","last_peered":"2024-05-04T03:09:44.998984+0000","last_clean":"2024-05-04T03:09:44.998984+0000","last_became_active":"2024-05-04T03:07:37.059294+0000","last_became_peered":"2024-05-04T03:07:37.059294+0000","last_unstale":"2024-05-04T03:09:44.998984+0000","last_undegraded":"2024-05-04T03:09:44.998984+0000","last_fullsized":"2024-05-04T03:09:44.998984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:50:06.251706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":419,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:10.470054+0000","last_change":"2024-05-04T03:07:34.019055+0000","last_active":"2024-05-04T03:11:10.470054+0000","last_peered":"2024-05-04T03:11:10.470054+0000","last_clean":"2024-05-04T03:11:10.470054+0000","last_became_active":"2024-05-04T03:07:34.018638+0000","last_became_peered":"2024-05-04T03:07:34.018638+0000","last_unstale":"2024-05-04T03:11:10.470054+0000","last_undegraded":"2024-05-04T03:11:10.470054+0000","last_fullsized":"2024-05-04T03:11:10.470054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:58:01.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":386,"num_read_kb":386,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017350+0000","last_change":"2024-05-04T03:07:37.063037+0000","last_active":"2024-05-04T03:09:45.017350+0000","last_peered":"2024-05-04T03:09:45.017350+0000","last_clean":"2024-05-04T03:09:45.017350+0000","last_became_active":"2024-05-04T03:07:37.062350+0000","last_became_peered":"2024-05-04T03:07:37.062350+0000","last_unstale":"2024-05-04T03:09:45.017350+0000","last_undegraded":"2024-05-04T03:09:45.017350+0000","last_fullsized":"2024-05-04T03:09:45.017350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:41:15.898241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.374402+0000","last_change":"2024-05-04T03:08:16.373765+0000","last_active":"2024-05-04T03:08:16.374402+0000","last_peered":"2024-05-04T03:08:16.374402+0000","last_clean":"2024-05-04T03:08:16.374402+0000","last_became_active":"2024-05-04T03:08:16.368588+0000","last_became_peered":"2024-05-04T03:08:16.368588+0000","last_unstale":"2024-05-04T03:08:16.374402+0000","last_undegraded":"2024-05-04T03:08:16.374402+0000","last_fullsized":"2024-05-04T03:08:16.374402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:55:07.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007709+0000","last_change":"2024-05-04T03:07:37.071330+0000","last_active":"2024-05-04T03:09:45.007709+0000","last_peered":"2024-05-04T03:09:45.007709+0000","last_clean":"2024-05-04T03:09:45.007709+0000","last_became_active":"2024-05-04T03:07:37.070743+0000","last_became_peered":"2024-05-04T03:07:37.070743+0000","last_unstale":"2024-05-04T03:09:45.007709+0000","last_undegraded":"2024-05-04T03:09:45.007709+0000","last_fullsized":"2024-05-04T03:09:45.007709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:10:00.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248881+0000","last_change":"2024-05-04T03:07:34.157432+0000","last_active":"2024-05-04T03:08:16.248881+0000","last_peered":"2024-05-04T03:08:16.248881+0000","last_clean":"2024-05-04T03:08:16.248881+0000","last_became_active":"2024-05-04T03:07:34.156976+0000","last_became_peered":"2024-05-04T03:07:34.156976+0000","last_unstale":"2024-05-04T03:08:16.248881+0000","last_undegraded":"2024-05-04T03:08:16.248881+0000","last_fullsized":"2024-05-04T03:08:16.248881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:56.082287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017872+0000","last_change":"2024-05-04T03:07:37.064301+0000","last_active":"2024-05-04T03:09:45.017872+0000","last_peered":"2024-05-04T03:09:45.017872+0000","last_clean":"2024-05-04T03:09:45.017872+0000","last_became_active":"2024-05-04T03:07:37.063691+0000","last_became_peered":"2024-05-04T03:07:37.063691+0000","last_unstale":"2024-05-04T03:09:45.017872+0000","last_undegraded":"2024-05-04T03:09:45.017872+0000","last_fullsized":"2024-05-04T03:09:45.017872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:22:13.844703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'38","reported_seq":464,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:10.469748+0000","last_change":"2024-05-04T03:07:34.021595+0000","last_active":"2024-05-04T03:11:10.469748+0000","last_peered":"2024-05-04T03:11:10.469748+0000","last_clean":"2024-05-04T03:11:10.469748+0000","last_became_active":"2024-05-04T03:07:34.021126+0000","last_became_peered":"2024-05-04T03:07:34.021126+0000","last_unstale":"2024-05-04T03:11:10.469748+0000","last_undegraded":"2024-05-04T03:11:10.469748+0000","last_fullsized":"2024-05-04T03:11:10.469748+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:55.094800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65778,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":399,"num_read_kb":264,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999219+0000","last_change":"2024-05-04T03:07:37.066365+0000","last_active":"2024-05-04T03:09:44.999219+0000","last_peered":"2024-05-04T03:09:44.999219+0000","last_clean":"2024-05-04T03:09:44.999219+0000","last_became_active":"2024-05-04T03:07:37.065851+0000","last_became_peered":"2024-05-04T03:07:37.065851+0000","last_unstale":"2024-05-04T03:09:44.999219+0000","last_undegraded":"2024-05-04T03:09:44.999219+0000","last_fullsized":"2024-05-04T03:09:44.999219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:03:43.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.518904+0000","last_change":"2024-05-04T03:08:16.262600+0000","last_active":"2024-05-04T03:08:16.518904+0000","last_peered":"2024-05-04T03:08:16.518904+0000","last_clean":"2024-05-04T03:08:16.518904+0000","last_became_active":"2024-05-04T03:08:16.250619+0000","last_became_peered":"2024-05-04T03:08:16.250619+0000","last_unstale":"2024-05-04T03:08:16.518904+0000","last_undegraded":"2024-05-04T03:08:16.518904+0000","last_fullsized":"2024-05-04T03:08:16.518904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:51:21.596467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002712+0000","last_change":"2024-05-04T03:07:37.055356+0000","last_active":"2024-05-04T03:09:45.002712+0000","last_peered":"2024-05-04T03:09:45.002712+0000","last_clean":"2024-05-04T03:09:45.002712+0000","last_became_active":"2024-05-04T03:07:37.055045+0000","last_became_peered":"2024-05-04T03:07:37.055045+0000","last_unstale":"2024-05-04T03:09:45.002712+0000","last_undegraded":"2024-05-04T03:09:45.002712+0000","last_fullsized":"2024-05-04T03:09:45.002712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:06.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247445+0000","last_change":"2024-05-04T03:07:34.021525+0000","last_active":"2024-05-04T03:08:16.247445+0000","last_peered":"2024-05-04T03:08:16.247445+0000","last_clean":"2024-05-04T03:08:16.247445+0000","last_became_active":"2024-05-04T03:07:34.021042+0000","last_became_peered":"2024-05-04T03:07:34.021042+0000","last_unstale":"2024-05-04T03:08:16.247445+0000","last_undegraded":"2024-05-04T03:08:16.247445+0000","last_fullsized":"2024-05-04T03:08:16.247445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:44.090176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.004521+0000","last_change":"2024-05-04T03:07:37.066471+0000","last_active":"2024-05-04T03:09:45.004521+0000","last_peered":"2024-05-04T03:09:45.004521+0000","last_clean":"2024-05-04T03:09:45.004521+0000","last_became_active":"2024-05-04T03:07:37.066059+0000","last_became_peered":"2024-05-04T03:07:37.066059+0000","last_unstale":"2024-05-04T03:09:45.004521+0000","last_undegraded":"2024-05-04T03:09:45.004521+0000","last_fullsized":"2024-05-04T03:09:45.004521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:16.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'8","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:06.464575+0000","last_change":"2024-05-04T03:07:34.021531+0000","last_active":"2024-05-04T03:11:06.464575+0000","last_peered":"2024-05-04T03:11:06.464575+0000","last_clean":"2024-05-04T03:11:06.464575+0000","last_became_active":"2024-05-04T03:07:34.021112+0000","last_became_peered":"2024-05-04T03:07:34.021112+0000","last_unstale":"2024-05-04T03:11:06.464575+0000","last_undegraded":"2024-05-04T03:11:06.464575+0000","last_fullsized":"2024-05-04T03:11:06.464575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:28.826426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1597,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996662+0000","last_change":"2024-05-04T03:07:37.067155+0000","last_active":"2024-05-04T03:09:44.996662+0000","last_peered":"2024-05-04T03:09:44.996662+0000","last_clean":"2024-05-04T03:09:44.996662+0000","last_became_active":"2024-05-04T03:07:37.066783+0000","last_became_peered":"2024-05-04T03:07:37.066783+0000","last_unstale":"2024-05-04T03:09:44.996662+0000","last_undegraded":"2024-05-04T03:09:44.996662+0000","last_fullsized":"2024-05-04T03:09:44.996662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:23.217930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245106+0000","last_change":"2024-05-04T03:07:34.158227+0000","last_active":"2024-05-04T03:08:16.245106+0000","last_peered":"2024-05-04T03:08:16.245106+0000","last_clean":"2024-05-04T03:08:16.245106+0000","last_became_active":"2024-05-04T03:07:34.156905+0000","last_became_peered":"2024-05-04T03:07:34.156905+0000","last_unstale":"2024-05-04T03:08:16.245106+0000","last_undegraded":"2024-05-04T03:08:16.245106+0000","last_fullsized":"2024-05-04T03:08:16.245106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:40:41.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.006787+0000","last_change":"2024-05-04T03:07:37.075559+0000","last_active":"2024-05-04T03:09:45.006787+0000","last_peered":"2024-05-04T03:09:45.006787+0000","last_clean":"2024-05-04T03:09:45.006787+0000","last_became_active":"2024-05-04T03:07:37.075220+0000","last_became_peered":"2024-05-04T03:07:37.075220+0000","last_unstale":"2024-05-04T03:09:45.006787+0000","last_undegraded":"2024-05-04T03:09:45.006787+0000","last_fullsized":"2024-05-04T03:09:45.006787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:51:23.942514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:52.084062+0000","last_change":"2024-05-04T03:07:34.020326+0000","last_active":"2024-05-04T03:09:52.084062+0000","last_peered":"2024-05-04T03:09:52.084062+0000","last_clean":"2024-05-04T03:09:52.084062+0000","last_became_active":"2024-05-04T03:07:34.019472+0000","last_became_peered":"2024-05-04T03:07:34.019472+0000","last_unstale":"2024-05-04T03:09:52.084062+0000","last_undegraded":"2024-05-04T03:09:52.084062+0000","last_fullsized":"2024-05-04T03:09:52.084062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:33:28.375901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002619+0000","last_change":"2024-05-04T03:07:37.066223+0000","last_active":"2024-05-04T03:09:45.002619+0000","last_peered":"2024-05-04T03:09:45.002619+0000","last_clean":"2024-05-04T03:09:45.002619+0000","last_became_active":"2024-05-04T03:07:37.065859+0000","last_became_peered":"2024-05-04T03:07:37.065859+0000","last_unstale":"2024-05-04T03:09:45.002619+0000","last_undegraded":"2024-05-04T03:09:45.002619+0000","last_fullsized":"2024-05-04T03:09:45.002619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:44.902986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245539+0000","last_change":"2024-05-04T03:07:34.158674+0000","last_active":"2024-05-04T03:08:16.245539+0000","last_peered":"2024-05-04T03:08:16.245539+0000","last_clean":"2024-05-04T03:08:16.245539+0000","last_became_active":"2024-05-04T03:07:34.157952+0000","last_became_peered":"2024-05-04T03:07:34.157952+0000","last_unstale":"2024-05-04T03:08:16.245539+0000","last_undegraded":"2024-05-04T03:08:16.245539+0000","last_fullsized":"2024-05-04T03:08:16.245539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:19:11.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005664+0000","last_change":"2024-05-04T03:07:37.053897+0000","last_active":"2024-05-04T03:09:45.005664+0000","last_peered":"2024-05-04T03:09:45.005664+0000","last_clean":"2024-05-04T03:09:45.005664+0000","last_became_active":"2024-05-04T03:07:37.053466+0000","last_became_peered":"2024-05-04T03:07:37.053466+0000","last_unstale":"2024-05-04T03:09:45.005664+0000","last_undegraded":"2024-05-04T03:09:45.005664+0000","last_fullsized":"2024-05-04T03:09:45.005664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:50.650805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":224,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:05.478212+0000","last_change":"2024-05-04T03:07:34.021494+0000","last_active":"2024-05-04T03:11:05.478212+0000","last_peered":"2024-05-04T03:11:05.478212+0000","last_clean":"2024-05-04T03:11:05.478212+0000","last_became_active":"2024-05-04T03:07:34.021113+0000","last_became_peered":"2024-05-04T03:07:34.021113+0000","last_unstale":"2024-05-04T03:11:05.478212+0000","last_undegraded":"2024-05-04T03:11:05.478212+0000","last_fullsized":"2024-05-04T03:11:05.478212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:02:09.514961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007408+0000","last_change":"2024-05-04T03:07:37.068199+0000","last_active":"2024-05-04T03:09:45.007408+0000","last_peered":"2024-05-04T03:09:45.007408+0000","last_clean":"2024-05-04T03:09:45.007408+0000","last_became_active":"2024-05-04T03:07:37.067828+0000","last_became_peered":"2024-05-04T03:07:37.067828+0000","last_unstale":"2024-05-04T03:09:45.007408+0000","last_undegraded":"2024-05-04T03:09:45.007408+0000","last_fullsized":"2024-05-04T03:09:45.007408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:28.882920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996821+0000","last_change":"2024-05-04T03:07:37.069753+0000","last_active":"2024-05-04T03:09:44.996821+0000","last_peered":"2024-05-04T03:09:44.996821+0000","last_clean":"2024-05-04T03:09:44.996821+0000","last_became_active":"2024-05-04T03:07:37.069338+0000","last_became_peered":"2024-05-04T03:07:37.069338+0000","last_unstale":"2024-05-04T03:09:44.996821+0000","last_undegraded":"2024-05-04T03:09:44.996821+0000","last_fullsized":"2024-05-04T03:09:44.996821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:37:11.355749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'7","reported_seq":224,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:05.462880+0000","last_change":"2024-05-04T03:07:34.157229+0000","last_active":"2024-05-04T03:11:05.462880+0000","last_peered":"2024-05-04T03:11:05.462880+0000","last_clean":"2024-05-04T03:11:05.462880+0000","last_became_active":"2024-05-04T03:07:34.156776+0000","last_became_peered":"2024-05-04T03:07:34.156776+0000","last_unstale":"2024-05-04T03:11:05.462880+0000","last_undegraded":"2024-05-04T03:11:05.462880+0000","last_fullsized":"2024-05-04T03:11:05.462880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:26.879754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010488+0000","last_change":"2024-05-04T03:07:37.067025+0000","last_active":"2024-05-04T03:09:45.010488+0000","last_peered":"2024-05-04T03:09:45.010488+0000","last_clean":"2024-05-04T03:09:45.010488+0000","last_became_active":"2024-05-04T03:07:37.066359+0000","last_became_peered":"2024-05-04T03:07:37.066359+0000","last_unstale":"2024-05-04T03:09:45.010488+0000","last_undegraded":"2024-05-04T03:09:45.010488+0000","last_fullsized":"2024-05-04T03:09:45.010488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:44:35.604859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367184+0000","last_change":"2024-05-04T03:07:34.018019+0000","last_active":"2024-05-04T03:08:16.367184+0000","last_peered":"2024-05-04T03:08:16.367184+0000","last_clean":"2024-05-04T03:08:16.367184+0000","last_became_active":"2024-05-04T03:07:34.017556+0000","last_became_peered":"2024-05-04T03:07:34.017556+0000","last_unstale":"2024-05-04T03:08:16.367184+0000","last_undegraded":"2024-05-04T03:08:16.367184+0000","last_fullsized":"2024-05-04T03:08:16.367184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:31:34.158164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:52.260319+0000","last_change":"2024-05-04T03:06:06.704755+0000","last_active":"2024-05-04T03:08:52.260319+0000","last_peered":"2024-05-04T03:08:52.260319+0000","last_clean":"2024-05-04T03:08:52.260319+0000","last_became_active":"2024-05-04T03:06:06.698747+0000","last_became_peered":"2024-05-04T03:06:06.698747+0000","last_unstale":"2024-05-04T03:08:52.260319+0000","last_undegraded":"2024-05-04T03:08:52.260319+0000","last_fullsized":"2024-05-04T03:08:52.260319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_clean_scrub_stamp":"2024-05-04T03:02:43.385377+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:04:50.012414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008902+0000","last_change":"2024-05-04T03:07:37.057043+0000","last_active":"2024-05-04T03:09:45.008902+0000","last_peered":"2024-05-04T03:09:45.008902+0000","last_clean":"2024-05-04T03:09:45.008902+0000","last_became_active":"2024-05-04T03:07:37.056725+0000","last_became_peered":"2024-05-04T03:07:37.056725+0000","last_unstale":"2024-05-04T03:09:45.008902+0000","last_undegraded":"2024-05-04T03:09:45.008902+0000","last_fullsized":"2024-05-04T03:09:45.008902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:13:28.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.377000+0000","last_change":"2024-05-04T03:08:16.376510+0000","last_active":"2024-05-04T03:08:16.377000+0000","last_peered":"2024-05-04T03:08:16.377000+0000","last_clean":"2024-05-04T03:08:16.377000+0000","last_became_active":"2024-05-04T03:08:16.368625+0000","last_became_peered":"2024-05-04T03:08:16.368625+0000","last_unstale":"2024-05-04T03:08:16.377000+0000","last_undegraded":"2024-05-04T03:08:16.377000+0000","last_fullsized":"2024-05-04T03:08:16.377000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:27:50.504487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245652+0000","last_change":"2024-05-04T03:07:34.157346+0000","last_active":"2024-05-04T03:08:16.245652+0000","last_peered":"2024-05-04T03:08:16.245652+0000","last_clean":"2024-05-04T03:08:16.245652+0000","last_became_active":"2024-05-04T03:07:34.156914+0000","last_became_peered":"2024-05-04T03:07:34.156914+0000","last_unstale":"2024-05-04T03:08:16.245652+0000","last_undegraded":"2024-05-04T03:08:16.245652+0000","last_fullsized":"2024-05-04T03:08:16.245652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:54:17.059644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.987131+0000","last_change":"2024-05-04T03:07:37.054373+0000","last_active":"2024-05-04T03:09:44.987131+0000","last_peered":"2024-05-04T03:09:44.987131+0000","last_clean":"2024-05-04T03:09:44.987131+0000","last_became_active":"2024-05-04T03:07:37.053764+0000","last_became_peered":"2024-05-04T03:07:37.053764+0000","last_unstale":"2024-05-04T03:09:44.987131+0000","last_undegraded":"2024-05-04T03:09:44.987131+0000","last_fullsized":"2024-05-04T03:09:44.987131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:13:47.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'18","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:04.104348+0000","last_change":"2024-05-04T03:07:34.020190+0000","last_active":"2024-05-04T03:11:04.104348+0000","last_peered":"2024-05-04T03:11:04.104348+0000","last_clean":"2024-05-04T03:11:04.104348+0000","last_became_active":"2024-05-04T03:07:34.019786+0000","last_became_peered":"2024-05-04T03:07:34.019786+0000","last_unstale":"2024-05-04T03:11:04.104348+0000","last_undegraded":"2024-05-04T03:11:04.104348+0000","last_fullsized":"2024-05-04T03:11:04.104348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:11.243414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999034+0000","last_change":"2024-05-04T03:07:37.055274+0000","last_active":"2024-05-04T03:09:44.999034+0000","last_peered":"2024-05-04T03:09:44.999034+0000","last_clean":"2024-05-04T03:09:44.999034+0000","last_became_active":"2024-05-04T03:07:37.054968+0000","last_became_peered":"2024-05-04T03:07:37.054968+0000","last_unstale":"2024-05-04T03:09:44.999034+0000","last_undegraded":"2024-05-04T03:09:44.999034+0000","last_fullsized":"2024-05-04T03:09:44.999034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.229266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367534+0000","last_change":"2024-05-04T03:07:34.132966+0000","last_active":"2024-05-04T03:08:16.367534+0000","last_peered":"2024-05-04T03:08:16.367534+0000","last_clean":"2024-05-04T03:08:16.367534+0000","last_became_active":"2024-05-04T03:07:34.132529+0000","last_became_peered":"2024-05-04T03:07:34.132529+0000","last_unstale":"2024-05-04T03:08:16.367534+0000","last_undegraded":"2024-05-04T03:08:16.367534+0000","last_fullsized":"2024-05-04T03:08:16.367534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:31:37.122097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008076+0000","last_change":"2024-05-04T03:08:16.368747+0000","last_active":"2024-05-04T03:09:45.008076+0000","last_peered":"2024-05-04T03:09:45.008076+0000","last_clean":"2024-05-04T03:09:45.008076+0000","last_became_active":"2024-05-04T03:08:16.368376+0000","last_became_peered":"2024-05-04T03:08:16.368376+0000","last_unstale":"2024-05-04T03:09:45.008076+0000","last_undegraded":"2024-05-04T03:09:45.008076+0000","last_fullsized":"2024-05-04T03:09:45.008076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:42:32.085650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242516+0000","last_change":"2024-05-04T03:07:34.157219+0000","last_active":"2024-05-04T03:08:16.242516+0000","last_peered":"2024-05-04T03:08:16.242516+0000","last_clean":"2024-05-04T03:08:16.242516+0000","last_became_active":"2024-05-04T03:07:34.156866+0000","last_became_peered":"2024-05-04T03:07:34.156866+0000","last_unstale":"2024-05-04T03:08:16.242516+0000","last_undegraded":"2024-05-04T03:08:16.242516+0000","last_fullsized":"2024-05-04T03:08:16.242516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:15:34.138825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.990376+0000","last_change":"2024-05-04T03:07:37.058733+0000","last_active":"2024-05-04T03:09:44.990376+0000","last_peered":"2024-05-04T03:09:44.990376+0000","last_clean":"2024-05-04T03:09:44.990376+0000","last_became_active":"2024-05-04T03:07:37.058396+0000","last_became_peered":"2024-05-04T03:07:37.058396+0000","last_unstale":"2024-05-04T03:09:44.990376+0000","last_undegraded":"2024-05-04T03:09:44.990376+0000","last_fullsized":"2024-05-04T03:09:44.990376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:35.563728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243578+0000","last_change":"2024-05-04T03:07:34.016458+0000","last_active":"2024-05-04T03:08:16.243578+0000","last_peered":"2024-05-04T03:08:16.243578+0000","last_clean":"2024-05-04T03:08:16.243578+0000","last_became_active":"2024-05-04T03:07:34.016038+0000","last_became_peered":"2024-05-04T03:07:34.016038+0000","last_unstale":"2024-05-04T03:08:16.243578+0000","last_undegraded":"2024-05-04T03:08:16.243578+0000","last_fullsized":"2024-05-04T03:08:16.243578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:39:25.156047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998594+0000","last_change":"2024-05-04T03:07:37.064024+0000","last_active":"2024-05-04T03:09:44.998594+0000","last_peered":"2024-05-04T03:09:44.998594+0000","last_clean":"2024-05-04T03:09:44.998594+0000","last_became_active":"2024-05-04T03:07:37.063642+0000","last_became_peered":"2024-05-04T03:07:37.063642+0000","last_unstale":"2024-05-04T03:09:44.998594+0000","last_undegraded":"2024-05-04T03:09:44.998594+0000","last_fullsized":"2024-05-04T03:09:44.998594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:59:58.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'4","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:10:26.534919+0000","last_change":"2024-05-04T03:08:16.266763+0000","last_active":"2024-05-04T03:10:26.534919+0000","last_peered":"2024-05-04T03:10:26.534919+0000","last_clean":"2024-05-04T03:10:26.534919+0000","last_became_active":"2024-05-04T03:08:16.251212+0000","last_became_peered":"2024-05-04T03:08:16.251212+0000","last_unstale":"2024-05-04T03:10:26.534919+0000","last_undegraded":"2024-05-04T03:10:26.534919+0000","last_fullsized":"2024-05-04T03:10:26.534919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:31:27.444610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998226+0000","last_change":"2024-05-04T03:07:37.063753+0000","last_active":"2024-05-04T03:09:44.998226+0000","last_peered":"2024-05-04T03:09:44.998226+0000","last_clean":"2024-05-04T03:09:44.998226+0000","last_became_active":"2024-05-04T03:07:37.063318+0000","last_became_peered":"2024-05-04T03:07:37.063318+0000","last_unstale":"2024-05-04T03:09:44.998226+0000","last_undegraded":"2024-05-04T03:09:44.998226+0000","last_fullsized":"2024-05-04T03:09:44.998226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:48.217136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245182+0000","last_change":"2024-05-04T03:07:34.022522+0000","last_active":"2024-05-04T03:08:16.245182+0000","last_peered":"2024-05-04T03:08:16.245182+0000","last_clean":"2024-05-04T03:08:16.245182+0000","last_became_active":"2024-05-04T03:07:34.021018+0000","last_became_peered":"2024-05-04T03:07:34.021018+0000","last_unstale":"2024-05-04T03:08:16.245182+0000","last_undegraded":"2024-05-04T03:08:16.245182+0000","last_fullsized":"2024-05-04T03:08:16.245182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:47:04.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.014755+0000","last_change":"2024-05-04T03:07:37.069930+0000","last_active":"2024-05-04T03:09:45.014755+0000","last_peered":"2024-05-04T03:09:45.014755+0000","last_clean":"2024-05-04T03:09:45.014755+0000","last_became_active":"2024-05-04T03:07:37.069580+0000","last_became_peered":"2024-05-04T03:07:37.069580+0000","last_unstale":"2024-05-04T03:09:45.014755+0000","last_undegraded":"2024-05-04T03:09:45.014755+0000","last_fullsized":"2024-05-04T03:09:45.014755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:15:33.815970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":225,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:06.541656+0000","last_change":"2024-05-04T03:07:34.157969+0000","last_active":"2024-05-04T03:11:06.541656+0000","last_peered":"2024-05-04T03:11:06.541656+0000","last_clean":"2024-05-04T03:11:06.541656+0000","last_became_active":"2024-05-04T03:07:34.157563+0000","last_became_peered":"2024-05-04T03:07:34.157563+0000","last_unstale":"2024-05-04T03:11:06.541656+0000","last_undegraded":"2024-05-04T03:11:06.541656+0000","last_fullsized":"2024-05-04T03:11:06.541656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:00:43.562109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.015438+0000","last_change":"2024-05-04T03:07:37.071279+0000","last_active":"2024-05-04T03:09:45.015438+0000","last_peered":"2024-05-04T03:09:45.015438+0000","last_clean":"2024-05-04T03:09:45.015438+0000","last_became_active":"2024-05-04T03:07:37.070449+0000","last_became_peered":"2024-05-04T03:07:37.070449+0000","last_unstale":"2024-05-04T03:09:45.015438+0000","last_undegraded":"2024-05-04T03:09:45.015438+0000","last_fullsized":"2024-05-04T03:09:45.015438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:13:22.535788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248601+0000","last_change":"2024-05-04T03:07:34.017010+0000","last_active":"2024-05-04T03:08:16.248601+0000","last_peered":"2024-05-04T03:08:16.248601+0000","last_clean":"2024-05-04T03:08:16.248601+0000","last_became_active":"2024-05-04T03:07:34.016521+0000","last_became_peered":"2024-05-04T03:07:34.016521+0000","last_unstale":"2024-05-04T03:08:16.248601+0000","last_undegraded":"2024-05-04T03:08:16.248601+0000","last_fullsized":"2024-05-04T03:08:16.248601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:36.684656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.000434+0000","last_change":"2024-05-04T03:07:37.066526+0000","last_active":"2024-05-04T03:09:45.000434+0000","last_peered":"2024-05-04T03:09:45.000434+0000","last_clean":"2024-05-04T03:09:45.000434+0000","last_became_active":"2024-05-04T03:07:37.065312+0000","last_became_peered":"2024-05-04T03:07:37.065312+0000","last_unstale":"2024-05-04T03:09:45.000434+0000","last_undegraded":"2024-05-04T03:09:45.000434+0000","last_fullsized":"2024-05-04T03:09:45.000434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:10.077295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248987+0000","last_change":"2024-05-04T03:07:34.016898+0000","last_active":"2024-05-04T03:08:16.248987+0000","last_peered":"2024-05-04T03:08:16.248987+0000","last_clean":"2024-05-04T03:08:16.248987+0000","last_became_active":"2024-05-04T03:07:34.016524+0000","last_became_peered":"2024-05-04T03:07:34.016524+0000","last_unstale":"2024-05-04T03:08:16.248987+0000","last_undegraded":"2024-05-04T03:08:16.248987+0000","last_fullsized":"2024-05-04T03:08:16.248987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:01.236207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.001328+0000","last_change":"2024-05-04T03:07:37.061501+0000","last_active":"2024-05-04T03:09:45.001328+0000","last_peered":"2024-05-04T03:09:45.001328+0000","last_clean":"2024-05-04T03:09:45.001328+0000","last_became_active":"2024-05-04T03:07:37.061077+0000","last_became_peered":"2024-05-04T03:07:37.061077+0000","last_unstale":"2024-05-04T03:09:45.001328+0000","last_undegraded":"2024-05-04T03:09:45.001328+0000","last_fullsized":"2024-05-04T03:09:45.001328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:32.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996375+0000","last_change":"2024-05-04T03:07:37.064612+0000","last_active":"2024-05-04T03:09:44.996375+0000","last_peered":"2024-05-04T03:09:44.996375+0000","last_clean":"2024-05-04T03:09:44.996375+0000","last_became_active":"2024-05-04T03:07:37.064303+0000","last_became_peered":"2024-05-04T03:07:37.064303+0000","last_unstale":"2024-05-04T03:09:44.996375+0000","last_undegraded":"2024-05-04T03:09:44.996375+0000","last_fullsized":"2024-05-04T03:09:44.996375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:47:07.811765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005934+0000","last_change":"2024-05-04T03:07:37.061577+0000","last_active":"2024-05-04T03:09:45.005934+0000","last_peered":"2024-05-04T03:09:45.005934+0000","last_clean":"2024-05-04T03:09:45.005934+0000","last_became_active":"2024-05-04T03:07:37.061214+0000","last_became_peered":"2024-05-04T03:07:37.061214+0000","last_unstale":"2024-05-04T03:09:45.005934+0000","last_undegraded":"2024-05-04T03:09:45.005934+0000","last_fullsized":"2024-05-04T03:09:45.005934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:52:11.733257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005073+0000","last_change":"2024-05-04T03:07:37.062906+0000","last_active":"2024-05-04T03:09:45.005073+0000","last_peered":"2024-05-04T03:09:45.005073+0000","last_clean":"2024-05-04T03:09:45.005073+0000","last_became_active":"2024-05-04T03:07:37.062510+0000","last_became_peered":"2024-05-04T03:07:37.062510+0000","last_unstale":"2024-05-04T03:09:45.005073+0000","last_undegraded":"2024-05-04T03:09:45.005073+0000","last_fullsized":"2024-05-04T03:09:45.005073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:57:58.910490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008959+0000","last_change":"2024-05-04T03:07:37.068367+0000","last_active":"2024-05-04T03:09:45.008959+0000","last_peered":"2024-05-04T03:09:45.008959+0000","last_clean":"2024-05-04T03:09:45.008959+0000","last_became_active":"2024-05-04T03:07:37.068038+0000","last_became_peered":"2024-05-04T03:07:37.068038+0000","last_unstale":"2024-05-04T03:09:45.008959+0000","last_undegraded":"2024-05-04T03:09:45.008959+0000","last_fullsized":"2024-05-04T03:09:45.008959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:11:19.195002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003363+0000","last_change":"2024-05-04T03:08:16.250759+0000","last_active":"2024-05-04T03:09:45.003363+0000","last_peered":"2024-05-04T03:09:45.003363+0000","last_clean":"2024-05-04T03:09:45.003363+0000","last_became_active":"2024-05-04T03:08:16.250379+0000","last_became_peered":"2024-05-04T03:08:16.250379+0000","last_unstale":"2024-05-04T03:09:45.003363+0000","last_undegraded":"2024-05-04T03:09:45.003363+0000","last_fullsized":"2024-05-04T03:09:45.003363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:58:09.419009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.991925+0000","last_change":"2024-05-04T03:07:37.064584+0000","last_active":"2024-05-04T03:09:44.991925+0000","last_peered":"2024-05-04T03:09:44.991925+0000","last_clean":"2024-05-04T03:09:44.991925+0000","last_became_active":"2024-05-04T03:07:37.064183+0000","last_became_peered":"2024-05-04T03:07:37.064183+0000","last_unstale":"2024-05-04T03:09:44.991925+0000","last_undegraded":"2024-05-04T03:09:44.991925+0000","last_fullsized":"2024-05-04T03:09:44.991925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:17.121081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.013789+0000","last_change":"2024-05-04T03:07:37.066742+0000","last_active":"2024-05-04T03:09:45.013789+0000","last_peered":"2024-05-04T03:09:45.013789+0000","last_clean":"2024-05-04T03:09:45.013789+0000","last_became_active":"2024-05-04T03:07:37.066126+0000","last_became_peered":"2024-05-04T03:07:37.066126+0000","last_unstale":"2024-05-04T03:09:45.013789+0000","last_undegraded":"2024-05-04T03:09:45.013789+0000","last_fullsized":"2024-05-04T03:09:45.013789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:57.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.997576+0000","last_change":"2024-05-04T03:07:37.068122+0000","last_active":"2024-05-04T03:09:44.997576+0000","last_peered":"2024-05-04T03:09:44.997576+0000","last_clean":"2024-05-04T03:09:44.997576+0000","last_became_active":"2024-05-04T03:07:37.066321+0000","last_became_peered":"2024-05-04T03:07:37.066321+0000","last_unstale":"2024-05-04T03:09:44.997576+0000","last_undegraded":"2024-05-04T03:09:44.997576+0000","last_fullsized":"2024-05-04T03:09:44.997576+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:25:09.321853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994963+0000","last_change":"2024-05-04T03:07:37.060322+0000","last_active":"2024-05-04T03:09:44.994963+0000","last_peered":"2024-05-04T03:09:44.994963+0000","last_clean":"2024-05-04T03:09:44.994963+0000","last_became_active":"2024-05-04T03:07:37.059944+0000","last_became_peered":"2024-05-04T03:07:37.059944+0000","last_unstale":"2024-05-04T03:09:44.994963+0000","last_undegraded":"2024-05-04T03:09:44.994963+0000","last_fullsized":"2024-05-04T03:09:44.994963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.105085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.993232+0000","last_change":"2024-05-04T03:07:37.056269+0000","last_active":"2024-05-04T03:09:44.993232+0000","last_peered":"2024-05-04T03:09:44.993232+0000","last_clean":"2024-05-04T03:09:44.993232+0000","last_became_active":"2024-05-04T03:07:37.055886+0000","last_became_peered":"2024-05-04T03:07:37.055886+0000","last_unstale":"2024-05-04T03:09:44.993232+0000","last_undegraded":"2024-05-04T03:09:44.993232+0000","last_fullsized":"2024-05-04T03:09:44.993232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:55:11.292577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017574+0000","last_change":"2024-05-04T03:07:37.071013+0000","last_active":"2024-05-04T03:09:45.017574+0000","last_peered":"2024-05-04T03:09:45.017574+0000","last_clean":"2024-05-04T03:09:45.017574+0000","last_became_active":"2024-05-04T03:07:37.069346+0000","last_became_peered":"2024-05-04T03:07:37.069346+0000","last_unstale":"2024-05-04T03:09:45.017574+0000","last_undegraded":"2024-05-04T03:09:45.017574+0000","last_fullsized":"2024-05-04T03:09:45.017574+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:19:11.898766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003394+0000","last_change":"2024-05-04T03:07:37.064962+0000","last_active":"2024-05-04T03:09:45.003394+0000","last_peered":"2024-05-04T03:09:45.003394+0000","last_clean":"2024-05-04T03:09:45.003394+0000","last_became_active":"2024-05-04T03:07:37.064637+0000","last_became_peered":"2024-05-04T03:07:37.064637+0000","last_unstale":"2024-05-04T03:09:45.003394+0000","last_undegraded":"2024-05-04T03:09:45.003394+0000","last_fullsized":"2024-05-04T03:09:45.003394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:10:10.659621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002661+0000","last_change":"2024-05-04T03:07:37.056721+0000","last_active":"2024-05-04T03:09:45.002661+0000","last_peered":"2024-05-04T03:09:45.002661+0000","last_clean":"2024-05-04T03:09:45.002661+0000","last_became_active":"2024-05-04T03:07:37.056353+0000","last_became_peered":"2024-05-04T03:07:37.056353+0000","last_unstale":"2024-05-04T03:09:45.002661+0000","last_undegraded":"2024-05-04T03:09:45.002661+0000","last_fullsized":"2024-05-04T03:09:45.002661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:30:47.476579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286351,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75031,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1962,"num_read_kb":1066,"num_write":272,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":224702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710779,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":426309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1083580,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":425694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266762,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":426072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462991,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":1020162,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626517,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":425888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":492579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":427618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":493599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182500,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":425236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986271,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":426281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822746,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":1018354,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:11:12.974 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-04T03:11:12.974 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:11:13.231 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:13.382 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:13 smithi039 ceph-mon[29715]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:13.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:13 smithi167 ceph-mon[33614]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:13.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:13 smithi066 ceph-mon[33264]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:14.390 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:14 smithi039 ceph-mon[29715]: pgmap v399: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-04T03:11:14.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:14 smithi167 ceph-mon[33614]: pgmap v399: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-04T03:11:14.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:14 smithi066 ceph-mon[33264]: pgmap v399: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-04T03:11:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.960+0000 7f5378298700 1 -- 172.21.15.39:0/2242833405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370108740 msgr2=0x7f53701100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.960+0000 7f5378298700 1 --2- 172.21.15.39:0/2242833405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370108740 0x7f53701100a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f536800b0d0 tx=0x7f536800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:11:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 -- 172.21.15.39:0/2242833405 shutdown_connections 2024-05-04T03:11:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 --2- 172.21.15.39:0/2242833405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370108740 0x7f53701100a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 --2- 172.21.15.39:0/2242833405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53701034c0 0x7f53701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 --2- 172.21.15.39:0/2242833405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5370107df0 0x7f5370108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 -- 172.21.15.39:0/2242833405 >> 172.21.15.39:0/2242833405 conn(0x7f53700fedc0 msgr2=0x7f53701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 -- 172.21.15.39:0/2242833405 shutdown_connections 2024-05-04T03:11:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 -- 172.21.15.39:0/2242833405 wait complete. 2024-05-04T03:11:14.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 Processor -- start 2024-05-04T03:11:14.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.961+0000 7f5378298700 1 -- start start 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5378298700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53701034c0 0x7f537019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5378298700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370107df0 0x7f537019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5378298700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5370108740 0x7f537019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5378298700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f53701132a0 con 0x7f5370107df0 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5378298700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5370113120 con 0x7f5370108740 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5378298700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5370113420 con 0x7f53701034c0 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370107df0 0x7f537019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370107df0 0x7f537019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58070/0 (socket says 172.21.15.39:58070) 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 -- 172.21.15.39:0/2345147507 learned_addr learned my addr 172.21.15.39:0/2345147507 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5376034700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53701034c0 0x7f537019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5376835700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5370108740 0x7f537019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 -- 172.21.15.39:0/2345147507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53701034c0 msgr2=0x7f537019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53701034c0 0x7f537019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 -- 172.21.15.39:0/2345147507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5370108740 msgr2=0x7f537019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5370108740 0x7f537019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 -- 172.21.15.39:0/2345147507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537019fc10 con 0x7f5370107df0 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.962+0000 7f5375833700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370107df0 0x7f537019b2e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c8f0 tx=0x7f536c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.963+0000 7f53637fe700 1 -- 172.21.15.39:0/2345147507 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c007b00 con 0x7f5370107df0 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.963+0000 7f53637fe700 1 -- 172.21.15.39:0/2345147507 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f536c007ca0 con 0x7f5370107df0 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.963+0000 7f53637fe700 1 -- 172.21.15.39:0/2345147507 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c018660 con 0x7f5370107df0 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.963+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537019ff00 con 0x7f5370107df0 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.963+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53700ff1d0 con 0x7f5370107df0 2024-05-04T03:11:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.964+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5370104970 con 0x7f5370107df0 2024-05-04T03:11:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.965+0000 7f53637fe700 1 -- 172.21.15.39:0/2345147507 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f536c00f450 con 0x7f5370107df0 2024-05-04T03:11:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.966+0000 7f53637fe700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f535c0b2910 0x7f535c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.966+0000 7f5376034700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f535c0b2910 0x7f535c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.966+0000 7f53637fe700 1 -- 172.21.15.39:0/2345147507 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f536c0e2430 con 0x7f5370107df0 2024-05-04T03:11:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.967+0000 7f5376034700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f535c0b2910 0x7f535c0b4e20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5364006fd0 tx=0x7f5364008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:14.969+0000 7f53637fe700 1 -- 172.21.15.39:0/2345147507 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f536c014030 con 0x7f5370107df0 2024-05-04T03:11:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.183+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 --> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f537019c0a0 con 0x7f535c0b2910 2024-05-04T03:11:15.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.190+0000 7f53637fe700 1 -- 172.21.15.39:0/2345147507 <== mgr.14152 v2:172.21.15.39:6800/2627629846 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259049 (secure 0 0 0) 0x7f537019c0a0 con 0x7f535c0b2910 2024-05-04T03:11:15.195 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f535c0b2910 msgr2=0x7f535c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f535c0b2910 0x7f535c0b4e20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5364006fd0 tx=0x7f5364008040 comp rx=0 tx=0).stop 2024-05-04T03:11:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370107df0 msgr2=0x7f537019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370107df0 0x7f537019b2e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c8f0 tx=0x7f536c00ccb0 comp rx=0 tx=0).stop 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 shutdown_connections 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f535c0b2910 0x7f535c0b4e20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5370108740 0x7f537019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53701034c0 0x7f537019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 --2- 172.21.15.39:0/2345147507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5370107df0 0x7f537019b2e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 >> 172.21.15.39:0/2345147507 conn(0x7f53700fedc0 msgr2=0x7f537006b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.193+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 shutdown_connections 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:15.194+0000 7f5378298700 1 -- 172.21.15.39:0/2345147507 wait complete. 2024-05-04T03:11:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-05-04T03:11:15.880 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-05-04T03:11:15.054424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665633,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2064,"num_read_kb":1141,"num_write":399,"num_write_kb":2314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":2951908,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":827,"ondisk_log_size":827,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376612,"kb_used_data":10820,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565212,"statfs":{"total":1151940427776,"available":1151554777088,"internally_reserved":0,"allocated":11079680,"data_stored":7096803,"data_compressed":109203,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7481,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":67,"num_write":23,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998775+0000","last_change":"2024-05-04T03:07:37.070347+0000","last_active":"2024-05-04T03:09:44.998775+0000","last_peered":"2024-05-04T03:09:44.998775+0000","last_clean":"2024-05-04T03:09:44.998775+0000","last_became_active":"2024-05-04T03:07:37.069997+0000","last_became_peered":"2024-05-04T03:07:37.069997+0000","last_unstale":"2024-05-04T03:09:44.998775+0000","last_undegraded":"2024-05-04T03:09:44.998775+0000","last_fullsized":"2024-05-04T03:09:44.998775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:24:11.362228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010264+0000","last_change":"2024-05-04T03:07:37.071598+0000","last_active":"2024-05-04T03:09:45.010264+0000","last_peered":"2024-05-04T03:09:45.010264+0000","last_clean":"2024-05-04T03:09:45.010264+0000","last_became_active":"2024-05-04T03:07:37.071059+0000","last_became_peered":"2024-05-04T03:07:37.071059+0000","last_unstale":"2024-05-04T03:09:45.010264+0000","last_undegraded":"2024-05-04T03:09:45.010264+0000","last_fullsized":"2024-05-04T03:09:45.010264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:33:33.929246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017072+0000","last_change":"2024-05-04T03:07:37.069951+0000","last_active":"2024-05-04T03:09:45.017072+0000","last_peered":"2024-05-04T03:09:45.017072+0000","last_clean":"2024-05-04T03:09:45.017072+0000","last_became_active":"2024-05-04T03:07:37.069558+0000","last_became_peered":"2024-05-04T03:07:37.069558+0000","last_unstale":"2024-05-04T03:09:45.017072+0000","last_undegraded":"2024-05-04T03:09:45.017072+0000","last_fullsized":"2024-05-04T03:09:45.017072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:37:52.596455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.001811+0000","last_change":"2024-05-04T03:07:37.068820+0000","last_active":"2024-05-04T03:09:45.001811+0000","last_peered":"2024-05-04T03:09:45.001811+0000","last_clean":"2024-05-04T03:09:45.001811+0000","last_became_active":"2024-05-04T03:07:37.068323+0000","last_became_peered":"2024-05-04T03:07:37.068323+0000","last_unstale":"2024-05-04T03:09:45.001811+0000","last_undegraded":"2024-05-04T03:09:45.001811+0000","last_fullsized":"2024-05-04T03:09:45.001811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:52:54.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.981848+0000","last_change":"2024-05-04T03:07:37.061982+0000","last_active":"2024-05-04T03:09:44.981848+0000","last_peered":"2024-05-04T03:09:44.981848+0000","last_clean":"2024-05-04T03:09:44.981848+0000","last_became_active":"2024-05-04T03:07:37.061621+0000","last_became_peered":"2024-05-04T03:07:37.061621+0000","last_unstale":"2024-05-04T03:09:44.981848+0000","last_undegraded":"2024-05-04T03:09:44.981848+0000","last_fullsized":"2024-05-04T03:09:44.981848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:17:47.421011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003074+0000","last_change":"2024-05-04T03:07:37.058641+0000","last_active":"2024-05-04T03:09:45.003074+0000","last_peered":"2024-05-04T03:09:45.003074+0000","last_clean":"2024-05-04T03:09:45.003074+0000","last_became_active":"2024-05-04T03:07:37.058235+0000","last_became_peered":"2024-05-04T03:07:37.058235+0000","last_unstale":"2024-05-04T03:09:45.003074+0000","last_undegraded":"2024-05-04T03:09:45.003074+0000","last_fullsized":"2024-05-04T03:09:45.003074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:17:45.441777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998905+0000","last_change":"2024-05-04T03:07:37.065959+0000","last_active":"2024-05-04T03:09:44.998905+0000","last_peered":"2024-05-04T03:09:44.998905+0000","last_clean":"2024-05-04T03:09:44.998905+0000","last_became_active":"2024-05-04T03:07:37.065508+0000","last_became_peered":"2024-05-04T03:07:37.065508+0000","last_unstale":"2024-05-04T03:09:44.998905+0000","last_undegraded":"2024-05-04T03:09:44.998905+0000","last_fullsized":"2024-05-04T03:09:44.998905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:24:17.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994908+0000","last_change":"2024-05-04T03:07:37.065620+0000","last_active":"2024-05-04T03:09:44.994908+0000","last_peered":"2024-05-04T03:09:44.994908+0000","last_clean":"2024-05-04T03:09:44.994908+0000","last_became_active":"2024-05-04T03:07:37.065261+0000","last_became_peered":"2024-05-04T03:07:37.065261+0000","last_unstale":"2024-05-04T03:09:44.994908+0000","last_undegraded":"2024-05-04T03:09:44.994908+0000","last_fullsized":"2024-05-04T03:09:44.994908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:08:27.255791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017270+0000","last_change":"2024-05-04T03:07:37.072204+0000","last_active":"2024-05-04T03:09:45.017270+0000","last_peered":"2024-05-04T03:09:45.017270+0000","last_clean":"2024-05-04T03:09:45.017270+0000","last_became_active":"2024-05-04T03:07:37.070622+0000","last_became_peered":"2024-05-04T03:07:37.070622+0000","last_unstale":"2024-05-04T03:09:45.017270+0000","last_undegraded":"2024-05-04T03:09:45.017270+0000","last_fullsized":"2024-05-04T03:09:45.017270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:55:25.931369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003159+0000","last_change":"2024-05-04T03:08:16.253125+0000","last_active":"2024-05-04T03:09:45.003159+0000","last_peered":"2024-05-04T03:09:45.003159+0000","last_clean":"2024-05-04T03:09:45.003159+0000","last_became_active":"2024-05-04T03:08:16.252682+0000","last_became_peered":"2024-05-04T03:08:16.252682+0000","last_unstale":"2024-05-04T03:09:45.003159+0000","last_undegraded":"2024-05-04T03:09:45.003159+0000","last_fullsized":"2024-05-04T03:09:45.003159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:30:05.933491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.015463+0000","last_change":"2024-05-04T03:07:37.071614+0000","last_active":"2024-05-04T03:09:45.015463+0000","last_peered":"2024-05-04T03:09:45.015463+0000","last_clean":"2024-05-04T03:09:45.015463+0000","last_became_active":"2024-05-04T03:07:37.070927+0000","last_became_peered":"2024-05-04T03:07:37.070927+0000","last_unstale":"2024-05-04T03:09:45.015463+0000","last_undegraded":"2024-05-04T03:09:45.015463+0000","last_fullsized":"2024-05-04T03:09:45.015463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:30.432959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005280+0000","last_change":"2024-05-04T03:07:37.069305+0000","last_active":"2024-05-04T03:09:45.005280+0000","last_peered":"2024-05-04T03:09:45.005280+0000","last_clean":"2024-05-04T03:09:45.005280+0000","last_became_active":"2024-05-04T03:07:37.068804+0000","last_became_peered":"2024-05-04T03:07:37.068804+0000","last_unstale":"2024-05-04T03:09:45.005280+0000","last_undegraded":"2024-05-04T03:09:45.005280+0000","last_fullsized":"2024-05-04T03:09:45.005280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:43.693161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.006446+0000","last_change":"2024-05-04T03:07:37.062848+0000","last_active":"2024-05-04T03:09:45.006446+0000","last_peered":"2024-05-04T03:09:45.006446+0000","last_clean":"2024-05-04T03:09:45.006446+0000","last_became_active":"2024-05-04T03:07:37.062548+0000","last_became_peered":"2024-05-04T03:07:37.062548+0000","last_unstale":"2024-05-04T03:09:45.006446+0000","last_undegraded":"2024-05-04T03:09:45.006446+0000","last_fullsized":"2024-05-04T03:09:45.006446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010679+0000","last_change":"2024-05-04T03:07:37.062598+0000","last_active":"2024-05-04T03:09:45.010679+0000","last_peered":"2024-05-04T03:09:45.010679+0000","last_clean":"2024-05-04T03:09:45.010679+0000","last_became_active":"2024-05-04T03:07:37.061711+0000","last_became_peered":"2024-05-04T03:07:37.061711+0000","last_unstale":"2024-05-04T03:09:45.010679+0000","last_undegraded":"2024-05-04T03:09:45.010679+0000","last_fullsized":"2024-05-04T03:09:45.010679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:29:03.153680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007786+0000","last_change":"2024-05-04T03:07:37.067889+0000","last_active":"2024-05-04T03:09:45.007786+0000","last_peered":"2024-05-04T03:09:45.007786+0000","last_clean":"2024-05-04T03:09:45.007786+0000","last_became_active":"2024-05-04T03:07:37.066768+0000","last_became_peered":"2024-05-04T03:07:37.066768+0000","last_unstale":"2024-05-04T03:09:45.007786+0000","last_undegraded":"2024-05-04T03:09:45.007786+0000","last_fullsized":"2024-05-04T03:09:45.007786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:01:18.136379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017415+0000","last_change":"2024-05-04T03:07:37.070438+0000","last_active":"2024-05-04T03:09:45.017415+0000","last_peered":"2024-05-04T03:09:45.017415+0000","last_clean":"2024-05-04T03:09:45.017415+0000","last_became_active":"2024-05-04T03:07:37.070007+0000","last_became_peered":"2024-05-04T03:07:37.070007+0000","last_unstale":"2024-05-04T03:09:45.017415+0000","last_undegraded":"2024-05-04T03:09:45.017415+0000","last_fullsized":"2024-05-04T03:09:45.017415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:22.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999016+0000","last_change":"2024-05-04T03:07:37.070020+0000","last_active":"2024-05-04T03:09:44.999016+0000","last_peered":"2024-05-04T03:09:44.999016+0000","last_clean":"2024-05-04T03:09:44.999016+0000","last_became_active":"2024-05-04T03:07:37.069629+0000","last_became_peered":"2024-05-04T03:07:37.069629+0000","last_unstale":"2024-05-04T03:09:44.999016+0000","last_undegraded":"2024-05-04T03:09:44.999016+0000","last_fullsized":"2024-05-04T03:09:44.999016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:28.541752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.011076+0000","last_change":"2024-05-04T03:07:37.068651+0000","last_active":"2024-05-04T03:09:45.011076+0000","last_peered":"2024-05-04T03:09:45.011076+0000","last_clean":"2024-05-04T03:09:45.011076+0000","last_became_active":"2024-05-04T03:07:37.067135+0000","last_became_peered":"2024-05-04T03:07:37.067135+0000","last_unstale":"2024-05-04T03:09:45.011076+0000","last_undegraded":"2024-05-04T03:09:45.011076+0000","last_fullsized":"2024-05-04T03:09:45.011076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:00:59.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994607+0000","last_change":"2024-05-04T03:07:37.055419+0000","last_active":"2024-05-04T03:09:44.994607+0000","last_peered":"2024-05-04T03:09:44.994607+0000","last_clean":"2024-05-04T03:09:44.994607+0000","last_became_active":"2024-05-04T03:07:37.055024+0000","last_became_peered":"2024-05-04T03:07:37.055024+0000","last_unstale":"2024-05-04T03:09:44.994607+0000","last_undegraded":"2024-05-04T03:09:44.994607+0000","last_fullsized":"2024-05-04T03:09:44.994607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:55:05.506505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'14","reported_seq":237,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:12.489401+0000","last_change":"2024-05-04T03:07:34.019975+0000","last_active":"2024-05-04T03:11:12.489401+0000","last_peered":"2024-05-04T03:11:12.489401+0000","last_clean":"2024-05-04T03:11:12.489401+0000","last_became_active":"2024-05-04T03:07:34.019532+0000","last_became_peered":"2024-05-04T03:07:34.019532+0000","last_unstale":"2024-05-04T03:11:12.489401+0000","last_undegraded":"2024-05-04T03:11:12.489401+0000","last_fullsized":"2024-05-04T03:11:12.489401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:11:03.656329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":197,"num_read_kb":197,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.990313+0000","last_change":"2024-05-04T03:07:37.064725+0000","last_active":"2024-05-04T03:09:44.990313+0000","last_peered":"2024-05-04T03:09:44.990313+0000","last_clean":"2024-05-04T03:09:44.990313+0000","last_became_active":"2024-05-04T03:07:37.064385+0000","last_became_peered":"2024-05-04T03:07:37.064385+0000","last_unstale":"2024-05-04T03:09:44.990313+0000","last_undegraded":"2024-05-04T03:09:44.990313+0000","last_fullsized":"2024-05-04T03:09:44.990313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:22:12.024328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.244201+0000","last_change":"2024-05-04T03:07:34.157179+0000","last_active":"2024-05-04T03:08:16.244201+0000","last_peered":"2024-05-04T03:08:16.244201+0000","last_clean":"2024-05-04T03:08:16.244201+0000","last_became_active":"2024-05-04T03:07:34.156790+0000","last_became_peered":"2024-05-04T03:07:34.156790+0000","last_unstale":"2024-05-04T03:08:16.244201+0000","last_undegraded":"2024-05-04T03:08:16.244201+0000","last_fullsized":"2024-05-04T03:08:16.244201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:32.066410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017111+0000","last_change":"2024-05-04T03:07:37.070481+0000","last_active":"2024-05-04T03:09:45.017111+0000","last_peered":"2024-05-04T03:09:45.017111+0000","last_clean":"2024-05-04T03:09:45.017111+0000","last_became_active":"2024-05-04T03:07:37.069903+0000","last_became_peered":"2024-05-04T03:07:37.069903+0000","last_unstale":"2024-05-04T03:09:45.017111+0000","last_undegraded":"2024-05-04T03:09:45.017111+0000","last_fullsized":"2024-05-04T03:09:45.017111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:45:40.764716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245782+0000","last_change":"2024-05-04T03:07:34.023978+0000","last_active":"2024-05-04T03:08:16.245782+0000","last_peered":"2024-05-04T03:08:16.245782+0000","last_clean":"2024-05-04T03:08:16.245782+0000","last_became_active":"2024-05-04T03:07:34.023199+0000","last_became_peered":"2024-05-04T03:07:34.023199+0000","last_unstale":"2024-05-04T03:08:16.245782+0000","last_undegraded":"2024-05-04T03:08:16.245782+0000","last_fullsized":"2024-05-04T03:08:16.245782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:17:43.885103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017273+0000","last_change":"2024-05-04T03:07:37.072661+0000","last_active":"2024-05-04T03:09:45.017273+0000","last_peered":"2024-05-04T03:09:45.017273+0000","last_clean":"2024-05-04T03:09:45.017273+0000","last_became_active":"2024-05-04T03:07:37.071393+0000","last_became_peered":"2024-05-04T03:07:37.071393+0000","last_unstale":"2024-05-04T03:09:45.017273+0000","last_undegraded":"2024-05-04T03:09:45.017273+0000","last_fullsized":"2024-05-04T03:09:45.017273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:39:39.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.246044+0000","last_change":"2024-05-04T03:07:34.018543+0000","last_active":"2024-05-04T03:08:16.246044+0000","last_peered":"2024-05-04T03:08:16.246044+0000","last_clean":"2024-05-04T03:08:16.246044+0000","last_became_active":"2024-05-04T03:07:34.018145+0000","last_became_peered":"2024-05-04T03:07:34.018145+0000","last_unstale":"2024-05-04T03:08:16.246044+0000","last_undegraded":"2024-05-04T03:08:16.246044+0000","last_fullsized":"2024-05-04T03:08:16.246044+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:23:44.529492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007771+0000","last_change":"2024-05-04T03:07:37.070811+0000","last_active":"2024-05-04T03:09:45.007771+0000","last_peered":"2024-05-04T03:09:45.007771+0000","last_clean":"2024-05-04T03:09:45.007771+0000","last_became_active":"2024-05-04T03:07:37.070441+0000","last_became_peered":"2024-05-04T03:07:37.070441+0000","last_unstale":"2024-05-04T03:09:45.007771+0000","last_undegraded":"2024-05-04T03:09:45.007771+0000","last_fullsized":"2024-05-04T03:09:45.007771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:01:48.146339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:09.469193+0000","last_change":"2024-05-04T03:07:34.020508+0000","last_active":"2024-05-04T03:11:09.469193+0000","last_peered":"2024-05-04T03:11:09.469193+0000","last_clean":"2024-05-04T03:11:09.469193+0000","last_became_active":"2024-05-04T03:07:34.020076+0000","last_became_peered":"2024-05-04T03:07:34.020076+0000","last_unstale":"2024-05-04T03:11:09.469193+0000","last_undegraded":"2024-05-04T03:11:09.469193+0000","last_fullsized":"2024-05-04T03:11:09.469193+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:27.365257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1099,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.988466+0000","last_change":"2024-05-04T03:07:37.060004+0000","last_active":"2024-05-04T03:09:44.988466+0000","last_peered":"2024-05-04T03:09:44.988466+0000","last_clean":"2024-05-04T03:09:44.988466+0000","last_became_active":"2024-05-04T03:07:37.059350+0000","last_became_peered":"2024-05-04T03:07:37.059350+0000","last_unstale":"2024-05-04T03:09:44.988466+0000","last_undegraded":"2024-05-04T03:09:44.988466+0000","last_fullsized":"2024-05-04T03:09:44.988466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:37:07.315935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248143+0000","last_change":"2024-05-04T03:07:34.020174+0000","last_active":"2024-05-04T03:08:16.248143+0000","last_peered":"2024-05-04T03:08:16.248143+0000","last_clean":"2024-05-04T03:08:16.248143+0000","last_became_active":"2024-05-04T03:07:34.019790+0000","last_became_peered":"2024-05-04T03:07:34.019790+0000","last_unstale":"2024-05-04T03:08:16.248143+0000","last_undegraded":"2024-05-04T03:08:16.248143+0000","last_fullsized":"2024-05-04T03:08:16.248143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:37:37.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.987213+0000","last_change":"2024-05-04T03:08:16.251277+0000","last_active":"2024-05-04T03:09:44.987213+0000","last_peered":"2024-05-04T03:09:44.987213+0000","last_clean":"2024-05-04T03:09:44.987213+0000","last_became_active":"2024-05-04T03:08:16.250836+0000","last_became_peered":"2024-05-04T03:08:16.250836+0000","last_unstale":"2024-05-04T03:09:44.987213+0000","last_undegraded":"2024-05-04T03:09:44.987213+0000","last_fullsized":"2024-05-04T03:09:44.987213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:59:47.451789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248454+0000","last_change":"2024-05-04T03:07:34.013188+0000","last_active":"2024-05-04T03:08:16.248454+0000","last_peered":"2024-05-04T03:08:16.248454+0000","last_clean":"2024-05-04T03:08:16.248454+0000","last_became_active":"2024-05-04T03:07:34.012759+0000","last_became_peered":"2024-05-04T03:07:34.012759+0000","last_unstale":"2024-05-04T03:08:16.248454+0000","last_undegraded":"2024-05-04T03:08:16.248454+0000","last_fullsized":"2024-05-04T03:08:16.248454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:54:29.621250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.004186+0000","last_change":"2024-05-04T03:07:37.056265+0000","last_active":"2024-05-04T03:09:45.004186+0000","last_peered":"2024-05-04T03:09:45.004186+0000","last_clean":"2024-05-04T03:09:45.004186+0000","last_became_active":"2024-05-04T03:07:37.055957+0000","last_became_peered":"2024-05-04T03:07:37.055957+0000","last_unstale":"2024-05-04T03:09:45.004186+0000","last_undegraded":"2024-05-04T03:09:45.004186+0000","last_fullsized":"2024-05-04T03:09:45.004186+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:59:41.739034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247922+0000","last_change":"2024-05-04T03:07:34.020722+0000","last_active":"2024-05-04T03:08:16.247922+0000","last_peered":"2024-05-04T03:08:16.247922+0000","last_clean":"2024-05-04T03:08:16.247922+0000","last_became_active":"2024-05-04T03:07:34.020028+0000","last_became_peered":"2024-05-04T03:07:34.020028+0000","last_unstale":"2024-05-04T03:08:16.247922+0000","last_undegraded":"2024-05-04T03:08:16.247922+0000","last_fullsized":"2024-05-04T03:08:16.247922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:23:20.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998984+0000","last_change":"2024-05-04T03:07:37.059680+0000","last_active":"2024-05-04T03:09:44.998984+0000","last_peered":"2024-05-04T03:09:44.998984+0000","last_clean":"2024-05-04T03:09:44.998984+0000","last_became_active":"2024-05-04T03:07:37.059294+0000","last_became_peered":"2024-05-04T03:07:37.059294+0000","last_unstale":"2024-05-04T03:09:44.998984+0000","last_undegraded":"2024-05-04T03:09:44.998984+0000","last_fullsized":"2024-05-04T03:09:44.998984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:50:06.251706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'6","reported_seq":419,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:10.470054+0000","last_change":"2024-05-04T03:07:34.019055+0000","last_active":"2024-05-04T03:11:10.470054+0000","last_peered":"2024-05-04T03:11:10.470054+0000","last_clean":"2024-05-04T03:11:10.470054+0000","last_became_active":"2024-05-04T03:07:34.018638+0000","last_became_peered":"2024-05-04T03:07:34.018638+0000","last_unstale":"2024-05-04T03:11:10.470054+0000","last_undegraded":"2024-05-04T03:11:10.470054+0000","last_fullsized":"2024-05-04T03:11:10.470054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:58:01.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":386,"num_read_kb":386,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017350+0000","last_change":"2024-05-04T03:07:37.063037+0000","last_active":"2024-05-04T03:09:45.017350+0000","last_peered":"2024-05-04T03:09:45.017350+0000","last_clean":"2024-05-04T03:09:45.017350+0000","last_became_active":"2024-05-04T03:07:37.062350+0000","last_became_peered":"2024-05-04T03:07:37.062350+0000","last_unstale":"2024-05-04T03:09:45.017350+0000","last_undegraded":"2024-05-04T03:09:45.017350+0000","last_fullsized":"2024-05-04T03:09:45.017350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:41:15.898241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.374402+0000","last_change":"2024-05-04T03:08:16.373765+0000","last_active":"2024-05-04T03:08:16.374402+0000","last_peered":"2024-05-04T03:08:16.374402+0000","last_clean":"2024-05-04T03:08:16.374402+0000","last_became_active":"2024-05-04T03:08:16.368588+0000","last_became_peered":"2024-05-04T03:08:16.368588+0000","last_unstale":"2024-05-04T03:08:16.374402+0000","last_undegraded":"2024-05-04T03:08:16.374402+0000","last_fullsized":"2024-05-04T03:08:16.374402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:55:07.584881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007709+0000","last_change":"2024-05-04T03:07:37.071330+0000","last_active":"2024-05-04T03:09:45.007709+0000","last_peered":"2024-05-04T03:09:45.007709+0000","last_clean":"2024-05-04T03:09:45.007709+0000","last_became_active":"2024-05-04T03:07:37.070743+0000","last_became_peered":"2024-05-04T03:07:37.070743+0000","last_unstale":"2024-05-04T03:09:45.007709+0000","last_undegraded":"2024-05-04T03:09:45.007709+0000","last_fullsized":"2024-05-04T03:09:45.007709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:10:00.805690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248881+0000","last_change":"2024-05-04T03:07:34.157432+0000","last_active":"2024-05-04T03:08:16.248881+0000","last_peered":"2024-05-04T03:08:16.248881+0000","last_clean":"2024-05-04T03:08:16.248881+0000","last_became_active":"2024-05-04T03:07:34.156976+0000","last_became_peered":"2024-05-04T03:07:34.156976+0000","last_unstale":"2024-05-04T03:08:16.248881+0000","last_undegraded":"2024-05-04T03:08:16.248881+0000","last_fullsized":"2024-05-04T03:08:16.248881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:56.082287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017872+0000","last_change":"2024-05-04T03:07:37.064301+0000","last_active":"2024-05-04T03:09:45.017872+0000","last_peered":"2024-05-04T03:09:45.017872+0000","last_clean":"2024-05-04T03:09:45.017872+0000","last_became_active":"2024-05-04T03:07:37.063691+0000","last_became_peered":"2024-05-04T03:07:37.063691+0000","last_unstale":"2024-05-04T03:09:45.017872+0000","last_undegraded":"2024-05-04T03:09:45.017872+0000","last_fullsized":"2024-05-04T03:09:45.017872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:22:13.844703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'38","reported_seq":464,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:10.469748+0000","last_change":"2024-05-04T03:07:34.021595+0000","last_active":"2024-05-04T03:11:10.469748+0000","last_peered":"2024-05-04T03:11:10.469748+0000","last_clean":"2024-05-04T03:11:10.469748+0000","last_became_active":"2024-05-04T03:07:34.021126+0000","last_became_peered":"2024-05-04T03:07:34.021126+0000","last_unstale":"2024-05-04T03:11:10.469748+0000","last_undegraded":"2024-05-04T03:11:10.469748+0000","last_fullsized":"2024-05-04T03:11:10.469748+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:55.094800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65778,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":399,"num_read_kb":264,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999219+0000","last_change":"2024-05-04T03:07:37.066365+0000","last_active":"2024-05-04T03:09:44.999219+0000","last_peered":"2024-05-04T03:09:44.999219+0000","last_clean":"2024-05-04T03:09:44.999219+0000","last_became_active":"2024-05-04T03:07:37.065851+0000","last_became_peered":"2024-05-04T03:07:37.065851+0000","last_unstale":"2024-05-04T03:09:44.999219+0000","last_undegraded":"2024-05-04T03:09:44.999219+0000","last_fullsized":"2024-05-04T03:09:44.999219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:03:43.258235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.518904+0000","last_change":"2024-05-04T03:08:16.262600+0000","last_active":"2024-05-04T03:08:16.518904+0000","last_peered":"2024-05-04T03:08:16.518904+0000","last_clean":"2024-05-04T03:08:16.518904+0000","last_became_active":"2024-05-04T03:08:16.250619+0000","last_became_peered":"2024-05-04T03:08:16.250619+0000","last_unstale":"2024-05-04T03:08:16.518904+0000","last_undegraded":"2024-05-04T03:08:16.518904+0000","last_fullsized":"2024-05-04T03:08:16.518904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:51:21.596467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002712+0000","last_change":"2024-05-04T03:07:37.055356+0000","last_active":"2024-05-04T03:09:45.002712+0000","last_peered":"2024-05-04T03:09:45.002712+0000","last_clean":"2024-05-04T03:09:45.002712+0000","last_became_active":"2024-05-04T03:07:37.055045+0000","last_became_peered":"2024-05-04T03:07:37.055045+0000","last_unstale":"2024-05-04T03:09:45.002712+0000","last_undegraded":"2024-05-04T03:09:45.002712+0000","last_fullsized":"2024-05-04T03:09:45.002712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:49:06.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.247445+0000","last_change":"2024-05-04T03:07:34.021525+0000","last_active":"2024-05-04T03:08:16.247445+0000","last_peered":"2024-05-04T03:08:16.247445+0000","last_clean":"2024-05-04T03:08:16.247445+0000","last_became_active":"2024-05-04T03:07:34.021042+0000","last_became_peered":"2024-05-04T03:07:34.021042+0000","last_unstale":"2024-05-04T03:08:16.247445+0000","last_undegraded":"2024-05-04T03:08:16.247445+0000","last_fullsized":"2024-05-04T03:08:16.247445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:05:44.090176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.004521+0000","last_change":"2024-05-04T03:07:37.066471+0000","last_active":"2024-05-04T03:09:45.004521+0000","last_peered":"2024-05-04T03:09:45.004521+0000","last_clean":"2024-05-04T03:09:45.004521+0000","last_became_active":"2024-05-04T03:07:37.066059+0000","last_became_peered":"2024-05-04T03:07:37.066059+0000","last_unstale":"2024-05-04T03:09:45.004521+0000","last_undegraded":"2024-05-04T03:09:45.004521+0000","last_fullsized":"2024-05-04T03:09:45.004521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:16.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'8","reported_seq":234,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:11.471209+0000","last_change":"2024-05-04T03:07:34.021531+0000","last_active":"2024-05-04T03:11:11.471209+0000","last_peered":"2024-05-04T03:11:11.471209+0000","last_clean":"2024-05-04T03:11:11.471209+0000","last_became_active":"2024-05-04T03:07:34.021112+0000","last_became_peered":"2024-05-04T03:07:34.021112+0000","last_unstale":"2024-05-04T03:11:11.471209+0000","last_undegraded":"2024-05-04T03:11:11.471209+0000","last_fullsized":"2024-05-04T03:11:11.471209+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:28.826426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1597,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":201,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996662+0000","last_change":"2024-05-04T03:07:37.067155+0000","last_active":"2024-05-04T03:09:44.996662+0000","last_peered":"2024-05-04T03:09:44.996662+0000","last_clean":"2024-05-04T03:09:44.996662+0000","last_became_active":"2024-05-04T03:07:37.066783+0000","last_became_peered":"2024-05-04T03:07:37.066783+0000","last_unstale":"2024-05-04T03:09:44.996662+0000","last_undegraded":"2024-05-04T03:09:44.996662+0000","last_fullsized":"2024-05-04T03:09:44.996662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:01:23.217930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245106+0000","last_change":"2024-05-04T03:07:34.158227+0000","last_active":"2024-05-04T03:08:16.245106+0000","last_peered":"2024-05-04T03:08:16.245106+0000","last_clean":"2024-05-04T03:08:16.245106+0000","last_became_active":"2024-05-04T03:07:34.156905+0000","last_became_peered":"2024-05-04T03:07:34.156905+0000","last_unstale":"2024-05-04T03:08:16.245106+0000","last_undegraded":"2024-05-04T03:08:16.245106+0000","last_fullsized":"2024-05-04T03:08:16.245106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:40:41.865944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.006787+0000","last_change":"2024-05-04T03:07:37.075559+0000","last_active":"2024-05-04T03:09:45.006787+0000","last_peered":"2024-05-04T03:09:45.006787+0000","last_clean":"2024-05-04T03:09:45.006787+0000","last_became_active":"2024-05-04T03:07:37.075220+0000","last_became_peered":"2024-05-04T03:07:37.075220+0000","last_unstale":"2024-05-04T03:09:45.006787+0000","last_undegraded":"2024-05-04T03:09:45.006787+0000","last_fullsized":"2024-05-04T03:09:45.006787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:51:23.942514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:52.084062+0000","last_change":"2024-05-04T03:07:34.020326+0000","last_active":"2024-05-04T03:09:52.084062+0000","last_peered":"2024-05-04T03:09:52.084062+0000","last_clean":"2024-05-04T03:09:52.084062+0000","last_became_active":"2024-05-04T03:07:34.019472+0000","last_became_peered":"2024-05-04T03:07:34.019472+0000","last_unstale":"2024-05-04T03:09:52.084062+0000","last_undegraded":"2024-05-04T03:09:52.084062+0000","last_fullsized":"2024-05-04T03:09:52.084062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:33:28.375901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002619+0000","last_change":"2024-05-04T03:07:37.066223+0000","last_active":"2024-05-04T03:09:45.002619+0000","last_peered":"2024-05-04T03:09:45.002619+0000","last_clean":"2024-05-04T03:09:45.002619+0000","last_became_active":"2024-05-04T03:07:37.065859+0000","last_became_peered":"2024-05-04T03:07:37.065859+0000","last_unstale":"2024-05-04T03:09:45.002619+0000","last_undegraded":"2024-05-04T03:09:45.002619+0000","last_fullsized":"2024-05-04T03:09:45.002619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:44.902986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245539+0000","last_change":"2024-05-04T03:07:34.158674+0000","last_active":"2024-05-04T03:08:16.245539+0000","last_peered":"2024-05-04T03:08:16.245539+0000","last_clean":"2024-05-04T03:08:16.245539+0000","last_became_active":"2024-05-04T03:07:34.157952+0000","last_became_peered":"2024-05-04T03:07:34.157952+0000","last_unstale":"2024-05-04T03:08:16.245539+0000","last_undegraded":"2024-05-04T03:08:16.245539+0000","last_fullsized":"2024-05-04T03:08:16.245539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:19:11.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005664+0000","last_change":"2024-05-04T03:07:37.053897+0000","last_active":"2024-05-04T03:09:45.005664+0000","last_peered":"2024-05-04T03:09:45.005664+0000","last_clean":"2024-05-04T03:09:45.005664+0000","last_became_active":"2024-05-04T03:07:37.053466+0000","last_became_peered":"2024-05-04T03:07:37.053466+0000","last_unstale":"2024-05-04T03:09:45.005664+0000","last_undegraded":"2024-05-04T03:09:45.005664+0000","last_fullsized":"2024-05-04T03:09:45.005664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:50.650805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:10.487112+0000","last_change":"2024-05-04T03:07:34.021494+0000","last_active":"2024-05-04T03:11:10.487112+0000","last_peered":"2024-05-04T03:11:10.487112+0000","last_clean":"2024-05-04T03:11:10.487112+0000","last_became_active":"2024-05-04T03:07:34.021113+0000","last_became_peered":"2024-05-04T03:07:34.021113+0000","last_unstale":"2024-05-04T03:11:10.487112+0000","last_undegraded":"2024-05-04T03:11:10.487112+0000","last_fullsized":"2024-05-04T03:11:10.487112+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:02:09.514961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.007408+0000","last_change":"2024-05-04T03:07:37.068199+0000","last_active":"2024-05-04T03:09:45.007408+0000","last_peered":"2024-05-04T03:09:45.007408+0000","last_clean":"2024-05-04T03:09:45.007408+0000","last_became_active":"2024-05-04T03:07:37.067828+0000","last_became_peered":"2024-05-04T03:07:37.067828+0000","last_unstale":"2024-05-04T03:09:45.007408+0000","last_undegraded":"2024-05-04T03:09:45.007408+0000","last_fullsized":"2024-05-04T03:09:45.007408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:28.882920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996821+0000","last_change":"2024-05-04T03:07:37.069753+0000","last_active":"2024-05-04T03:09:44.996821+0000","last_peered":"2024-05-04T03:09:44.996821+0000","last_clean":"2024-05-04T03:09:44.996821+0000","last_became_active":"2024-05-04T03:07:37.069338+0000","last_became_peered":"2024-05-04T03:07:37.069338+0000","last_unstale":"2024-05-04T03:09:44.996821+0000","last_undegraded":"2024-05-04T03:09:44.996821+0000","last_fullsized":"2024-05-04T03:09:44.996821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:37:11.355749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'7","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:10.469230+0000","last_change":"2024-05-04T03:07:34.157229+0000","last_active":"2024-05-04T03:11:10.469230+0000","last_peered":"2024-05-04T03:11:10.469230+0000","last_clean":"2024-05-04T03:11:10.469230+0000","last_became_active":"2024-05-04T03:07:34.156776+0000","last_became_peered":"2024-05-04T03:07:34.156776+0000","last_unstale":"2024-05-04T03:11:10.469230+0000","last_undegraded":"2024-05-04T03:11:10.469230+0000","last_fullsized":"2024-05-04T03:11:10.469230+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:30:26.879754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.010488+0000","last_change":"2024-05-04T03:07:37.067025+0000","last_active":"2024-05-04T03:09:45.010488+0000","last_peered":"2024-05-04T03:09:45.010488+0000","last_clean":"2024-05-04T03:09:45.010488+0000","last_became_active":"2024-05-04T03:07:37.066359+0000","last_became_peered":"2024-05-04T03:07:37.066359+0000","last_unstale":"2024-05-04T03:09:45.010488+0000","last_undegraded":"2024-05-04T03:09:45.010488+0000","last_fullsized":"2024-05-04T03:09:45.010488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:44:35.604859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367184+0000","last_change":"2024-05-04T03:07:34.018019+0000","last_active":"2024-05-04T03:08:16.367184+0000","last_peered":"2024-05-04T03:08:16.367184+0000","last_clean":"2024-05-04T03:08:16.367184+0000","last_became_active":"2024-05-04T03:07:34.017556+0000","last_became_peered":"2024-05-04T03:07:34.017556+0000","last_unstale":"2024-05-04T03:08:16.367184+0000","last_undegraded":"2024-05-04T03:08:16.367184+0000","last_fullsized":"2024-05-04T03:08:16.367184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:31:34.158164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:52.260319+0000","last_change":"2024-05-04T03:06:06.704755+0000","last_active":"2024-05-04T03:08:52.260319+0000","last_peered":"2024-05-04T03:08:52.260319+0000","last_clean":"2024-05-04T03:08:52.260319+0000","last_became_active":"2024-05-04T03:06:06.698747+0000","last_became_peered":"2024-05-04T03:06:06.698747+0000","last_unstale":"2024-05-04T03:08:52.260319+0000","last_undegraded":"2024-05-04T03:08:52.260319+0000","last_fullsized":"2024-05-04T03:08:52.260319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:02:43.385377+0000","last_clean_scrub_stamp":"2024-05-04T03:02:43.385377+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:04:50.012414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008902+0000","last_change":"2024-05-04T03:07:37.057043+0000","last_active":"2024-05-04T03:09:45.008902+0000","last_peered":"2024-05-04T03:09:45.008902+0000","last_clean":"2024-05-04T03:09:45.008902+0000","last_became_active":"2024-05-04T03:07:37.056725+0000","last_became_peered":"2024-05-04T03:07:37.056725+0000","last_unstale":"2024-05-04T03:09:45.008902+0000","last_undegraded":"2024-05-04T03:09:45.008902+0000","last_fullsized":"2024-05-04T03:09:45.008902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:13:28.647089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.377000+0000","last_change":"2024-05-04T03:08:16.376510+0000","last_active":"2024-05-04T03:08:16.377000+0000","last_peered":"2024-05-04T03:08:16.377000+0000","last_clean":"2024-05-04T03:08:16.377000+0000","last_became_active":"2024-05-04T03:08:16.368625+0000","last_became_peered":"2024-05-04T03:08:16.368625+0000","last_unstale":"2024-05-04T03:08:16.377000+0000","last_undegraded":"2024-05-04T03:08:16.377000+0000","last_fullsized":"2024-05-04T03:08:16.377000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:27:50.504487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245652+0000","last_change":"2024-05-04T03:07:34.157346+0000","last_active":"2024-05-04T03:08:16.245652+0000","last_peered":"2024-05-04T03:08:16.245652+0000","last_clean":"2024-05-04T03:08:16.245652+0000","last_became_active":"2024-05-04T03:07:34.156914+0000","last_became_peered":"2024-05-04T03:07:34.156914+0000","last_unstale":"2024-05-04T03:08:16.245652+0000","last_undegraded":"2024-05-04T03:08:16.245652+0000","last_fullsized":"2024-05-04T03:08:16.245652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:54:17.059644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.987131+0000","last_change":"2024-05-04T03:07:37.054373+0000","last_active":"2024-05-04T03:09:44.987131+0000","last_peered":"2024-05-04T03:09:44.987131+0000","last_clean":"2024-05-04T03:09:44.987131+0000","last_became_active":"2024-05-04T03:07:37.053764+0000","last_became_peered":"2024-05-04T03:07:37.053764+0000","last_unstale":"2024-05-04T03:09:44.987131+0000","last_undegraded":"2024-05-04T03:09:44.987131+0000","last_fullsized":"2024-05-04T03:09:44.987131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:13:47.020618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'18","reported_seq":44,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:04.104348+0000","last_change":"2024-05-04T03:07:34.020190+0000","last_active":"2024-05-04T03:11:04.104348+0000","last_peered":"2024-05-04T03:11:04.104348+0000","last_clean":"2024-05-04T03:11:04.104348+0000","last_became_active":"2024-05-04T03:07:34.019786+0000","last_became_peered":"2024-05-04T03:07:34.019786+0000","last_unstale":"2024-05-04T03:11:04.104348+0000","last_undegraded":"2024-05-04T03:11:04.104348+0000","last_fullsized":"2024-05-04T03:11:04.104348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:11.243414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.999034+0000","last_change":"2024-05-04T03:07:37.055274+0000","last_active":"2024-05-04T03:09:44.999034+0000","last_peered":"2024-05-04T03:09:44.999034+0000","last_clean":"2024-05-04T03:09:44.999034+0000","last_became_active":"2024-05-04T03:07:37.054968+0000","last_became_peered":"2024-05-04T03:07:37.054968+0000","last_unstale":"2024-05-04T03:09:44.999034+0000","last_undegraded":"2024-05-04T03:09:44.999034+0000","last_fullsized":"2024-05-04T03:09:44.999034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.229266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.367534+0000","last_change":"2024-05-04T03:07:34.132966+0000","last_active":"2024-05-04T03:08:16.367534+0000","last_peered":"2024-05-04T03:08:16.367534+0000","last_clean":"2024-05-04T03:08:16.367534+0000","last_became_active":"2024-05-04T03:07:34.132529+0000","last_became_peered":"2024-05-04T03:07:34.132529+0000","last_unstale":"2024-05-04T03:08:16.367534+0000","last_undegraded":"2024-05-04T03:08:16.367534+0000","last_fullsized":"2024-05-04T03:08:16.367534+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:31:37.122097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008076+0000","last_change":"2024-05-04T03:08:16.368747+0000","last_active":"2024-05-04T03:09:45.008076+0000","last_peered":"2024-05-04T03:09:45.008076+0000","last_clean":"2024-05-04T03:09:45.008076+0000","last_became_active":"2024-05-04T03:08:16.368376+0000","last_became_peered":"2024-05-04T03:08:16.368376+0000","last_unstale":"2024-05-04T03:09:45.008076+0000","last_undegraded":"2024-05-04T03:09:45.008076+0000","last_fullsized":"2024-05-04T03:09:45.008076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:42:32.085650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.242516+0000","last_change":"2024-05-04T03:07:34.157219+0000","last_active":"2024-05-04T03:08:16.242516+0000","last_peered":"2024-05-04T03:08:16.242516+0000","last_clean":"2024-05-04T03:08:16.242516+0000","last_became_active":"2024-05-04T03:07:34.156866+0000","last_became_peered":"2024-05-04T03:07:34.156866+0000","last_unstale":"2024-05-04T03:08:16.242516+0000","last_undegraded":"2024-05-04T03:08:16.242516+0000","last_fullsized":"2024-05-04T03:08:16.242516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:15:34.138825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.990376+0000","last_change":"2024-05-04T03:07:37.058733+0000","last_active":"2024-05-04T03:09:44.990376+0000","last_peered":"2024-05-04T03:09:44.990376+0000","last_clean":"2024-05-04T03:09:44.990376+0000","last_became_active":"2024-05-04T03:07:37.058396+0000","last_became_peered":"2024-05-04T03:07:37.058396+0000","last_unstale":"2024-05-04T03:09:44.990376+0000","last_undegraded":"2024-05-04T03:09:44.990376+0000","last_fullsized":"2024-05-04T03:09:44.990376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:35.563728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.243578+0000","last_change":"2024-05-04T03:07:34.016458+0000","last_active":"2024-05-04T03:08:16.243578+0000","last_peered":"2024-05-04T03:08:16.243578+0000","last_clean":"2024-05-04T03:08:16.243578+0000","last_became_active":"2024-05-04T03:07:34.016038+0000","last_became_peered":"2024-05-04T03:07:34.016038+0000","last_unstale":"2024-05-04T03:08:16.243578+0000","last_undegraded":"2024-05-04T03:08:16.243578+0000","last_fullsized":"2024-05-04T03:08:16.243578+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:39:25.156047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998594+0000","last_change":"2024-05-04T03:07:37.064024+0000","last_active":"2024-05-04T03:09:44.998594+0000","last_peered":"2024-05-04T03:09:44.998594+0000","last_clean":"2024-05-04T03:09:44.998594+0000","last_became_active":"2024-05-04T03:07:37.063642+0000","last_became_peered":"2024-05-04T03:07:37.063642+0000","last_unstale":"2024-05-04T03:09:44.998594+0000","last_undegraded":"2024-05-04T03:09:44.998594+0000","last_fullsized":"2024-05-04T03:09:44.998594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:59:58.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'5","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:04.156994+0000","last_change":"2024-05-04T03:08:16.266763+0000","last_active":"2024-05-04T03:11:04.156994+0000","last_peered":"2024-05-04T03:11:04.156994+0000","last_clean":"2024-05-04T03:11:04.156994+0000","last_became_active":"2024-05-04T03:08:16.251212+0000","last_became_peered":"2024-05-04T03:08:16.251212+0000","last_unstale":"2024-05-04T03:11:04.156994+0000","last_undegraded":"2024-05-04T03:11:04.156994+0000","last_fullsized":"2024-05-04T03:11:04.156994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:31:27.444610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.998226+0000","last_change":"2024-05-04T03:07:37.063753+0000","last_active":"2024-05-04T03:09:44.998226+0000","last_peered":"2024-05-04T03:09:44.998226+0000","last_clean":"2024-05-04T03:09:44.998226+0000","last_became_active":"2024-05-04T03:07:37.063318+0000","last_became_peered":"2024-05-04T03:07:37.063318+0000","last_unstale":"2024-05-04T03:09:44.998226+0000","last_undegraded":"2024-05-04T03:09:44.998226+0000","last_fullsized":"2024-05-04T03:09:44.998226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:48.217136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.245182+0000","last_change":"2024-05-04T03:07:34.022522+0000","last_active":"2024-05-04T03:08:16.245182+0000","last_peered":"2024-05-04T03:08:16.245182+0000","last_clean":"2024-05-04T03:08:16.245182+0000","last_became_active":"2024-05-04T03:07:34.021018+0000","last_became_peered":"2024-05-04T03:07:34.021018+0000","last_unstale":"2024-05-04T03:08:16.245182+0000","last_undegraded":"2024-05-04T03:08:16.245182+0000","last_fullsized":"2024-05-04T03:08:16.245182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:47:04.277178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.014755+0000","last_change":"2024-05-04T03:07:37.069930+0000","last_active":"2024-05-04T03:09:45.014755+0000","last_peered":"2024-05-04T03:09:45.014755+0000","last_clean":"2024-05-04T03:09:45.014755+0000","last_became_active":"2024-05-04T03:07:37.069580+0000","last_became_peered":"2024-05-04T03:07:37.069580+0000","last_unstale":"2024-05-04T03:09:45.014755+0000","last_undegraded":"2024-05-04T03:09:45.014755+0000","last_fullsized":"2024-05-04T03:09:45.014755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:15:33.815970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'7","reported_seq":231,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:11:10.548481+0000","last_change":"2024-05-04T03:07:34.157969+0000","last_active":"2024-05-04T03:11:10.548481+0000","last_peered":"2024-05-04T03:11:10.548481+0000","last_clean":"2024-05-04T03:11:10.548481+0000","last_became_active":"2024-05-04T03:07:34.157563+0000","last_became_peered":"2024-05-04T03:07:34.157563+0000","last_unstale":"2024-05-04T03:11:10.548481+0000","last_undegraded":"2024-05-04T03:11:10.548481+0000","last_fullsized":"2024-05-04T03:11:10.548481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:00:43.562109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.015438+0000","last_change":"2024-05-04T03:07:37.071279+0000","last_active":"2024-05-04T03:09:45.015438+0000","last_peered":"2024-05-04T03:09:45.015438+0000","last_clean":"2024-05-04T03:09:45.015438+0000","last_became_active":"2024-05-04T03:07:37.070449+0000","last_became_peered":"2024-05-04T03:07:37.070449+0000","last_unstale":"2024-05-04T03:09:45.015438+0000","last_undegraded":"2024-05-04T03:09:45.015438+0000","last_fullsized":"2024-05-04T03:09:45.015438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:13:22.535788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248601+0000","last_change":"2024-05-04T03:07:34.017010+0000","last_active":"2024-05-04T03:08:16.248601+0000","last_peered":"2024-05-04T03:08:16.248601+0000","last_clean":"2024-05-04T03:08:16.248601+0000","last_became_active":"2024-05-04T03:07:34.016521+0000","last_became_peered":"2024-05-04T03:07:34.016521+0000","last_unstale":"2024-05-04T03:08:16.248601+0000","last_undegraded":"2024-05-04T03:08:16.248601+0000","last_fullsized":"2024-05-04T03:08:16.248601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:20:36.684656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.000434+0000","last_change":"2024-05-04T03:07:37.066526+0000","last_active":"2024-05-04T03:09:45.000434+0000","last_peered":"2024-05-04T03:09:45.000434+0000","last_clean":"2024-05-04T03:09:45.000434+0000","last_became_active":"2024-05-04T03:07:37.065312+0000","last_became_peered":"2024-05-04T03:07:37.065312+0000","last_unstale":"2024-05-04T03:09:45.000434+0000","last_undegraded":"2024-05-04T03:09:45.000434+0000","last_fullsized":"2024-05-04T03:09:45.000434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:54:10.077295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:08:16.248987+0000","last_change":"2024-05-04T03:07:34.016898+0000","last_active":"2024-05-04T03:08:16.248987+0000","last_peered":"2024-05-04T03:08:16.248987+0000","last_clean":"2024-05-04T03:08:16.248987+0000","last_became_active":"2024-05-04T03:07:34.016524+0000","last_became_peered":"2024-05-04T03:07:34.016524+0000","last_unstale":"2024-05-04T03:08:16.248987+0000","last_undegraded":"2024-05-04T03:08:16.248987+0000","last_fullsized":"2024-05-04T03:08:16.248987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:32.988174+0000","last_clean_scrub_stamp":"2024-05-04T03:07:32.988174+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:01.236207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.001328+0000","last_change":"2024-05-04T03:07:37.061501+0000","last_active":"2024-05-04T03:09:45.001328+0000","last_peered":"2024-05-04T03:09:45.001328+0000","last_clean":"2024-05-04T03:09:45.001328+0000","last_became_active":"2024-05-04T03:07:37.061077+0000","last_became_peered":"2024-05-04T03:07:37.061077+0000","last_unstale":"2024-05-04T03:09:45.001328+0000","last_undegraded":"2024-05-04T03:09:45.001328+0000","last_fullsized":"2024-05-04T03:09:45.001328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:32.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.996375+0000","last_change":"2024-05-04T03:07:37.064612+0000","last_active":"2024-05-04T03:09:44.996375+0000","last_peered":"2024-05-04T03:09:44.996375+0000","last_clean":"2024-05-04T03:09:44.996375+0000","last_became_active":"2024-05-04T03:07:37.064303+0000","last_became_peered":"2024-05-04T03:07:37.064303+0000","last_unstale":"2024-05-04T03:09:44.996375+0000","last_undegraded":"2024-05-04T03:09:44.996375+0000","last_fullsized":"2024-05-04T03:09:44.996375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:47:07.811765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005934+0000","last_change":"2024-05-04T03:07:37.061577+0000","last_active":"2024-05-04T03:09:45.005934+0000","last_peered":"2024-05-04T03:09:45.005934+0000","last_clean":"2024-05-04T03:09:45.005934+0000","last_became_active":"2024-05-04T03:07:37.061214+0000","last_became_peered":"2024-05-04T03:07:37.061214+0000","last_unstale":"2024-05-04T03:09:45.005934+0000","last_undegraded":"2024-05-04T03:09:45.005934+0000","last_fullsized":"2024-05-04T03:09:45.005934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:52:11.733257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.005073+0000","last_change":"2024-05-04T03:07:37.062906+0000","last_active":"2024-05-04T03:09:45.005073+0000","last_peered":"2024-05-04T03:09:45.005073+0000","last_clean":"2024-05-04T03:09:45.005073+0000","last_became_active":"2024-05-04T03:07:37.062510+0000","last_became_peered":"2024-05-04T03:07:37.062510+0000","last_unstale":"2024-05-04T03:09:45.005073+0000","last_undegraded":"2024-05-04T03:09:45.005073+0000","last_fullsized":"2024-05-04T03:09:45.005073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:57:58.910490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.008959+0000","last_change":"2024-05-04T03:07:37.068367+0000","last_active":"2024-05-04T03:09:45.008959+0000","last_peered":"2024-05-04T03:09:45.008959+0000","last_clean":"2024-05-04T03:09:45.008959+0000","last_became_active":"2024-05-04T03:07:37.068038+0000","last_became_peered":"2024-05-04T03:07:37.068038+0000","last_unstale":"2024-05-04T03:09:45.008959+0000","last_undegraded":"2024-05-04T03:09:45.008959+0000","last_fullsized":"2024-05-04T03:09:45.008959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:11:19.195002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003363+0000","last_change":"2024-05-04T03:08:16.250759+0000","last_active":"2024-05-04T03:09:45.003363+0000","last_peered":"2024-05-04T03:09:45.003363+0000","last_clean":"2024-05-04T03:09:45.003363+0000","last_became_active":"2024-05-04T03:08:16.250379+0000","last_became_peered":"2024-05-04T03:08:16.250379+0000","last_unstale":"2024-05-04T03:09:45.003363+0000","last_undegraded":"2024-05-04T03:09:45.003363+0000","last_fullsized":"2024-05-04T03:09:45.003363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:58:09.419009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.991925+0000","last_change":"2024-05-04T03:07:37.064584+0000","last_active":"2024-05-04T03:09:44.991925+0000","last_peered":"2024-05-04T03:09:44.991925+0000","last_clean":"2024-05-04T03:09:44.991925+0000","last_became_active":"2024-05-04T03:07:37.064183+0000","last_became_peered":"2024-05-04T03:07:37.064183+0000","last_unstale":"2024-05-04T03:09:44.991925+0000","last_undegraded":"2024-05-04T03:09:44.991925+0000","last_fullsized":"2024-05-04T03:09:44.991925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:27:17.121081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.013789+0000","last_change":"2024-05-04T03:07:37.066742+0000","last_active":"2024-05-04T03:09:45.013789+0000","last_peered":"2024-05-04T03:09:45.013789+0000","last_clean":"2024-05-04T03:09:45.013789+0000","last_became_active":"2024-05-04T03:07:37.066126+0000","last_became_peered":"2024-05-04T03:07:37.066126+0000","last_unstale":"2024-05-04T03:09:45.013789+0000","last_undegraded":"2024-05-04T03:09:45.013789+0000","last_fullsized":"2024-05-04T03:09:45.013789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:44:57.971837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.997576+0000","last_change":"2024-05-04T03:07:37.068122+0000","last_active":"2024-05-04T03:09:44.997576+0000","last_peered":"2024-05-04T03:09:44.997576+0000","last_clean":"2024-05-04T03:09:44.997576+0000","last_became_active":"2024-05-04T03:07:37.066321+0000","last_became_peered":"2024-05-04T03:07:37.066321+0000","last_unstale":"2024-05-04T03:09:44.997576+0000","last_undegraded":"2024-05-04T03:09:44.997576+0000","last_fullsized":"2024-05-04T03:09:44.997576+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:25:09.321853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.994963+0000","last_change":"2024-05-04T03:07:37.060322+0000","last_active":"2024-05-04T03:09:44.994963+0000","last_peered":"2024-05-04T03:09:44.994963+0000","last_clean":"2024-05-04T03:09:44.994963+0000","last_became_active":"2024-05-04T03:07:37.059944+0000","last_became_peered":"2024-05-04T03:07:37.059944+0000","last_unstale":"2024-05-04T03:09:44.994963+0000","last_undegraded":"2024-05-04T03:09:44.994963+0000","last_fullsized":"2024-05-04T03:09:44.994963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:43:30.105085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:44.993232+0000","last_change":"2024-05-04T03:07:37.056269+0000","last_active":"2024-05-04T03:09:44.993232+0000","last_peered":"2024-05-04T03:09:44.993232+0000","last_clean":"2024-05-04T03:09:44.993232+0000","last_became_active":"2024-05-04T03:07:37.055886+0000","last_became_peered":"2024-05-04T03:07:37.055886+0000","last_unstale":"2024-05-04T03:09:44.993232+0000","last_undegraded":"2024-05-04T03:09:44.993232+0000","last_fullsized":"2024-05-04T03:09:44.993232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:55:11.292577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.017574+0000","last_change":"2024-05-04T03:07:37.071013+0000","last_active":"2024-05-04T03:09:45.017574+0000","last_peered":"2024-05-04T03:09:45.017574+0000","last_clean":"2024-05-04T03:09:45.017574+0000","last_became_active":"2024-05-04T03:07:37.069346+0000","last_became_peered":"2024-05-04T03:07:37.069346+0000","last_unstale":"2024-05-04T03:09:45.017574+0000","last_undegraded":"2024-05-04T03:09:45.017574+0000","last_fullsized":"2024-05-04T03:09:45.017574+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:19:11.898766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.003394+0000","last_change":"2024-05-04T03:07:37.064962+0000","last_active":"2024-05-04T03:09:45.003394+0000","last_peered":"2024-05-04T03:09:45.003394+0000","last_clean":"2024-05-04T03:09:45.003394+0000","last_became_active":"2024-05-04T03:07:37.064637+0000","last_became_peered":"2024-05-04T03:07:37.064637+0000","last_unstale":"2024-05-04T03:09:45.003394+0000","last_undegraded":"2024-05-04T03:09:45.003394+0000","last_fullsized":"2024-05-04T03:09:45.003394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:10:10.659621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-04T03:09:45.002661+0000","last_change":"2024-05-04T03:07:37.056721+0000","last_active":"2024-05-04T03:09:45.002661+0000","last_peered":"2024-05-04T03:09:45.002661+0000","last_clean":"2024-05-04T03:09:45.002661+0000","last_became_active":"2024-05-04T03:07:37.056353+0000","last_became_peered":"2024-05-04T03:07:37.056353+0000","last_unstale":"2024-05-04T03:09:45.002661+0000","last_undegraded":"2024-05-04T03:09:45.002661+0000","last_fullsized":"2024-05-04T03:09:45.002661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:07:36.020446+0000","last_clean_scrub_stamp":"2024-05-04T03:07:36.020446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:30:47.476579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":286351,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75031,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1988,"num_read_kb":1077,"num_write":274,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":230133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710780,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":426700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1083580,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070535,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":425694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266763,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":426072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462992,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":1020162,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626518,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":425888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":495099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738484,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":427618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":496119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182500,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":425236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986272,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":880640,"data_stored":426281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822746,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":1018354,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34782,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36401,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:11:15.884 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:11:16.137 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:16.395 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:16 smithi039 ceph-mon[29715]: pgmap v400: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-04T03:11:16.395 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:16 smithi039 ceph-mon[29715]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:16.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:16 smithi167 ceph-mon[33614]: pgmap v400: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-04T03:11:16.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:16 smithi167 ceph-mon[33614]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:16 smithi066 ceph-mon[33264]: pgmap v400: 97 pgs: 97 active+clean; 650 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-04T03:11:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:16 smithi066 ceph-mon[33264]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:11:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 -- 172.21.15.39:0/390186241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77901034c0 msgr2=0x7f77901038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 --2- 172.21.15.39:0/390186241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77901034c0 0x7f77901038d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f77800099c0 tx=0x7f7780009cd0 comp rx=0 tx=0).stop 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 -- 172.21.15.39:0/390186241 shutdown_connections 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 --2- 172.21.15.39:0/390186241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77901034c0 0x7f77901038d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 --2- 172.21.15.39:0/390186241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7790108740 0x7f77901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 --2- 172.21.15.39:0/390186241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7790107df0 0x7f7790108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 -- 172.21.15.39:0/390186241 >> 172.21.15.39:0/390186241 conn(0x7f77900fedf0 msgr2=0x7f7790101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 -- 172.21.15.39:0/390186241 shutdown_connections 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 -- 172.21.15.39:0/390186241 wait complete. 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.878+0000 7f7795eab700 1 Processor -- start 2024-05-04T03:11:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f7795eab700 1 -- start start 2024-05-04T03:11:17.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f7795eab700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77901034c0 0x7f779019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:17.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f7795eab700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790107df0 0x7f779019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:17.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f7795eab700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7790108740 0x7f779019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:17.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f7795eab700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7790113220 con 0x7f7790107df0 2024-05-04T03:11:17.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f7795eab700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f77901130a0 con 0x7f7790108740 2024-05-04T03:11:17.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f7795eab700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77901133a0 con 0x7f77901034c0 2024-05-04T03:11:17.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790107df0 0x7f779019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790107df0 0x7f779019c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51488/0 (socket says 172.21.15.39:51488) 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778effd700 1 -- 172.21.15.39:0/3893907453 learned_addr learned my addr 172.21.15.39:0/3893907453 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778ffff700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7790108740 0x7f779019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778f7fe700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77901034c0 0x7f779019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778effd700 1 -- 172.21.15.39:0/3893907453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77901034c0 msgr2=0x7f779019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778effd700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77901034c0 0x7f779019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778effd700 1 -- 172.21.15.39:0/3893907453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7790108740 msgr2=0x7f779019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778effd700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7790108740 0x7f779019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.879+0000 7f778effd700 1 -- 172.21.15.39:0/3893907453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77901a6ef0 con 0x7f7790107df0 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.880+0000 7f778effd700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790107df0 0x7f779019c0e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f777800d980 tx=0x7f777800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.880+0000 7f778cff9700 1 -- 172.21.15.39:0/3893907453 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778017070 con 0x7f7790107df0 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.880+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77901999f0 con 0x7f7790107df0 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.880+0000 7f778cff9700 1 -- 172.21.15.39:0/3893907453 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f777800fd20 con 0x7f7790107df0 2024-05-04T03:11:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.880+0000 7f778cff9700 1 -- 172.21.15.39:0/3893907453 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777800e3b0 con 0x7f7790107df0 2024-05-04T03:11:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.880+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7790199f80 con 0x7f7790107df0 2024-05-04T03:11:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.881+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77900512c0 con 0x7f7790107df0 2024-05-04T03:11:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.882+0000 7f778cff9700 1 -- 172.21.15.39:0/3893907453 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f777800f890 con 0x7f7790107df0 2024-05-04T03:11:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.883+0000 7f778cff9700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f777c0b2990 0x7f777c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.883+0000 7f778f7fe700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f777c0b2990 0x7f777c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.883+0000 7f778cff9700 1 -- 172.21.15.39:0/3893907453 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f77780e2120 con 0x7f7790107df0 2024-05-04T03:11:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.884+0000 7f778f7fe700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f777c0b2990 0x7f777c0b4ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7780000c00 tx=0x7f7780005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:17.888+0000 7f778cff9700 1 -- 172.21.15.39:0/3893907453 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77780a9cf0 con 0x7f7790107df0 2024-05-04T03:11:18.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.156+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7790075fe0 con 0x7f7790107df0 2024-05-04T03:11:18.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.157+0000 7f778cff9700 1 -- 172.21.15.39:0/3893907453 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f77780a95f0 con 0x7f7790107df0 2024-05-04T03:11:18.158 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:18.159 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:11:05.993264+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:11:18.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f777c0b2990 msgr2=0x7f777c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:18.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f777c0b2990 0x7f777c0b4ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7780000c00 tx=0x7f7780005fb0 comp rx=0 tx=0).stop 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790107df0 msgr2=0x7f779019c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790107df0 0x7f779019c0e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f777800d980 tx=0x7f777800dc90 comp rx=0 tx=0).stop 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 shutdown_connections 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f777c0b2990 0x7f777c0b4ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7790108740 0x7f779019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77901034c0 0x7f779019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 --2- 172.21.15.39:0/3893907453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790107df0 0x7f779019c0e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.159+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 >> 172.21.15.39:0/3893907453 conn(0x7f77900fedf0 msgr2=0x7f7790101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.160+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 shutdown_connections 2024-05-04T03:11:18.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:18.160+0000 7f7795eab700 1 -- 172.21.15.39:0/3893907453 wait complete. 2024-05-04T03:11:18.162 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-05-04T03:11:18.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:18 smithi167 ceph-mon[33614]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-04T03:11:18.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:18 smithi039 ceph-mon[29715]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-04T03:11:18.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:18 smithi066 ceph-mon[33264]: pgmap v401: 97 pgs: 97 active+clean; 652 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-05-04T03:11:18.967 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:11:19.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:19 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3893907453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:19.221 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:19.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:19 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3893907453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:19.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:19 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3893907453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 -- 172.21.15.39:0/1413174976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac102b50 msgr2=0x7ff5ac103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/1413174976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac102b50 0x7ff5ac103850 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009ca0 tx=0x7ff59c02a8c0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 -- 172.21.15.39:0/1413174976 shutdown_connections 2024-05-04T03:11:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/1413174976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ac108a90 0x7ff5ac10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/1413174976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5ac103d90 0x7ff5ac1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/1413174976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac102b50 0x7ff5ac103850 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 -- 172.21.15.39:0/1413174976 >> 172.21.15.39:0/1413174976 conn(0x7ff5ac0fdd40 msgr2=0x7ff5ac1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 -- 172.21.15.39:0/1413174976 shutdown_connections 2024-05-04T03:11:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.157+0000 7ff5b0e22640 1 -- 172.21.15.39:0/1413174976 wait complete. 2024-05-04T03:11:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5b0e22640 1 Processor -- start 2024-05-04T03:11:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5b0e22640 1 -- start start 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5b0e22640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ac102b50 0x7ff5ac0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5b0e22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5ac103d90 0x7ff5ac0fe270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5b0e22640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac108a90 0x7ff5ac0fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5b0e22640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac10e550 con 0x7ff5ac108a90 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5b0e22640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac10e3d0 con 0x7ff5ac102b50 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5b0e22640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac10e6d0 con 0x7ff5ac103d90 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5aad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac108a90 0x7ff5ac0fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5aad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac108a90 0x7ff5ac0fe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51508/0 (socket says 172.21.15.39:51508) 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5aad9d640 1 -- 172.21.15.39:0/2888852627 learned_addr learned my addr 172.21.15.39:0/2888852627 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ac102b50 0x7ff5ac0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.158+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5ac103d90 0x7ff5ac0fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5aad9d640 1 -- 172.21.15.39:0/2888852627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5ac103d90 msgr2=0x7ff5ac0fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5aad9d640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5ac103d90 0x7ff5ac0fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5aad9d640 1 -- 172.21.15.39:0/2888852627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ac102b50 msgr2=0x7ff5ac0ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5aad9d640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ac102b50 0x7ff5ac0ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5aad9d640 1 -- 172.21.15.39:0/2888852627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac0feee0 con 0x7ff5ac108a90 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5ac103d90 0x7ff5ac0fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5aad9d640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac108a90 0x7ff5ac0fe7b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400d950 tx=0x7ff5a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff59b7fe640 1 -- 172.21.15.39:0/2888852627 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4017030 con 0x7ff5ac108a90 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff59b7fe640 1 -- 172.21.15.39:0/2888852627 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a4004020 con 0x7ff5ac108a90 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5b0e22640 1 -- 172.21.15.39:0/2888852627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac1a43c0 con 0x7ff5ac108a90 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff59b7fe640 1 -- 172.21.15.39:0/2888852627 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4003ab0 con 0x7ff5ac108a90 2024-05-04T03:11:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.159+0000 7ff5b0e22640 1 -- 172.21.15.39:0/2888852627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac1a48b0 con 0x7ff5ac108a90 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff5b7dac740 1 Processor -- start 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff59b7fe640 1 -- 172.21.15.39:0/2888852627 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7ff5a401a070 con 0x7ff5ac108a90 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff5b7dac740 1 -- start start 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b4b1b0 0x55eb98eeca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b222f0 0x55eb98eecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eb98e515d0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eb98e0a890 con 0x55eb98b4b1b0 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.162+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb98e4d590 con 0x55eb98b222f0 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51524/0 (socket says 172.21.15.39:51524) 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/3356714894 learned_addr learned my addr 172.21.15.39:0/3356714894 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b222f0 0x55eb98eecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b4b1b0 0x55eb98eeca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff59b7fe640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5940b28f0 0x7ff5940b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5aad9d640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5940b28f0 0x7ff5940b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/3356714894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b222f0 msgr2=0x55eb98eecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b222f0 0x55eb98eecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/3356714894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b4b1b0 msgr2=0x55eb98eeca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b4b1b0 0x55eb98eeca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/3356714894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb98b63210 con 0x55eb98b2a4d0 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff59b7fe640 1 -- 172.21.15.39:0/2888852627 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff5a40e1aa0 con 0x7ff5ac108a90 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.163+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef3ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0014a50 tx=0x7ff5a000c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa21bf5772add17f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.164+0000 7ff598ff9640 1 -- 172.21.15.39:0/3356714894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00142d0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.164+0000 7ff5aad9d640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5940b28f0 0x7ff5940b4e00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40a8c70 tx=0x7ff5a4072860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.164+0000 7ff598ff9640 1 -- 172.21.15.39:0/3356714894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a000fbf0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.164+0000 7ff598ff9640 1 -- 172.21.15.39:0/3356714894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0012940 con 0x55eb98b2a4d0 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.164+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3356714894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 msgr2=0x55eb98ef3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.164+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef3ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0014a50 tx=0x7ff5a000c5a0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3356714894 shutdown_connections 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef3ee0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b222f0 0x55eb98eecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3356714894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b4b1b0 0x55eb98eeca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3356714894 >> 172.21.15.39:0/3356714894 conn(0x55eb98b3f300 msgr2=0x55eb98ee83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3356714894 shutdown_connections 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3356714894 wait complete. 2024-05-04T03:11:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 -- start start 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 Processor -- start 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98ef6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.165+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 0x55eb98efda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eb98ef48d0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eb98e0e370 con 0x55eb98b222f0 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb98ef4a50 con 0x55eb98b4b1b0 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5b7dac740 10 client.0 _ll_register_callbacks cb 0x55eb98b62a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51528/0 (socket says 172.21.15.39:51528) 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 -- 172.21.15.39:0/3572676157 learned_addr learned my addr 172.21.15.39:0/3572676157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98ef6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 0x55eb98efda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 -- 172.21.15.39:0/3572676157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 msgr2=0x55eb98efda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 0x55eb98efda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 -- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 msgr2=0x55eb98ef6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98ef6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 -- 172.21.15.39:0/3572676157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb98c2f620 con 0x55eb98b2a4d0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98ef6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef6ac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac102420 tx=0x7ff59c031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69ee2c5b7fbf9d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff591ffb640 1 -- 172.21.15.39:0/3572676157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c003970 con 0x55eb98b2a4d0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff591ffb640 1 -- 172.21.15.39:0/3572676157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff59c003b10 con 0x55eb98b2a4d0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.166+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3572676157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a000fbf0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff591ffb640 1 -- 172.21.15.39:0/3572676157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff59c003d50 con 0x55eb98b2a4d0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5b7dac740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3572676157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb98f00a10 con 0x55eb98b2a4d0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff591ffb640 1 -- 172.21.15.39:0/3572676157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a000fbf0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff591ffb640 1 -- 172.21.15.39:0/3572676157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff59c0046a0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5b7dac740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5b7dac740 10 client.15288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5b7dac740 20 client.15288 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb98b09d80 0x55eb98f040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5b7dac740 4 client.15288 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3572676157 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb98e5f1e0 con 0x55eb98b09d80 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.167+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb98b09d80 0x55eb98f040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.168+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb98b09d80 0x55eb98f040b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e22a00eadebbaf7 server_cookie=ddc5c1bc5084d in_seq=0 out_seq=0 2024-05-04T03:11:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.168+0000 7ff591ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff591ffb640 1 -- 172.21.15.39:0/3572676157 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb98e5f1e0 con 0x55eb98b09d80 2024-05-04T03:11:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff591ffb640 10 client.15288 handle_command_reply: tid=0 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 client.15288 shutdown 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3572676157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 msgr2=0x55eb98ef6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef6ac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac102420 tx=0x7ff59c031cb0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3572676157 shutdown_connections 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb98b09d80 0x55eb98f040b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 0x55eb98efda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b2a4d0 0x55eb98ef6ac0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff591ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/3572676157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98ef6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3572676157 >> 172.21.15.39:0/3572676157 conn(0x55eb98b3f300 msgr2=0x55eb98eeb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3572676157 shutdown_connections 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 1 -- 172.21.15.39:0/3572676157 wait complete. 2024-05-04T03:11:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.169+0000 7ff5b7dac740 20 client.15288 trim_cache size 0 max 16384 2024-05-04T03:11:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.184+0000 7ff5b7dac740 1 Processor -- start 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.184+0000 7ff5b7dac740 1 -- start start 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b222f0 0x55eb98ee7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b2a4d0 0x55eb98ee8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 0x55eb98efd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1ad4b0 con 0x55eb98b222f0 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1aeff0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1adb80 con 0x55eb98b4b1b0 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b222f0 0x55eb98ee7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b222f0 0x55eb98ee7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51536/0 (socket says 172.21.15.39:51536) 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/1377040073 learned_addr learned my addr 172.21.15.39:0/1377040073 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b2a4d0 0x55eb98ee8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 0x55eb98efd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/1377040073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 msgr2=0x55eb98efd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 0x55eb98efd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/1377040073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b2a4d0 msgr2=0x55eb98ee8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b2a4d0 0x55eb98ee8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/1377040073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb98ef55c0 con 0x55eb98b222f0 2024-05-04T03:11:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b222f0 0x55eb98ee7dd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0000c00 tx=0x7ff5a0016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6348e32d019730 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.185+0000 7ff593fff640 1 -- 172.21.15.39:0/1377040073 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a002b040 con 0x55eb98b222f0 2024-05-04T03:11:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff593fff640 1 -- 172.21.15.39:0/1377040073 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a000fbc0 con 0x55eb98b222f0 2024-05-04T03:11:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff593fff640 1 -- 172.21.15.39:0/1377040073 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0004680 con 0x55eb98b222f0 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1377040073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b222f0 msgr2=0x55eb98ee7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b222f0 0x55eb98ee7dd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0000c00 tx=0x7ff5a0016040 comp rx=0 tx=0).stop 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1377040073 shutdown_connections 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b4b1b0 0x55eb98efd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b2a4d0 0x55eb98ee8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1377040073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b222f0 0x55eb98ee7dd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1377040073 >> 172.21.15.39:0/1377040073 conn(0x55eb98b3f300 msgr2=0x55eb98ef0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1377040073 shutdown_connections 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.186+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1377040073 wait complete. 2024-05-04T03:11:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 1 -- start start 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 1 Processor -- start 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98efe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b2a4d0 0x55eb98efad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b4b1b0 0x55eb98efb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1ae920 con 0x55eb98b4b1b0 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1ae250 con 0x55eb98b222f0 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac1ab4e0 con 0x55eb98b2a4d0 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5b7dac740 10 client.0 _ll_register_callbacks cb 0x7ff59c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b4b1b0 0x55eb98efb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b4b1b0 0x55eb98efb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51546/0 (socket says 172.21.15.39:51546) 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/1552584579 learned_addr learned my addr 172.21.15.39:0/1552584579 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98efe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b2a4d0 0x55eb98efad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/1552584579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b2a4d0 msgr2=0x55eb98efad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b2a4d0 0x55eb98efad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/1552584579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 msgr2=0x55eb98efe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.187+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98efe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff5a9d9b640 1 -- 172.21.15.39:0/1552584579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb98f6a4d0 con 0x55eb98b4b1b0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff5a9d9b640 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b4b1b0 0x55eb98efb3a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000c350 tx=0x7ff5a0023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f50a29c70bfc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff591ffb640 1 -- 172.21.15.39:0/1552584579 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00140d0 con 0x55eb98b4b1b0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff591ffb640 1 -- 172.21.15.39:0/1552584579 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff5a0014270 con 0x55eb98b4b1b0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1552584579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a000fbc0 con 0x55eb98b4b1b0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff591ffb640 1 -- 172.21.15.39:0/1552584579 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5a00053a0 con 0x55eb98b4b1b0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff5b7dac740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1552584579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb98f6b740 con 0x55eb98b4b1b0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff591ffb640 1 -- 172.21.15.39:0/1552584579 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5a000fbc0 con 0x55eb98b4b1b0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff591ffb640 1 -- 172.21.15.39:0/1552584579 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff5a002c250 con 0x55eb98b4b1b0 2024-05-04T03:11:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.188+0000 7ff5b7dac740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-05-04T03:11:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.189+0000 7ff5b7dac740 10 client.15294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:11:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.189+0000 7ff5b7dac740 20 client.15294 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.189+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb98adf1c0 0x55eb98f6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.189+0000 7ff5b7dac740 4 client.15294 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:11:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.189+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1552584579 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb98e59050 con 0x55eb98adf1c0 2024-05-04T03:11:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.189+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb98adf1c0 0x55eb98f6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.189+0000 7ff591ffb640 10 client.15294 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.189+0000 7ff5aa59c640 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb98adf1c0 0x55eb98f6df70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7e03bd6e5871b9 server_cookie=d1f7e98b0e2f0b0b in_seq=0 out_seq=0 2024-05-04T03:11:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff591ffb640 1 -- 172.21.15.39:0/1552584579 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb98e59050 con 0x55eb98adf1c0 2024-05-04T03:11:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff591ffb640 10 client.15294 handle_command_reply: tid=0 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 client.15294 shutdown 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1552584579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b4b1b0 msgr2=0x55eb98efb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b4b1b0 0x55eb98efb3a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000c350 tx=0x7ff5a0023040 comp rx=0 tx=0).stop 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1552584579 shutdown_connections 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb98adf1c0 0x55eb98f6df70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb98b4b1b0 0x55eb98efb3a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb98b2a4d0 0x55eb98efad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff591ffb640 0 client.15294 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 --2- 172.21.15.39:0/1552584579 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb98b222f0 0x55eb98efe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1552584579 >> 172.21.15.39:0/1552584579 conn(0x55eb98b3f300 msgr2=0x55eb98ee7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1552584579 shutdown_connections 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 1 -- 172.21.15.39:0/1552584579 wait complete. 2024-05-04T03:11:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.203+0000 7ff5b7dac740 20 client.15294 trim_cache size 0 max 16384 2024-05-04T03:11:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:11:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:11:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:11:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:11:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 -- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5940b28f0 msgr2=0x7ff5940b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5940b28f0 0x7ff5940b4e00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40a8c70 tx=0x7ff5a4072860 comp rx=0 tx=0).stop 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 -- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac108a90 msgr2=0x7ff5ac0fe7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac108a90 0x7ff5ac0fe7b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400d950 tx=0x7ff5a400de10 comp rx=0 tx=0).stop 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 -- 172.21.15.39:0/2888852627 shutdown_connections 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5940b28f0 0x7ff5940b4e00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5ac108a90 0x7ff5ac0fe7b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5ac103d90 0x7ff5ac0fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 --2- 172.21.15.39:0/2888852627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5ac102b50 0x7ff5ac0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 -- 172.21.15.39:0/2888852627 >> 172.21.15.39:0/2888852627 conn(0x7ff5ac0fdd40 msgr2=0x7ff5ac109fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 -- 172.21.15.39:0/2888852627 shutdown_connections 2024-05-04T03:11:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:20.204+0000 7ff5b0e22640 1 -- 172.21.15.39:0/2888852627 wait complete. 2024-05-04T03:11:20.371 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:20 smithi039 ceph-mon[29715]: pgmap v402: 97 pgs: 97 active+clean; 652 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-04T03:11:20.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:20 smithi167 ceph-mon[33614]: pgmap v402: 97 pgs: 97 active+clean; 652 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-04T03:11:20.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:20 smithi066 ceph-mon[33264]: pgmap v402: 97 pgs: 97 active+clean; 652 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 -- 172.21.15.39:0/4210726029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 msgr2=0x7fda44102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 --2- 172.21.15.39:0/4210726029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 0x7fda44102f50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fda38009990 tx=0x7fda38009ca0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 -- 172.21.15.39:0/4210726029 shutdown_connections 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 --2- 172.21.15.39:0/4210726029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 0x7fda44102f50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 --2- 172.21.15.39:0/4210726029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda441098b0 0x7fda441111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 --2- 172.21.15.39:0/4210726029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda44108f60 0x7fda44109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 -- 172.21.15.39:0/4210726029 >> 172.21.15.39:0/4210726029 conn(0x7fda440fe4e0 msgr2=0x7fda44100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 -- 172.21.15.39:0/4210726029 shutdown_connections 2024-05-04T03:11:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.989+0000 7fda4adf7700 1 -- 172.21.15.39:0/4210726029 wait complete. 2024-05-04T03:11:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.990+0000 7fda4adf7700 1 Processor -- start 2024-05-04T03:11:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.990+0000 7fda4adf7700 1 -- start start 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda4adf7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 0x7fda4419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda4adf7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda44108f60 0x7fda4419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda4adf7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda441098b0 0x7fda4419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda4adf7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fda44114350 con 0x7fda44102b40 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda4adf7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda441141d0 con 0x7fda441098b0 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda4adf7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fda441144d0 con 0x7fda44108f60 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 0x7fda4419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 0x7fda4419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51564/0 (socket says 172.21.15.39:51564) 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 -- 172.21.15.39:0/1182709139 learned_addr learned my addr 172.21.15.39:0/1182709139 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda49394700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda441098b0 0x7fda4419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda43fff700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda44108f60 0x7fda4419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 -- 172.21.15.39:0/1182709139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda44108f60 msgr2=0x7fda4419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda44108f60 0x7fda4419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 -- 172.21.15.39:0/1182709139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda441098b0 msgr2=0x7fda4419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda441098b0 0x7fda4419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 -- 172.21.15.39:0/1182709139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4419fc10 con 0x7fda44102b40 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda43fff700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda44108f60 0x7fda4419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.991+0000 7fda48b93700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 0x7fda4419ad40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fda38000c00 tx=0x7fda380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.992+0000 7fda41ffb700 1 -- 172.21.15.39:0/1182709139 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38027070 con 0x7fda44102b40 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.992+0000 7fda41ffb700 1 -- 172.21.15.39:0/1182709139 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda3800fb80 con 0x7fda44102b40 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.992+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4419fea0 con 0x7fda44102b40 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.992+0000 7fda41ffb700 1 -- 172.21.15.39:0/1182709139 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38022d00 con 0x7fda44102b40 2024-05-04T03:11:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.992+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441a7820 con 0x7fda44102b40 2024-05-04T03:11:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.993+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda440512c0 con 0x7fda44102b40 2024-05-04T03:11:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.995+0000 7fda41ffb700 1 -- 172.21.15.39:0/1182709139 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fda3802a070 con 0x7fda44102b40 2024-05-04T03:11:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.995+0000 7fda41ffb700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fda340b2990 0x7fda340b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.995+0000 7fda43fff700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fda340b2990 0x7fda340b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.995+0000 7fda41ffb700 1 -- 172.21.15.39:0/1182709139 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fda380e3590 con 0x7fda44102b40 2024-05-04T03:11:20.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.996+0000 7fda43fff700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fda340b2990 0x7fda340b4ea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda30005950 tx=0x7fda300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:21.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:20.999+0000 7fda41ffb700 1 -- 172.21.15.39:0/1182709139 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda38031080 con 0x7fda44102b40 2024-05-04T03:11:21.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.271+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda4419c0a0 con 0x7fda44102b40 2024-05-04T03:11:21.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.271+0000 7fda41ffb700 1 -- 172.21.15.39:0/1182709139 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7fda38013070 con 0x7fda44102b40 2024-05-04T03:11:21.272 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:21.273 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:11:05.993264+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:11:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fda340b2990 msgr2=0x7fda340b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fda340b2990 0x7fda340b4ea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda30005950 tx=0x7fda300058e0 comp rx=0 tx=0).stop 2024-05-04T03:11:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 msgr2=0x7fda4419ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 0x7fda4419ad40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fda38000c00 tx=0x7fda380047b0 comp rx=0 tx=0).stop 2024-05-04T03:11:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 shutdown_connections 2024-05-04T03:11:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fda340b2990 0x7fda340b4ea0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda44102b40 0x7fda4419ad40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda441098b0 0x7fda4419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 --2- 172.21.15.39:0/1182709139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda44108f60 0x7fda4419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 >> 172.21.15.39:0/1182709139 conn(0x7fda440fe4e0 msgr2=0x7fda44101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 shutdown_connections 2024-05-04T03:11:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:21.274+0000 7fda4adf7700 1 -- 172.21.15.39:0/1182709139 wait complete. 2024-05-04T03:11:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-05-04T03:11:21.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:21 smithi066 ceph-mon[33264]: pgmap v403: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-04T03:11:21.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:21 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1182709139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:21.898 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:21 smithi039 ceph-mon[29715]: pgmap v403: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-04T03:11:21.898 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:21 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1182709139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:21.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:21 smithi167 ceph-mon[33614]: pgmap v403: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-04T03:11:21.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:21 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1182709139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:21.930 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:11:22.183 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 -- 172.21.15.39:0/3959395470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f0107df0 msgr2=0x7f16f0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 --2- 172.21.15.39:0/3959395470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f0107df0 0x7f16f0108200 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f16d80099c0 tx=0x7f16d8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 -- 172.21.15.39:0/3959395470 shutdown_connections 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 --2- 172.21.15.39:0/3959395470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f0108740 0x7f16f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 --2- 172.21.15.39:0/3959395470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16f01034c0 0x7f16f01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 --2- 172.21.15.39:0/3959395470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f0107df0 0x7f16f0108200 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 -- 172.21.15.39:0/3959395470 >> 172.21.15.39:0/3959395470 conn(0x7f16f00fedf0 msgr2=0x7f16f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 -- 172.21.15.39:0/3959395470 shutdown_connections 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.993+0000 7f16f5338700 1 -- 172.21.15.39:0/3959395470 wait complete. 2024-05-04T03:11:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16f5338700 1 Processor -- start 2024-05-04T03:11:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16f5338700 1 -- start start 2024-05-04T03:11:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16f5338700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f01034c0 0x7f16f019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16f5338700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16f0107df0 0x7f16f019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16f5338700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f0108740 0x7f16f019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16f5338700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0113200 con 0x7f16f01034c0 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16f5338700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0113080 con 0x7f16f0107df0 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16f5338700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0113380 con 0x7f16f0108740 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16eeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f01034c0 0x7f16f019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.994+0000 7f16eeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f01034c0 0x7f16f019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51572/0 (socket says 172.21.15.39:51572) 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16eeffd700 1 -- 172.21.15.39:0/1015384363 learned_addr learned my addr 172.21.15.39:0/1015384363 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16ee7fc700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16f0107df0 0x7f16f019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16ef7fe700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f0108740 0x7f16f019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16eeffd700 1 -- 172.21.15.39:0/1015384363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f0108740 msgr2=0x7f16f019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16eeffd700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f0108740 0x7f16f019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16eeffd700 1 -- 172.21.15.39:0/1015384363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16f0107df0 msgr2=0x7f16f019c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16eeffd700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16f0107df0 0x7f16f019c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16eeffd700 1 -- 172.21.15.39:0/1015384363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f01a6ea0 con 0x7f16f01034c0 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16ee7fc700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16f0107df0 0x7f16f019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16ef7fe700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f0108740 0x7f16f019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16eeffd700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f01034c0 0x7f16f019bbe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16e8007f10 tx=0x7f16e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16e7fff700 1 -- 172.21.15.39:0/1015384363 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8003a00 con 0x7f16f01034c0 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f01a7190 con 0x7f16f01034c0 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16e7fff700 1 -- 172.21.15.39:0/1015384363 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16e8003ba0 con 0x7f16f01034c0 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16e7fff700 1 -- 172.21.15.39:0/1015384363 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8015660 con 0x7f16f01034c0 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.995+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f01a76d0 con 0x7f16f01034c0 2024-05-04T03:11:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.996+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f00512c0 con 0x7f16f01034c0 2024-05-04T03:11:24.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:23.999+0000 7f16e7fff700 1 -- 172.21.15.39:0/1015384363 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f16e8004000 con 0x7f16f01034c0 2024-05-04T03:11:24.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.000+0000 7f16e7fff700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16dc0b2910 0x7f16dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:24.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.000+0000 7f16ee7fc700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16dc0b2910 0x7f16dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:24.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.000+0000 7f16e7fff700 1 -- 172.21.15.39:0/1015384363 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f16e80e2460 con 0x7f16f01034c0 2024-05-04T03:11:24.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.000+0000 7f16ee7fc700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16dc0b2910 0x7f16dc0b4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f16d8000c00 tx=0x7f16d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:24.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.003+0000 7f16e7fff700 1 -- 172.21.15.39:0/1015384363 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16e8010070 con 0x7f16f01034c0 2024-05-04T03:11:24.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.220+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16f002e5f0 con 0x7f16f01034c0 2024-05-04T03:11:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.221+0000 7f16e7fff700 1 -- 172.21.15.39:0/1015384363 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22541 (secure 0 0 0) 0x7f16e8004870 con 0x7f16f01034c0 2024-05-04T03:11:24.223 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:24.223 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:08:16.237923+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16dc0b2910 msgr2=0x7f16dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16dc0b2910 0x7f16dc0b4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f16d8000c00 tx=0x7f16d8005fb0 comp rx=0 tx=0).stop 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f01034c0 msgr2=0x7f16f019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f01034c0 0x7f16f019bbe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f16e8007f10 tx=0x7f16e800d3b0 comp rx=0 tx=0).stop 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 shutdown_connections 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16dc0b2910 0x7f16dc0b4e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f01034c0 0x7f16f019bbe0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16f0108740 0x7f16f019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 --2- 172.21.15.39:0/1015384363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16f0107df0 0x7f16f019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:24.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 >> 172.21.15.39:0/1015384363 conn(0x7f16f00fedf0 msgr2=0x7f16f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:24.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 shutdown_connections 2024-05-04T03:11:24.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:24.224+0000 7f16f5338700 1 -- 172.21.15.39:0/1015384363 wait complete. 2024-05-04T03:11:24.236 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:24 smithi039 ceph-mon[29715]: pgmap v404: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-04T03:11:24.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:24 smithi167 ceph-mon[33614]: pgmap v404: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-04T03:11:24.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:24 smithi066 ceph-mon[33264]: pgmap v404: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-04T03:11:24.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-04T03:11:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:25.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:25 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1015384363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:11:25.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:25 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1015384363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:11:25.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:25 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1015384363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:11:26.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:26 smithi039 ceph-mon[29715]: pgmap v405: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-04T03:11:26.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:26 smithi167 ceph-mon[33614]: pgmap v405: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-04T03:11:26.469 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:26 smithi066 ceph-mon[33264]: pgmap v405: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-04T03:11:27.631 INFO:teuthology.orchestra.run.smithi039.stdout:{"run_dir":"/var/run/ceph"} 2024-05-04T03:11:27.631 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-04T03:11:27.632 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1842 2024-05-04T03:11:27.632 DEBUG:tasks.quiescer:Yielding 2024-05-04T03:11:27.632 INFO:teuthology.run_tasks:Running task exec... 2024-05-04T03:11:27.642 INFO:teuthology.task.exec:Executing custom commands... 2024-05-04T03:11:27.642 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T03:11:27.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-04T03:11:27.643 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:11:27.679 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-05-04T03:11:27.690 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-04T03:11:27.690 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-fuse 340G 0 340G 0% /tmp/cephfs 2024-05-04T03:11:27.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-04T03:11:27.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-04T03:11:27.824 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-04T03:11:27.898 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:28.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:28 smithi039 ceph-mon[29715]: pgmap v406: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-05-04T03:11:28.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:28 smithi167 ceph-mon[33614]: pgmap v406: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-05-04T03:11:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:28 smithi066 ceph-mon[33264]: pgmap v406: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-05-04T03:11:29.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-04T03:11:29.717 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-04T03:11:29.719 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-04T03:11:29.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 .. 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 data 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:29.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:29.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 .. 2024-05-04T03:11:29.882 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-04T03:11:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:[Service] 2024-05-04T03:11:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-04T03:11:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-04T03:11:29.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-04T03:11:30.107 INFO:teuthology.orchestra.run.smithi039.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-04T03:11:30.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:30 smithi039 ceph-mon[29715]: pgmap v407: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-05-04T03:11:30.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:30 smithi167 ceph-mon[33614]: pgmap v407: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-05-04T03:11:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:30 smithi066 ceph-mon[33264]: pgmap v407: 97 pgs: 97 active+clean; 658 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-05-04T03:11:30.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.677+0000 7f09b8342700 1 -- 172.21.15.39:0/3211634392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 msgr2=0x7f09b0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:30.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.677+0000 7f09b8342700 1 --2- 172.21.15.39:0/3211634392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 0x7f09b0108200 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f09a00099c0 tx=0x7f09a0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:11:30.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.678+0000 7f09b8342700 1 -- 172.21.15.39:0/3211634392 shutdown_connections 2024-05-04T03:11:30.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.678+0000 7f09b8342700 1 --2- 172.21.15.39:0/3211634392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b0108740 0x7f09b01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:30.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.678+0000 7f09b8342700 1 --2- 172.21.15.39:0/3211634392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09b01034c0 0x7f09b01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:30.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.678+0000 7f09b8342700 1 --2- 172.21.15.39:0/3211634392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 0x7f09b0108200 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:30.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.678+0000 7f09b8342700 1 -- 172.21.15.39:0/3211634392 >> 172.21.15.39:0/3211634392 conn(0x7f09b00fedc0 msgr2=0x7f09b01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:30.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.678+0000 7f09b8342700 1 -- 172.21.15.39:0/3211634392 shutdown_connections 2024-05-04T03:11:30.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.678+0000 7f09b8342700 1 -- 172.21.15.39:0/3211634392 wait complete. 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b8342700 1 Processor -- start 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b8342700 1 -- start start 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b8342700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b01034c0 0x7f09b019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b8342700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 0x7f09b019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b8342700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09b0108740 0x7f09b019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b8342700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09b0113200 con 0x7f09b0107df0 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b8342700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f09b0113080 con 0x7f09b0108740 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b8342700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09b0113380 con 0x7f09b01034c0 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.679+0000 7f09b58dd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 0x7f09b019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:30.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b58dd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 0x7f09b019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56460/0 (socket says 172.21.15.39:56460) 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b58dd700 1 -- 172.21.15.39:0/2733733119 learned_addr learned my addr 172.21.15.39:0/2733733119 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b68df700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09b0108740 0x7f09b019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b60de700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b01034c0 0x7f09b019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b58dd700 1 -- 172.21.15.39:0/2733733119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b01034c0 msgr2=0x7f09b019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b58dd700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b01034c0 0x7f09b019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b58dd700 1 -- 172.21.15.39:0/2733733119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09b0108740 msgr2=0x7f09b019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b58dd700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09b0108740 0x7f09b019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b58dd700 1 -- 172.21.15.39:0/2733733119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b019fc30 con 0x7f09b0107df0 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09b58dd700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 0x7f09b019b300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f09a000b580 tx=0x7f09a000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09a77fe700 1 -- 172.21.15.39:0/2733733119 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0026070 con 0x7f09b0107df0 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09a77fe700 1 -- 172.21.15.39:0/2733733119 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09a0005030 con 0x7f09b0107df0 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.680+0000 7f09a77fe700 1 -- 172.21.15.39:0/2733733119 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00049f0 con 0x7f09b0107df0 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.681+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b019fec0 con 0x7f09b0107df0 2024-05-04T03:11:30.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.681+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b01a0350 con 0x7f09b0107df0 2024-05-04T03:11:30.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.682+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b00512c0 con 0x7f09b0107df0 2024-05-04T03:11:30.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.684+0000 7f09a77fe700 1 -- 172.21.15.39:0/2733733119 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f09a0029070 con 0x7f09b0107df0 2024-05-04T03:11:30.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.684+0000 7f09a77fe700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f099c0b2990 0x7f099c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:30.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.685+0000 7f09b60de700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f099c0b2990 0x7f099c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:30.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.685+0000 7f09a77fe700 1 -- 172.21.15.39:0/2733733119 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f09a00e3150 con 0x7f09b0107df0 2024-05-04T03:11:30.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.685+0000 7f09b60de700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f099c0b2990 0x7f099c0b4ea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005950 tx=0x7f09ac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:30.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:30.688+0000 7f09a77fe700 1 -- 172.21.15.39:0/2733733119 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09a0030080 con 0x7f09b0107df0 2024-05-04T03:11:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.006+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09b01a7900 con 0x7f09b0107df0 2024-05-04T03:11:31.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.007+0000 7f09a77fe700 1 -- 172.21.15.39:0/2733733119 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11930 (secure 0 0 0) 0x7f09a0013070 con 0x7f09b0107df0 2024-05-04T03:11:31.008 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:31.008 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:11:05.993264+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:11:31.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f099c0b2990 msgr2=0x7f099c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f099c0b2990 0x7f099c0b4ea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005950 tx=0x7f09ac0058e0 comp rx=0 tx=0).stop 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 msgr2=0x7f09b019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 0x7f09b019b300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f09a000b580 tx=0x7f09a000f510 comp rx=0 tx=0).stop 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 shutdown_connections 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f099c0b2990 0x7f099c0b4ea0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f09b0108740 0x7f09b019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09b01034c0 0x7f09b019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 --2- 172.21.15.39:0/2733733119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b0107df0 0x7f09b019b300 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 >> 172.21.15.39:0/2733733119 conn(0x7f09b00fedc0 msgr2=0x7f09b01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 shutdown_connections 2024-05-04T03:11:31.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:31.009+0000 7f09b8342700 1 -- 172.21.15.39:0/2733733119 wait complete. 2024-05-04T03:11:31.012 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-05-04T03:11:31.394 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:31 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2733733119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:31.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:31 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2733733119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:31.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:31 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2733733119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:11:31.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-04T03:07:39.087961+0000', 'modified': '2024-05-04T03:11:05.993264+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24506, 'mds_1': 24490, 'mds_2': 24472, 'mds_3': 24466, 'mds_4': 24455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14607': {'gid': 14607, 'name': 'h', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6837/2860496376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6836', 'nonce': 2860496376}, {'type': 'v1', 'addr': '172.21.15.66:6837', 'nonce': 2860496376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14622': {'gid': 14622, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6839/4175892611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6838', 'nonce': 4175892611}, {'type': 'v1', 'addr': '172.21.15.39:6839', 'nonce': 4175892611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14634': {'gid': 14634, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6841/1534369514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 1534369514}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 1534369514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/1077677237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 1077677237}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 1077677237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6833/462065299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 462065299}, {'type': 'v1', 'addr': '172.21.15.66:6833', 'nonce': 462065299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.66:6835/3463909305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6834', 'nonce': 3463909305}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 3463909305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.167:6835/2699315372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 2699315372}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 2699315372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.167:6837/2270685849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2270685849}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2270685849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.167:6839/975309166', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 975309166}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 975309166}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.66:6839/4050747247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 4050747247}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 4050747247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24506, 24472, 24466, 24455, 24490]} 2024-05-04T03:11:31.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-04T03:11:31.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.e status 2024-05-04T03:11:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:32.397 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:32 smithi039 ceph-mon[29715]: pgmap v408: 97 pgs: 97 active+clean; 8.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 695 KiB/s wr, 16 op/s 2024-05-04T03:11:32.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:32 smithi167 ceph-mon[33614]: pgmap v408: 97 pgs: 97 active+clean; 8.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 695 KiB/s wr, 16 op/s 2024-05-04T03:11:32.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:32 smithi066 ceph-mon[33264]: pgmap v408: 97 pgs: 97 active+clean; 8.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 695 KiB/s wr, 16 op/s 2024-05-04T03:11:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 -- 172.21.15.39:0/2085490763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 msgr2=0x7fa230103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 --2- 172.21.15.39:0/2085490763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 0x7fa230103840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa220009900 tx=0x7fa22002a410 comp rx=0 tx=0).stop 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 -- 172.21.15.39:0/2085490763 shutdown_connections 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 --2- 172.21.15.39:0/2085490763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa230108a80 0x7fa23010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 --2- 172.21.15.39:0/2085490763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230103d80 0x7fa230104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 --2- 172.21.15.39:0/2085490763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 0x7fa230103840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 -- 172.21.15.39:0/2085490763 >> 172.21.15.39:0/2085490763 conn(0x7fa2300fdd30 msgr2=0x7fa2301001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 -- 172.21.15.39:0/2085490763 shutdown_connections 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 -- 172.21.15.39:0/2085490763 wait complete. 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.315+0000 7fa2366e2640 1 Processor -- start 2024-05-04T03:11:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa2366e2640 1 -- start start 2024-05-04T03:11:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa2366e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 0x7fa23019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa2366e2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa230103d80 0x7fa23019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa2366e2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230108a80 0x7fa2301a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa2366e2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010e4f0 con 0x7fa230102b40 2024-05-04T03:11:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa2366e2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010e370 con 0x7fa230108a80 2024-05-04T03:11:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa2366e2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010e670 con 0x7fa230103d80 2024-05-04T03:11:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 0x7fa23019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 0x7fa23019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56476/0 (socket says 172.21.15.39:56476) 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa22ffff640 1 -- 172.21.15.39:0/3435377140 learned_addr learned my addr 172.21.15.39:0/3435377140 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa234c7f640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230108a80 0x7fa2301a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa22f7fe640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa230103d80 0x7fa23019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa22ffff640 1 -- 172.21.15.39:0/3435377140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa230103d80 msgr2=0x7fa23019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa230103d80 0x7fa23019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa22ffff640 1 -- 172.21.15.39:0/3435377140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230108a80 msgr2=0x7fa2301a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.316+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230108a80 0x7fa2301a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.317+0000 7fa22ffff640 1 -- 172.21.15.39:0/3435377140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2301a1e40 con 0x7fa230102b40 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.317+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 0x7fa23019a300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa220005760 tx=0x7fa220004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.317+0000 7fa22d7fa640 1 -- 172.21.15.39:0/3435377140 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22003f030 con 0x7fa230102b40 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.317+0000 7fa22d7fa640 1 -- 172.21.15.39:0/3435377140 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa220003a80 con 0x7fa230102b40 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.317+0000 7fa2366e2640 1 -- 172.21.15.39:0/3435377140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2301a20d0 con 0x7fa230102b40 2024-05-04T03:11:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.317+0000 7fa22d7fa640 1 -- 172.21.15.39:0/3435377140 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220035070 con 0x7fa230102b40 2024-05-04T03:11:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.317+0000 7fa2366e2640 1 -- 172.21.15.39:0/3435377140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301a25c0 con 0x7fa230102b40 2024-05-04T03:11:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.319+0000 7fa23d592740 1 Processor -- start 2024-05-04T03:11:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa23d592740 1 -- start start 2024-05-04T03:11:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22d7fa640 1 -- 172.21.15.39:0/3435377140 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fa220042030 con 0x7fa230102b40 2024-05-04T03:11:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa23d592740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a18857a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa23d592740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a18857fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa23d592740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184954d0 0x558a1885eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa23d592740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a187bc5d0 con 0x558a184954d0 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa23d592740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558a18775890 con 0x558a1848d2f0 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa23d592740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558a187b8590 con 0x558a184b61b0 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184954d0 0x558a1885eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a18857fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa234c7f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a18857a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184954d0 0x558a1885eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56486/0 (socket says 172.21.15.39:56486) 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22f7fe640 1 -- 172.21.15.39:0/356393117 learned_addr learned my addr 172.21.15.39:0/356393117 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa234c7f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a18857a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:44154/0 (socket says 172.21.15.39:44154) 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22f7fe640 1 -- 172.21.15.39:0/356393117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 msgr2=0x558a18857a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22f7fe640 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a18857a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22f7fe640 1 -- 172.21.15.39:0/356393117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 msgr2=0x558a18857fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22f7fe640 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a18857fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22f7fe640 1 -- 172.21.15.39:0/356393117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a184ce210 con 0x558a184954d0 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa234c7f640 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a18857a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.320+0000 7fa22ffff640 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a18857fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.321+0000 7fa22f7fe640 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184954d0 0x558a1885eee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa22400bcb0 tx=0x7fa2240107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436a0d02c15155f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.321+0000 7fa22d7fa640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2180b2870 0x7fa2180b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.321+0000 7fa234c7f640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2180b2870 0x7fa2180b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.321+0000 7fa2167fc640 1 -- 172.21.15.39:0/356393117 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22401c480 con 0x558a184954d0 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.321+0000 7fa22d7fa640 1 -- 172.21.15.39:0/3435377140 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa220102a10 con 0x7fa230102b40 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.321+0000 7fa2167fc640 1 -- 172.21.15.39:0/356393117 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22401cab0 con 0x558a184954d0 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.321+0000 7fa2167fc640 1 -- 172.21.15.39:0/356393117 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224011c10 con 0x558a184954d0 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 -- 172.21.15.39:0/356393117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184954d0 msgr2=0x558a1885eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa234c7f640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2180b2870 0x7fa2180b4d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x558a18858c30 tx=0x7fa228009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184954d0 0x558a1885eee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa22400bcb0 tx=0x7fa2240107a0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 -- 172.21.15.39:0/356393117 shutdown_connections 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184954d0 0x558a1885eee0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a18857fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 --2- 172.21.15.39:0/356393117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a18857a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 -- 172.21.15.39:0/356393117 >> 172.21.15.39:0/356393117 conn(0x558a184aa300 msgr2=0x558a188533a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 -- 172.21.15.39:0/356393117 shutdown_connections 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 -- 172.21.15.39:0/356393117 wait complete. 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.322+0000 7fa23d592740 1 -- start start 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa23d592740 1 Processor -- start 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa23d592740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a1848d2f0 0x558a18861670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa23d592740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18861bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa23d592740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a18868b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa23d592740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a1885f930 con 0x558a184b61b0 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa23d592740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558a18779370 con 0x558a184954d0 2024-05-04T03:11:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa23d592740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558a1885fab0 con 0x558a1848d2f0 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa23d592740 10 client.0 _ll_register_callbacks cb 0x558a184cda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa234c7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a18868b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa22f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18861bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa234c7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a18868b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56490/0 (socket says 172.21.15.39:56490) 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa234c7f640 1 -- 172.21.15.39:0/3146590200 learned_addr learned my addr 172.21.15.39:0/3146590200 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a1848d2f0 0x558a18861670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa234c7f640 1 -- 172.21.15.39:0/3146590200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a1848d2f0 msgr2=0x558a18861670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa234c7f640 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a1848d2f0 0x558a18861670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa234c7f640 1 -- 172.21.15.39:0/3146590200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 msgr2=0x558a18861bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.323+0000 7fa234c7f640 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18861bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa234c7f640 1 -- 172.21.15.39:0/3146590200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a1859a620 con 0x558a184b61b0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa22f7fe640 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18861bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa234c7f640 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a18868b10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa228011ef0 tx=0x7fa228004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd331db6792285b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa20f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa20f7fe640 1 -- 172.21.15.39:0/3146590200 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22801ed30 con 0x558a184b61b0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa23d592740 1 -- 172.21.15.39:0/3146590200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa22401cab0 con 0x558a184b61b0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa20f7fe640 1 -- 172.21.15.39:0/3146590200 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa22802a480 con 0x558a184b61b0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa20f7fe640 1 -- 172.21.15.39:0/3146590200 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa22802a680 con 0x558a184b61b0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa23d592740 10 client.15324 fetch_fsmap learned FSMap version 29 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa23d592740 1 -- 172.21.15.39:0/3146590200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a1886bb60 con 0x558a184b61b0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa20f7fe640 1 -- 172.21.15.39:0/3146590200 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22401cab0 con 0x558a184b61b0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.324+0000 7fa20f7fe640 1 -- 172.21.15.39:0/3146590200 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fa22800f040 con 0x558a184b61b0 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa23d592740 10 client.15324 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa23d592740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:11:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa23d592740 20 client.15324 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa23d592740 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558a18474d80 0x558a1886f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa23d592740 4 client.15324 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa23d592740 1 -- 172.21.15.39:0/3146590200 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a187ca1e0 con 0x558a18474d80 2024-05-04T03:11:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558a18474d80 0x558a1886f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa20f7fe640 10 client.15324 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.325+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558a18474d80 0x558a1886f320 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0150e4b8c8acc1 server_cookie=e0a8f62f66d4ef2a in_seq=0 out_seq=0 2024-05-04T03:11:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa20f7fe640 1 -- 172.21.15.39:0/3146590200 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a187ca1e0 con 0x558a18474d80 2024-05-04T03:11:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa20f7fe640 10 client.15324 handle_command_reply: tid=0 2024-05-04T03:11:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 client.15324 shutdown 2024-05-04T03:11:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 -- 172.21.15.39:0/3146590200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 msgr2=0x558a18868b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a18868b10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa228011ef0 tx=0x7fa228004a40 comp rx=0 tx=0).stop 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 -- 172.21.15.39:0/3146590200 shutdown_connections 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558a18474d80 0x558a1886f320 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a18868b10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18861bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa20f7fe640 0 client.15324 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 --2- 172.21.15.39:0/3146590200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a1848d2f0 0x558a18861670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.326+0000 7fa23d592740 1 -- 172.21.15.39:0/3146590200 >> 172.21.15.39:0/3146590200 conn(0x558a184aa300 msgr2=0x558a188523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.327+0000 7fa23d592740 1 -- 172.21.15.39:0/3146590200 shutdown_connections 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.327+0000 7fa23d592740 1 -- 172.21.15.39:0/3146590200 wait complete. 2024-05-04T03:11:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.327+0000 7fa23d592740 20 client.15324 trim_cache size 0 max 16384 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa23d592740 1 Processor -- start 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa23d592740 1 -- start start 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa23d592740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1848d2f0 0x558a1885baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa23d592740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18868a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa23d592740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a1885bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa23d592740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301ab2f0 con 0x558a1848d2f0 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa23d592740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301ace60 con 0x558a184954d0 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa23d592740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301ab9c0 con 0x558a184b61b0 2024-05-04T03:11:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1848d2f0 0x558a1885baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa22ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1848d2f0 0x558a1885baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56492/0 (socket says 172.21.15.39:56492) 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa22ffff640 1 -- 172.21.15.39:0/3430598234 learned_addr learned my addr 172.21.15.39:0/3430598234 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa22f7fe640 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18868a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.342+0000 7fa234c7f640 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a1885bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa22ffff640 1 -- 172.21.15.39:0/3430598234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 msgr2=0x558a1885bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a1885bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa22ffff640 1 -- 172.21.15.39:0/3430598234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 msgr2=0x558a18868a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18868a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa22ffff640 1 -- 172.21.15.39:0/3430598234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a18858750 con 0x558a1848d2f0 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa22ffff640 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1848d2f0 0x558a1885baa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa220100400 tx=0x7fa220106560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611569fa056d3c18 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa215ffb640 1 -- 172.21.15.39:0/3430598234 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220049070 con 0x558a1848d2f0 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa215ffb640 1 -- 172.21.15.39:0/3430598234 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa22003b460 con 0x558a1848d2f0 2024-05-04T03:11:33.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.343+0000 7fa215ffb640 1 -- 172.21.15.39:0/3430598234 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2200c9f90 con 0x558a1848d2f0 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 -- 172.21.15.39:0/3430598234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1848d2f0 msgr2=0x558a1885baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1848d2f0 0x558a1885baa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa220100400 tx=0x7fa220106560 comp rx=0 tx=0).stop 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 -- 172.21.15.39:0/3430598234 shutdown_connections 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184b61b0 0x558a1885bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a184954d0 0x558a18868a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 --2- 172.21.15.39:0/3430598234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1848d2f0 0x558a1885baa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 -- 172.21.15.39:0/3430598234 >> 172.21.15.39:0/3430598234 conn(0x558a184aa300 msgr2=0x558a18861670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 -- 172.21.15.39:0/3430598234 shutdown_connections 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 -- 172.21.15.39:0/3430598234 wait complete. 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 -- start start 2024-05-04T03:11:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 Processor -- start 2024-05-04T03:11:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a188657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184954d0 0x558a18865d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.344+0000 7fa23d592740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a188d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa23d592740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301ac760 con 0x558a184b61b0 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa23d592740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301ac090 con 0x558a1848d2f0 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa23d592740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301a92d0 con 0x558a184954d0 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa23d592740 10 client.0 _ll_register_callbacks cb 0x7fa228009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a188d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a188d5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56498/0 (socket says 172.21.15.39:56498) 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 -- 172.21.15.39:0/2936101438 learned_addr learned my addr 172.21.15.39:0/2936101438 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa22f7fe640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184954d0 0x558a18865d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa22ffff640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a188657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 -- 172.21.15.39:0/2936101438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184954d0 msgr2=0x558a18865d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184954d0 0x558a18865d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 -- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 msgr2=0x558a188657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a188657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 -- 172.21.15.39:0/2936101438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a18868c50 con 0x558a184b61b0 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa22f7fe640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184954d0 0x558a18865d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa22ffff640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a188657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa234c7f640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a188d5680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa22801f850 tx=0x7fa22801f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73dd5d29c4e5617 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa20f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa20f7fe640 1 -- 172.21.15.39:0/2936101438 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22803b070 con 0x558a184b61b0 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa20f7fe640 1 -- 172.21.15.39:0/2936101438 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa22801ed40 con 0x558a184b61b0 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.345+0000 7fa23d592740 1 -- 172.21.15.39:0/2936101438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558a187c4050 con 0x558a184b61b0 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa20f7fe640 1 -- 172.21.15.39:0/2936101438 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa228004ab0 con 0x558a184b61b0 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa23d592740 10 client.15330 fetch_fsmap learned FSMap version 29 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa23d592740 1 -- 172.21.15.39:0/2936101438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a188d6b20 con 0x558a184b61b0 2024-05-04T03:11:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa20f7fe640 1 -- 172.21.15.39:0/2936101438 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558a187c4050 con 0x558a184b61b0 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa20f7fe640 1 -- 172.21.15.39:0/2936101438 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fa22800f040 con 0x558a184b61b0 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa23d592740 10 client.15330 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa23d592740 10 client.15330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa23d592740 20 client.15330 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa23d592740 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558a1844a1c0 0x558a188d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa23d592740 4 client.15330 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa23d592740 1 -- 172.21.15.39:0/2936101438 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a1873f5f0 con 0x558a1844a1c0 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.346+0000 7fa22ffff640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558a1844a1c0 0x558a188d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.347+0000 7fa22ffff640 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558a1844a1c0 0x558a188d9570 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb375c3652340d04 server_cookie=e78a13b29601c8bc in_seq=0 out_seq=0 2024-05-04T03:11:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.347+0000 7fa20f7fe640 10 client.15330 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:33.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:33 smithi167 ceph-mon[33614]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:33.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:33 smithi039 ceph-mon[29715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa20f7fe640 1 -- 172.21.15.39:0/2936101438 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a1873f5f0 con 0x558a1844a1c0 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa20f7fe640 10 client.15330 handle_command_reply: tid=0 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 client.15330 shutdown 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 -- 172.21.15.39:0/2936101438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 msgr2=0x558a188d5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a188d5680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa22801f850 tx=0x7fa22801f640 comp rx=0 tx=0).stop 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 -- 172.21.15.39:0/2936101438 shutdown_connections 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558a1844a1c0 0x558a188d9570 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a184b61b0 0x558a188d5680 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558a184954d0 0x558a18865d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 --2- 172.21.15.39:0/2936101438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558a1848d2f0 0x558a188657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 -- 172.21.15.39:0/2936101438 >> 172.21.15.39:0/2936101438 conn(0x558a184aa300 msgr2=0x558a18861670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa20f7fe640 0 client.15330 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.456+0000 7fa23d592740 1 -- 172.21.15.39:0/2936101438 shutdown_connections 2024-05-04T03:11:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa23d592740 1 -- 172.21.15.39:0/2936101438 wait complete. 2024-05-04T03:11:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa23d592740 20 client.15330 trim_cache size 0 max 16384 2024-05-04T03:11:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:11:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:11:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:11:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:11:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 -- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2180b2870 msgr2=0x7fa2180b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2180b2870 0x7fa2180b4d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x558a18858c30 tx=0x7fa228009600 comp rx=0 tx=0).stop 2024-05-04T03:11:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 -- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 msgr2=0x7fa23019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 0x7fa23019a300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa220005760 tx=0x7fa220004270 comp rx=0 tx=0).stop 2024-05-04T03:11:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 -- 172.21.15.39:0/3435377140 shutdown_connections 2024-05-04T03:11:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa2180b2870 0x7fa2180b4d80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa230108a80 0x7fa2301a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa230103d80 0x7fa23019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 --2- 172.21.15.39:0/3435377140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa230102b40 0x7fa23019a300 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 -- 172.21.15.39:0/3435377140 >> 172.21.15.39:0/3435377140 conn(0x7fa2300fdd30 msgr2=0x7fa23010a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 -- 172.21.15.39:0/3435377140 shutdown_connections 2024-05-04T03:11:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:33.457+0000 7fa2366e2640 1 -- 172.21.15.39:0/3435377140 wait complete. 2024-05-04T03:11:33.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:33 smithi066 ceph-mon[33264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:33.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 -- 172.21.15.39:0/1474463245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 msgr2=0x7f99981100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 --2- 172.21.15.39:0/1474463245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 0x7f99981100a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f999000b0d0 tx=0x7f999000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 -- 172.21.15.39:0/1474463245 shutdown_connections 2024-05-04T03:11:33.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 --2- 172.21.15.39:0/1474463245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 0x7f99981100a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 --2- 172.21.15.39:0/1474463245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99981034c0 0x7f99981038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 --2- 172.21.15.39:0/1474463245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9998107df0 0x7f9998108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 -- 172.21.15.39:0/1474463245 >> 172.21.15.39:0/1474463245 conn(0x7f99980fedc0 msgr2=0x7f99981011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 -- 172.21.15.39:0/1474463245 shutdown_connections 2024-05-04T03:11:33.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.950+0000 7f999f00a700 1 -- 172.21.15.39:0/1474463245 wait complete. 2024-05-04T03:11:33.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.951+0000 7f999f00a700 1 Processor -- start 2024-05-04T03:11:33.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.951+0000 7f999f00a700 1 -- start start 2024-05-04T03:11:33.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.951+0000 7f999f00a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99981034c0 0x7f999819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.951+0000 7f999f00a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9998107df0 0x7f999819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.951+0000 7f999f00a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 0x7f999819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.951+0000 7f999f00a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9998113270 con 0x7f9998108740 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.951+0000 7f999f00a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99981130f0 con 0x7f99981034c0 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.951+0000 7f999f00a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99981133f0 con 0x7f9998107df0 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 0x7f999819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f9997fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9998107df0 0x7f999819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 0x7f999819f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56518/0 (socket says 172.21.15.39:56518) 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 -- 172.21.15.39:0/3751452069 learned_addr learned my addr 172.21.15.39:0/3751452069 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999cda6700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99981034c0 0x7f999819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 -- 172.21.15.39:0/3751452069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9998107df0 msgr2=0x7f999819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9998107df0 0x7f999819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 -- 172.21.15.39:0/3751452069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99981034c0 msgr2=0x7f999819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99981034c0 0x7f999819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 -- 172.21.15.39:0/3751452069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999819fc00 con 0x7f9998108740 2024-05-04T03:11:33.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999cda6700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99981034c0 0x7f999819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:33.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999d5a7700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 0x7f999819f560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f99900079a0 tx=0x7f99900079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f9995ffb700 1 -- 172.21.15.39:0/3751452069 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999001f070 con 0x7f9998108740 2024-05-04T03:11:33.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f999f00a700 1 -- 172.21.15.39:0/3751452069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999819fe90 con 0x7f9998108740 2024-05-04T03:11:33.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f9995ffb700 1 -- 172.21.15.39:0/3751452069 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9990003c00 con 0x7f9998108740 2024-05-04T03:11:33.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.952+0000 7f9995ffb700 1 -- 172.21.15.39:0/3751452069 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999001b430 con 0x7f9998108740 2024-05-04T03:11:33.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.953+0000 7f999f00a700 1 -- 172.21.15.39:0/3751452069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99981a7810 con 0x7f9998108740 2024-05-04T03:11:33.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.956+0000 7f9995ffb700 1 -- 172.21.15.39:0/3751452069 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f9990022070 con 0x7f9998108740 2024-05-04T03:11:33.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f99ada1eb80 1 Processor -- start 2024-05-04T03:11:33.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f9995ffb700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99880b2910 0x7f99880b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f99ada1eb80 1 -- start start 2024-05-04T03:11:33.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f999cda6700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99880b2910 0x7f99880b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924a8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5654923c08f0 con 0x5654924a2f20 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x565492325d50 con 0x5654924a25d0 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.957+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654923bc8b0 con 0x5654924a8450 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9997fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9997fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924a29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37516/0 (socket says 172.21.15.39:37516) 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9997fff700 1 -- 172.21.15.39:0/1919903699 learned_addr learned my addr 172.21.15.39:0/1919903699 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f999d5a7700 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f999cda6700 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f999cda6700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99880b2910 0x7f99880b4e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f998c00bc80 tx=0x7f998c00f640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9995ffb700 1 -- 172.21.15.39:0/3751452069 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f999000f020 con 0x7f9998108740 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9997fff700 1 -- 172.21.15.39:0/1919903699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 msgr2=0x5654924a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9997fff700 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924a8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9997fff700 1 -- 172.21.15.39:0/1919903699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 msgr2=0x5654924af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9997fff700 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f9997fff700 1 -- 172.21.15.39:0/1919903699 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549230d880 con 0x5654924a25d0 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f999cda6700 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.958+0000 7f999d5a7700 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.959+0000 7f9997fff700 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924a29e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9984009cb0 tx=0x7f9984011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=492ea0cc83c47247 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.959+0000 7f9982ffd700 1 -- 172.21.15.39:0/1919903699 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984011d20 con 0x5654924a25d0 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.959+0000 7f9982ffd700 1 -- 172.21.15.39:0/1919903699 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99840044e0 con 0x5654924a25d0 2024-05-04T03:11:33.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.959+0000 7f9982ffd700 1 -- 172.21.15.39:0/1919903699 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984005220 con 0x5654924a25d0 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1919903699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 msgr2=0x5654924a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924a29e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9984009cb0 tx=0x7f9984011440 comp rx=0 tx=0).stop 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1919903699 shutdown_connections 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/1919903699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924a29e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1919903699 >> 172.21.15.39:0/1919903699 conn(0x565492320df0 msgr2=0x565492499210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1919903699 shutdown_connections 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1919903699 wait complete. 2024-05-04T03:11:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.960+0000 7f99ada1eb80 1 -- start start 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f99ada1eb80 1 Processor -- start 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 0x5654924ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a8450 0x5654924ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5654924af820 con 0x5654924a2f20 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x565492379e80 con 0x5654924a8450 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654924af9a0 con 0x5654924a25d0 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f99ada1eb80 10 client.0 _ll_register_callbacks cb 0x565492358900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f9997fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f9997fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56528/0 (socket says 172.21.15.39:56528) 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f9997fff700 1 -- 172.21.15.39:0/3195147688 learned_addr learned my addr 172.21.15.39:0/3195147688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f999cda6700 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 0x5654924ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.961+0000 7f999d5a7700 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a8450 0x5654924ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9997fff700 1 -- 172.21.15.39:0/3195147688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 msgr2=0x5654924ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9997fff700 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 0x5654924ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9997fff700 1 -- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a8450 msgr2=0x5654924ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9997fff700 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a8450 0x5654924ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9997fff700 1 -- 172.21.15.39:0/3195147688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565492377710 con 0x5654924a2f20 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f999d5a7700 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a8450 0x5654924ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9997fff700 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924acb70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9984009c80 tx=0x7f9984004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a98c6051dc96e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9977fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9977fff700 1 -- 172.21.15.39:0/3195147688 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984026020 con 0x5654924a2f20 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.962+0000 7f9977fff700 1 -- 172.21.15.39:0/3195147688 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9984011b20 con 0x5654924a2f20 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3195147688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99840044e0 con 0x5654924a2f20 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f9977fff700 1 -- 172.21.15.39:0/3195147688 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9984011d60 con 0x5654924a2f20 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f99ada1eb80 10 client.15342 fetch_fsmap learned FSMap version 29 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3195147688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654924ba3b0 con 0x5654924a2f20 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f9977fff700 1 -- 172.21.15.39:0/3195147688 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99840044e0 con 0x5654924a2f20 2024-05-04T03:11:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f9977fff700 1 -- 172.21.15.39:0/3195147688 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9984018070 con 0x5654924a2f20 2024-05-04T03:11:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f99ada1eb80 10 client.15342 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:11:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f99ada1eb80 10 client.15342 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-04T03:11:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f99ada1eb80 20 client.15342 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.963+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5654924bc7e0 0x5654924bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.964+0000 7f99ada1eb80 4 client.15342 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.964+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3195147688 --> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654923ce2d0 con 0x5654924bc7e0 2024-05-04T03:11:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.964+0000 7f999cda6700 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5654924bc7e0 0x5654924bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.964+0000 7f9977fff700 10 client.15342 ms_handle_connect on v2:172.21.15.66:6834/3463909305 2024-05-04T03:11:33.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.965+0000 7f999cda6700 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5654924bc7e0 0x5654924bdc50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=620fdb2d75bf0ed6 server_cookie=d2ff6ef0339e4a4b in_seq=0 out_seq=0 2024-05-04T03:11:33.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.965+0000 7f9977fff700 1 -- 172.21.15.39:0/3195147688 <== mds.4 v2:172.21.15.66:6834/3463909305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654923ce2d0 con 0x5654924bc7e0 2024-05-04T03:11:33.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.965+0000 7f9977fff700 10 client.15342 handle_command_reply: tid=0 2024-05-04T03:11:33.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.965+0000 7f99ada1eb80 1 client.15342 shutdown 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3195147688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 msgr2=0x5654924acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924acb70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9984009c80 tx=0x7f9984004050 comp rx=0 tx=0).stop 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3195147688 shutdown_connections 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x5654924bc7e0 0x5654924bdc50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a8450 0x5654924ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 0x5654924ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3195147688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924acb70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3195147688 >> 172.21.15.39:0/3195147688 conn(0x565492320df0 msgr2=0x56549249d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f9977fff700 0 client.15342 ms_handle_reset on v2:172.21.15.66:6834/3463909305 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3195147688 shutdown_connections 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3195147688 wait complete. 2024-05-04T03:11:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.966+0000 7f99ada1eb80 20 client.15342 trim_cache size 0 max 16384 2024-05-04T03:11:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f99ada1eb80 1 Processor -- start 2024-05-04T03:11:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f99ada1eb80 1 -- start start 2024-05-04T03:11:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 0x5654924bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a2f20 0x5654924a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a8450 0x5654924ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5654924bbf90 con 0x5654924a8450 2024-05-04T03:11:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5654924bc660 con 0x5654924a2f20 2024-05-04T03:11:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654924bcd30 con 0x5654924a25d0 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f999cda6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 0x5654924bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f9997fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a2f20 0x5654924a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f9997fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a2f20 0x5654924a6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37524/0 (socket says 172.21.15.39:37524) 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f9997fff700 1 -- 172.21.15.39:0/1494438769 learned_addr learned my addr 172.21.15.39:0/1494438769 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.996+0000 7f999d5a7700 1 --2- 172.21.15.39:0/1494438769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a8450 0x5654924ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f9997fff700 1 -- 172.21.15.39:0/1494438769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 msgr2=0x5654924bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f9997fff700 1 --2- 172.21.15.39:0/1494438769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 0x5654924bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f9997fff700 1 -- 172.21.15.39:0/1494438769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a8450 msgr2=0x5654924ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f9997fff700 1 --2- 172.21.15.39:0/1494438769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a8450 0x5654924ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f9997fff700 1 -- 172.21.15.39:0/1494438769 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654924a35c0 con 0x5654924a2f20 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f9997fff700 1 --2- 172.21.15.39:0/1494438769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a2f20 0x5654924a6db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9984011480 tx=0x7f9984005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4134dd2366721e99 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f99827fc700 1 -- 172.21.15.39:0/1494438769 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984026070 con 0x5654924a2f20 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f99827fc700 1 -- 172.21.15.39:0/1494438769 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9984011b20 con 0x5654924a2f20 2024-05-04T03:11:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.997+0000 7f99827fc700 1 -- 172.21.15.39:0/1494438769 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984003b10 con 0x5654924a2f20 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1494438769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a2f20 msgr2=0x5654924a6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/1494438769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a2f20 0x5654924a6db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9984011480 tx=0x7f9984005a00 comp rx=0 tx=0).stop 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1494438769 shutdown_connections 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/1494438769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a8450 0x5654924ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/1494438769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a25d0 0x5654924bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/1494438769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a2f20 0x5654924a6db0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1494438769 >> 172.21.15.39:0/1494438769 conn(0x565492320df0 msgr2=0x5654924bb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1494438769 shutdown_connections 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.998+0000 7f99ada1eb80 1 -- 172.21.15.39:0/1494438769 wait complete. 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.999+0000 7f99ada1eb80 1 -- start start 2024-05-04T03:11:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.999+0000 7f99ada1eb80 1 Processor -- start 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.999+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.999+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924c0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.999+0000 7f99ada1eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:33.999+0000 7f99ada1eb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5654924bdad0 con 0x5654924a2f20 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999cda6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f9997fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999d5a7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999cda6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37536/0 (socket says 172.21.15.39:37536) 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f9997fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924c0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56546/0 (socket says 172.21.15.39:56546) 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999cda6700 1 -- 172.21.15.39:0/3807418130 learned_addr learned my addr 172.21.15.39:0/3807418130 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5654924bd400 con 0x5654924a25d0 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654924be1a0 con 0x5654924a8450 2024-05-04T03:11:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f99ada1eb80 10 client.0 _ll_register_callbacks cb 0x565492358900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999d5a7700 1 -- 172.21.15.39:0/3807418130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 msgr2=0x5654924c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999d5a7700 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999d5a7700 1 -- 172.21.15.39:0/3807418130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 msgr2=0x5654924c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999d5a7700 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924c0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999d5a7700 1 -- 172.21.15.39:0/3807418130 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565492495760 con 0x5654924a8450 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999cda6700 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f999d5a7700 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924c0dd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99900e0f80 tx=0x7f99900e1400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f28f49cb710614 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.000+0000 7f9977fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:11:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f9977fff700 1 -- 172.21.15.39:0/3807418130 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99900aa470 con 0x5654924a8450 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f9977fff700 1 -- 172.21.15.39:0/3807418130 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9990003db0 con 0x5654924a8450 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9984011b20 con 0x5654924a8450 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f9977fff700 1 -- 172.21.15.39:0/3807418130 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99900a62b0 con 0x5654924a8450 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f99ada1eb80 10 client.25048 fetch_fsmap learned FSMap version 29 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654924c5cd0 con 0x5654924a8450 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f9977fff700 1 -- 172.21.15.39:0/3807418130 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9984011b20 con 0x5654924a8450 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f9977fff700 1 -- 172.21.15.39:0/3807418130 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9990011070 con 0x5654924a8450 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f99ada1eb80 10 client.25048 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f99ada1eb80 10 client.25048 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f99ada1eb80 20 client.25048 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x565492529da0 0x56549252b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.001+0000 7f99ada1eb80 4 client.25048 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.002+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 --> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] -- command(tid 0: {"prefix": "status"}) -- 0x5654923c8370 con 0x565492529da0 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.002+0000 7f999cda6700 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x565492529da0 0x56549252b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.002+0000 7f9977fff700 10 client.25048 ms_handle_connect on v2:172.21.15.66:6834/3463909305 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.002+0000 7f999cda6700 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x565492529da0 0x56549252b210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c79f6862c681f139 server_cookie=c9a302fd33a9b373 in_seq=0 out_seq=0 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.002+0000 7f9977fff700 1 -- 172.21.15.39:0/3807418130 <== mds.4 v2:172.21.15.66:6834/3463909305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5654923c8370 con 0x565492529da0 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.002+0000 7f9977fff700 10 client.25048 handle_command_reply: tid=0 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.002+0000 7f99ada1eb80 1 client.25048 shutdown 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 msgr2=0x5654924c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924c0dd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99900e0f80 tx=0x7f99900e1400 comp rx=0 tx=0).stop 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 shutdown_connections 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.66:6834/3463909305,v1:172.21.15.66:6835/3463909305] conn(0x565492529da0 0x56549252b210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5654924a8450 0x5654924c0dd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5654924a25d0 0x5654924c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 --2- 172.21.15.39:0/3807418130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5654924a2f20 0x5654924c0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f9977fff700 0 client.25048 ms_handle_reset on v2:172.21.15.66:6834/3463909305 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 >> 172.21.15.39:0/3807418130 conn(0x565492320df0 msgr2=0x5654924bb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 shutdown_connections 2024-05-04T03:11:34.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 1 -- 172.21.15.39:0/3807418130 wait complete. 2024-05-04T03:11:34.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.003+0000 7f99ada1eb80 20 client.25048 trim_cache size 0 max 16384 2024-05-04T03:11:34.008 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:11:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:11:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-05-04T03:11:34.008 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24455, 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 216.837552303, 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 80, 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 275.53150215599999 2024-05-04T03:11:34.009 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:11:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 -- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99880b2910 msgr2=0x7f99880b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99880b2910 0x7f99880b4e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f998c00bc80 tx=0x7f998c00f640 comp rx=0 tx=0).stop 2024-05-04T03:11:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 -- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 msgr2=0x7f999819f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 0x7f999819f560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f99900079a0 tx=0x7f99900079d0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 -- 172.21.15.39:0/3751452069 shutdown_connections 2024-05-04T03:11:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99880b2910 0x7f99880b4e20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9998108740 0x7f999819f560 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99981034c0 0x7f999819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 --2- 172.21.15.39:0/3751452069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9998107df0 0x7f999819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 -- 172.21.15.39:0/3751452069 >> 172.21.15.39:0/3751452069 conn(0x7f99980fedc0 msgr2=0x7f999806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 -- 172.21.15.39:0/3751452069 shutdown_connections 2024-05-04T03:11:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:34.004+0000 7f999f00a700 1 -- 172.21.15.39:0/3751452069 wait complete. 2024-05-04T03:11:34.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:34 smithi167 ceph-mon[33614]: pgmap v409: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 17 op/s 2024-05-04T03:11:34.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:34 smithi039 ceph-mon[29715]: pgmap v409: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 17 op/s 2024-05-04T03:11:34.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:34 smithi066 ceph-mon[33264]: pgmap v409: 97 pgs: 97 active+clean; 13 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 17 op/s 2024-05-04T03:11:35.077 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-04T03:11:35.369 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:36.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:36 smithi167 ceph-mon[33614]: pgmap v410: 97 pgs: 97 active+clean; 14 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 20 op/s 2024-05-04T03:11:36.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:36 smithi039 ceph-mon[29715]: pgmap v410: 97 pgs: 97 active+clean; 14 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 20 op/s 2024-05-04T03:11:36.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:36 smithi066 ceph-mon[33264]: pgmap v410: 97 pgs: 97 active+clean; 14 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 20 op/s 2024-05-04T03:11:37.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 -- 172.21.15.39:0/116625537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8108760 msgr2=0x7f94e81100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 --2- 172.21.15.39:0/116625537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8108760 0x7f94e81100c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b0d0 tx=0x7f94e000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 -- 172.21.15.39:0/116625537 shutdown_connections 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 --2- 172.21.15.39:0/116625537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8108760 0x7f94e81100c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 --2- 172.21.15.39:0/116625537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94e81034e0 0x7f94e81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 --2- 172.21.15.39:0/116625537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e8107e10 0x7f94e8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 -- 172.21.15.39:0/116625537 >> 172.21.15.39:0/116625537 conn(0x7f94e80fef30 msgr2=0x7f94e8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 -- 172.21.15.39:0/116625537 shutdown_connections 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 -- 172.21.15.39:0/116625537 wait complete. 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 Processor -- start 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.257+0000 7f94eed92700 1 -- start start 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94eed92700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e81034e0 0x7f94e819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94eed92700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e8107e10 0x7f94e819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94eed92700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94e8108760 0x7f94e819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94eed92700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8113290 con 0x7f94e81034e0 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94eed92700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8113110 con 0x7f94e8108760 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94eed92700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8113410 con 0x7f94e8107e10 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ecb2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e81034e0 0x7f94e819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ecb2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e81034e0 0x7f94e819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39514/0 (socket says 172.21.15.39:39514) 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ecb2e700 1 -- 172.21.15.39:0/1514939004 learned_addr learned my addr 172.21.15.39:0/1514939004 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ed32f700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94e8108760 0x7f94e819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94e7fff700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e8107e10 0x7f94e819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ed32f700 1 -- 172.21.15.39:0/1514939004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e8107e10 msgr2=0x7f94e819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ed32f700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e8107e10 0x7f94e819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ed32f700 1 -- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e81034e0 msgr2=0x7f94e819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ed32f700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e81034e0 0x7f94e819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ed32f700 1 -- 172.21.15.39:0/1514939004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e819fc80 con 0x7f94e8108760 2024-05-04T03:11:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.258+0000 7f94ed32f700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94e8108760 0x7f94e819f5e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b770 tx=0x7f94e00079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.259+0000 7f94e5ffb700 1 -- 172.21.15.39:0/1514939004 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e001f070 con 0x7f94e8108760 2024-05-04T03:11:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.259+0000 7f94eed92700 1 -- 172.21.15.39:0/1514939004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e819ff10 con 0x7f94e8108760 2024-05-04T03:11:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.259+0000 7f94e5ffb700 1 -- 172.21.15.39:0/1514939004 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94e0004c50 con 0x7f94e8108760 2024-05-04T03:11:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.259+0000 7f94e5ffb700 1 -- 172.21.15.39:0/1514939004 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e001b430 con 0x7f94e8108760 2024-05-04T03:11:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.259+0000 7f94eed92700 1 -- 172.21.15.39:0/1514939004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e81a03f0 con 0x7f94e8108760 2024-05-04T03:11:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.261+0000 7f94e5ffb700 1 -- 172.21.15.39:0/1514939004 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f94e0022070 con 0x7f94e8108760 2024-05-04T03:11:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.262+0000 7f94e5ffb700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94d80b2910 0x7f94d80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.262+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94d80b2910 0x7f94d80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.262+0000 7f94e5ffb700 1 -- 172.21.15.39:0/1514939004 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f94e000f020 con 0x7f94e8108760 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.262+0000 7f94fd7a6b80 1 Processor -- start 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94fd7a6b80 1 -- start start 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94d80b2910 0x7f94d80b4e20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009d20 tx=0x7f94dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634dbe4e0 0x561634dbe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db85d0 0x561634db89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db8f20 0x561634dc51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561634cd68f0 con 0x561634db8f20 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561634c3bd50 con 0x561634db85d0 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561634cd28b0 con 0x561634dbe4e0 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ecb2e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634dbe4e0 0x561634dbe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ed32f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db8f20 0x561634dc51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94e7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db85d0 0x561634db89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ecb2e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634dbe4e0 0x561634dbe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:55106/0 (socket says 172.21.15.39:55106) 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ecb2e700 1 -- 172.21.15.39:0/712814331 learned_addr learned my addr 172.21.15.39:0/712814331 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ed32f700 1 -- 172.21.15.39:0/712814331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634dbe4e0 msgr2=0x561634dbe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ed32f700 1 --2- 172.21.15.39:0/712814331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634dbe4e0 0x561634dbe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ed32f700 1 -- 172.21.15.39:0/712814331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db85d0 msgr2=0x561634db89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ed32f700 1 --2- 172.21.15.39:0/712814331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db85d0 0x561634db89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.263+0000 7f94ed32f700 1 -- 172.21.15.39:0/712814331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561634c23880 con 0x561634db8f20 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.264+0000 7f94ed32f700 1 --2- 172.21.15.39:0/712814331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db8f20 0x561634dc51c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f94e00e0e00 tx=0x7f94e00a6130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67e90b2e4a3e311 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.264+0000 7f94d2ffd700 1 -- 172.21.15.39:0/712814331 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00b30c0 con 0x561634db8f20 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.264+0000 7f94d2ffd700 1 -- 172.21.15.39:0/712814331 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94e00ab0a0 con 0x561634db8f20 2024-05-04T03:11:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.264+0000 7f94d2ffd700 1 -- 172.21.15.39:0/712814331 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00b10d0 con 0x561634db8f20 2024-05-04T03:11:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/712814331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db8f20 msgr2=0x561634dc51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/712814331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db8f20 0x561634dc51c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f94e00e0e00 tx=0x7f94e00a6130 comp rx=0 tx=0).stop 2024-05-04T03:11:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/712814331 shutdown_connections 2024-05-04T03:11:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/712814331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db8f20 0x561634dc51c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/712814331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634dbe4e0 0x561634dbe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/712814331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db85d0 0x561634db89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/712814331 >> 172.21.15.39:0/712814331 conn(0x561634c36df0 msgr2=0x561634daf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/712814331 shutdown_connections 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/712814331 wait complete. 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 -- start start 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.265+0000 7f94fd7a6b80 1 Processor -- start 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dc26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dc2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561634dc5700 con 0x561634dbe4e0 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561634c8fe80 con 0x561634db8f20 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561634dc5880 con 0x561634db85d0 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94fd7a6b80 10 client.0 _ll_register_callbacks cb 0x561634c6e900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94e7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dc2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39528/0 (socket says 172.21.15.39:39528) 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 -- 172.21.15.39:0/2090953506 learned_addr learned my addr 172.21.15.39:0/2090953506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dc26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 -- 172.21.15.39:0/2090953506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 msgr2=0x561634dc26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dc26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 -- 172.21.15.39:0/2090953506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 msgr2=0x561634dc2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dc2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 -- 172.21.15.39:0/2090953506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561634c8d710 con 0x561634dbe4e0 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94e7fff700 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dc2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94ed32f700 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc3130 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f94e00aa6a0 tx=0x7f94e000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d811bdbf89b60f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.266+0000 7f94c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94c7fff700 1 -- 172.21.15.39:0/2090953506 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00c4060 con 0x561634dbe4e0 2024-05-04T03:11:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94c7fff700 1 -- 172.21.15.39:0/2090953506 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f94e00cd0c0 con 0x561634dbe4e0 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/2090953506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e00ab0a0 con 0x561634dbe4e0 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94c7fff700 1 -- 172.21.15.39:0/2090953506 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94e00aed50 con 0x561634dbe4e0 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 10 client.15360 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/2090953506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561634dc5700 con 0x561634dbe4e0 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94c7fff700 1 -- 172.21.15.39:0/2090953506 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94e00ab0a0 con 0x561634dbe4e0 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94c7fff700 1 -- 172.21.15.39:0/2090953506 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f94e006f030 con 0x561634dbe4e0 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 10 client.15360 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 20 client.15360 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x561634dd2650 0x561634dd3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 4 client.15360 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.267+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/2090953506 --> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561634ce42d0 con 0x561634dd2650 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.268+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x561634dd2650 0x561634dd3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.268+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x561634dd2650 0x561634dd3ac0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a35d9fbca1d056c0 server_cookie=d5ef0ee22bbba0e8 in_seq=0 out_seq=0 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.268+0000 7f94c7fff700 10 client.15360 ms_handle_connect on v2:172.21.15.167:6834/2699315372 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94c7fff700 1 -- 172.21.15.39:0/2090953506 <== mds.3 v2:172.21.15.167:6834/2699315372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561634ce42d0 con 0x561634dd2650 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94c7fff700 10 client.15360 handle_command_reply: tid=0 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 client.15360 shutdown 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/2090953506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 msgr2=0x561634dc3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc3130 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f94e00aa6a0 tx=0x7f94e000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/2090953506 shutdown_connections 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x561634dd2650 0x561634dd3ac0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc3130 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dc26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/2090953506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dc2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94c7fff700 0 client.15360 ms_handle_reset on v2:172.21.15.167:6834/2699315372 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/2090953506 >> 172.21.15.39:0/2090953506 conn(0x561634c36df0 msgr2=0x561634db75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/2090953506 shutdown_connections 2024-05-04T03:11:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/2090953506 wait complete. 2024-05-04T03:11:37.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.269+0000 7f94fd7a6b80 20 client.15360 trim_cache size 0 max 16384 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94fd7a6b80 1 Processor -- start 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94fd7a6b80 1 -- start start 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db85d0 0x561634dc19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db8f20 0x561634dce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634dbe4e0 0x561634dc2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561634dd1e00 con 0x561634db85d0 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561634dd24d0 con 0x561634dbe4e0 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561634dd2ba0 con 0x561634db8f20 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ecb2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db85d0 0x561634dc19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ecb2e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db85d0 0x561634dc19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39536/0 (socket says 172.21.15.39:39536) 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ecb2e700 1 -- 172.21.15.39:0/562392703 learned_addr learned my addr 172.21.15.39:0/562392703 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ed32f700 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634dbe4e0 0x561634dc2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94e7fff700 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db8f20 0x561634dce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ecb2e700 1 -- 172.21.15.39:0/562392703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db8f20 msgr2=0x561634dce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db8f20 0x561634dce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ecb2e700 1 -- 172.21.15.39:0/562392703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634dbe4e0 msgr2=0x561634dc2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634dbe4e0 0x561634dc2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.291+0000 7f94ecb2e700 1 -- 172.21.15.39:0/562392703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561634db0b60 con 0x561634db85d0 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.292+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db85d0 0x561634dc19d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f94dc013010 tx=0x7f94dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb76f5ee303f41fd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.292+0000 7f94d27fc700 1 -- 172.21.15.39:0/562392703 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc031070 con 0x561634db85d0 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.292+0000 7f94d27fc700 1 -- 172.21.15.39:0/562392703 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94dc02c7f0 con 0x561634db85d0 2024-05-04T03:11:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.292+0000 7f94d27fc700 1 -- 172.21.15.39:0/562392703 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc01da50 con 0x561634db85d0 2024-05-04T03:11:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/562392703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db85d0 msgr2=0x561634dc19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db85d0 0x561634dc19d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f94dc013010 tx=0x7f94dc014000 comp rx=0 tx=0).stop 2024-05-04T03:11:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/562392703 shutdown_connections 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634db85d0 0x561634dc19d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634dbe4e0 0x561634dc2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/562392703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db8f20 0x561634dce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/562392703 >> 172.21.15.39:0/562392703 conn(0x561634c36df0 msgr2=0x561634dc3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/562392703 shutdown_connections 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/562392703 wait complete. 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 -- start start 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.293+0000 7f94fd7a6b80 1 Processor -- start 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dbf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dbfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94fd7a6b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561634dd3940 con 0x561634dbe4e0 2024-05-04T03:11:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561634dd3270 con 0x561634db8f20 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94fd7a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561634dd4010 con 0x561634db85d0 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94fd7a6b80 10 client.0 _ll_register_callbacks cb 0x561634c6e900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dbf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ed32f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94e7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dbfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dbf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:55140/0 (socket says 172.21.15.39:55140) 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 -- 172.21.15.39:0/1401314900 learned_addr learned my addr 172.21.15.39:0/1401314900 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ed32f700 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39538/0 (socket says 172.21.15.39:39538) 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 -- 172.21.15.39:0/1401314900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 msgr2=0x561634dbfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dbfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 -- 172.21.15.39:0/1401314900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 msgr2=0x561634dc0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 -- 172.21.15.39:0/1401314900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561634ddb730 con 0x561634db85d0 2024-05-04T03:11:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ed32f700 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94ecb2e700 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dbf710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94dc006010 tx=0x7f94dc02cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c69ffef749bb74a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.294+0000 7f94c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94c7fff700 1 -- 172.21.15.39:0/1401314900 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc03a070 con 0x561634db85d0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94c7fff700 1 -- 172.21.15.39:0/1401314900 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f94dc005040 con 0x561634db85d0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/1401314900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc02c7f0 con 0x561634db85d0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94c7fff700 1 -- 172.21.15.39:0/1401314900 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94dc046050 con 0x561634db85d0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 10 client.25066 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/1401314900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561634e3df50 con 0x561634db85d0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94c7fff700 1 -- 172.21.15.39:0/1401314900 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94dc02c7f0 con 0x561634db85d0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94c7fff700 1 -- 172.21.15.39:0/1401314900 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f94dc036450 con 0x561634db85d0 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 10 client.25066 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 10 client.25066 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 20 client.25066 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x561634e40080 0x561634e414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 4 client.25066 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:11:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.295+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/1401314900 --> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] -- command(tid 0: {"prefix": "status"}) -- 0x561634cde370 con 0x561634e40080 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94e7fff700 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x561634e40080 0x561634e414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94c7fff700 10 client.25066 ms_handle_connect on v2:172.21.15.167:6834/2699315372 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94e7fff700 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x561634e40080 0x561634e414f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e93e9f9a77e084a server_cookie=4ec9f4131e479de8 in_seq=0 out_seq=0 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94c7fff700 1 -- 172.21.15.39:0/1401314900 <== mds.3 v2:172.21.15.167:6834/2699315372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561634cde370 con 0x561634e40080 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94c7fff700 10 client.25066 handle_command_reply: tid=0 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 client.25066 shutdown 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/1401314900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 msgr2=0x561634dbf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dbf710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94dc006010 tx=0x7f94dc02cbe0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/1401314900 shutdown_connections 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.167:6834/2699315372,v1:172.21.15.167:6835/2699315372] conn(0x561634e40080 0x561634e414f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561634dbe4e0 0x561634dc0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561634db85d0 0x561634dbf710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94c7fff700 0 client.25066 ms_handle_reset on v2:172.21.15.167:6834/2699315372 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 --2- 172.21.15.39:0/1401314900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561634db8f20 0x561634dbfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/1401314900 >> 172.21.15.39:0/1401314900 conn(0x561634c36df0 msgr2=0x561634dc1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.296+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/1401314900 shutdown_connections 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94fd7a6b80 1 -- 172.21.15.39:0/1401314900 wait complete. 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94fd7a6b80 20 client.25066 trim_cache size 0 max 16384 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24466, 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:11:37.306 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:11:37.307 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 221.147843435, 2024-05-04T03:11:37.307 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-05-04T03:11:37.307 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:11:37.307 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 80, 2024-05-04T03:11:37.307 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 276.03761086600002 2024-05-04T03:11:37.307 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:11:37.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 -- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94d80b2910 msgr2=0x7f94d80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94d80b2910 0x7f94d80b4e20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009d20 tx=0x7f94dc009260 comp rx=0 tx=0).stop 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 -- 172.21.15.39:0/1514939004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94e8108760 msgr2=0x7f94e819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94e8108760 0x7f94e819f5e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b770 tx=0x7f94e00079a0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 -- 172.21.15.39:0/1514939004 shutdown_connections 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f94d80b2910 0x7f94d80b4e20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e81034e0 0x7f94e819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94e8108760 0x7f94e819f5e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 --2- 172.21.15.39:0/1514939004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94e8107e10 0x7f94e819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 -- 172.21.15.39:0/1514939004 >> 172.21.15.39:0/1514939004 conn(0x7f94e80fef30 msgr2=0x7f94e806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 -- 172.21.15.39:0/1514939004 shutdown_connections 2024-05-04T03:11:37.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:37.297+0000 7f94eed92700 1 -- 172.21.15.39:0/1514939004 wait complete. 2024-05-04T03:11:38.197 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-04T03:11:38.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:38 smithi167 ceph-mon[33614]: pgmap v411: 97 pgs: 97 active+clean; 22 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.0 MiB/s wr, 36 op/s 2024-05-04T03:11:38.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:38 smithi167 ceph-mon[33614]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:38.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:38 smithi039 ceph-mon[29715]: pgmap v411: 97 pgs: 97 active+clean; 22 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.0 MiB/s wr, 36 op/s 2024-05-04T03:11:38.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:38 smithi039 ceph-mon[29715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:38.450 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:38.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:38 smithi066 ceph-mon[33264]: pgmap v411: 97 pgs: 97 active+clean; 22 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.0 MiB/s wr, 36 op/s 2024-05-04T03:11:38.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:38 smithi066 ceph-mon[33264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:11:40.212 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:40 smithi039 ceph-mon[29715]: pgmap v412: 97 pgs: 97 active+clean; 28 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 71 op/s 2024-05-04T03:11:40.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.234+0000 7f255ceb0700 1 -- 172.21.15.39:0/1849485807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558108740 msgr2=0x7f25581100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.234+0000 7f255ceb0700 1 --2- 172.21.15.39:0/1849485807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558108740 0x7f25581100a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f25500099c0 tx=0x7f2550009cd0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.234+0000 7f255ceb0700 1 -- 172.21.15.39:0/1849485807 shutdown_connections 2024-05-04T03:11:40.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.234+0000 7f255ceb0700 1 --2- 172.21.15.39:0/1849485807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558108740 0x7f25581100a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.234+0000 7f255ceb0700 1 --2- 172.21.15.39:0/1849485807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25581034c0 0x7f25581038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.234+0000 7f255ceb0700 1 --2- 172.21.15.39:0/1849485807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2558107df0 0x7f2558108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.234+0000 7f255ceb0700 1 -- 172.21.15.39:0/1849485807 >> 172.21.15.39:0/1849485807 conn(0x7f25580fedc0 msgr2=0x7f25581011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.235+0000 7f255ceb0700 1 -- 172.21.15.39:0/1849485807 shutdown_connections 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.235+0000 7f255ceb0700 1 -- 172.21.15.39:0/1849485807 wait complete. 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.235+0000 7f255ceb0700 1 Processor -- start 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.235+0000 7f255ceb0700 1 -- start start 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f255ceb0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25581034c0 0x7f25581a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f255ceb0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558107df0 0x7f25581a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f255ceb0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2558108740 0x7f255819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f255ceb0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2558114370 con 0x7f2558107df0 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f255ceb0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f25581141f0 con 0x7f2558108740 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f255ceb0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25581144f0 con 0x7f25581034c0 2024-05-04T03:11:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558107df0 0x7f25581a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558107df0 0x7f25581a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39564/0 (socket says 172.21.15.39:39564) 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2555d9b700 1 -- 172.21.15.39:0/360705896 learned_addr learned my addr 172.21.15.39:0/360705896 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2556d9d700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2558108740 0x7f255819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f255659c700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25581034c0 0x7f25581a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2555d9b700 1 -- 172.21.15.39:0/360705896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25581034c0 msgr2=0x7f25581a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2555d9b700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25581034c0 0x7f25581a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2555d9b700 1 -- 172.21.15.39:0/360705896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2558108740 msgr2=0x7f255819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2555d9b700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2558108740 0x7f255819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2555d9b700 1 -- 172.21.15.39:0/360705896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255819b1f0 con 0x7f2558107df0 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.236+0000 7f2556d9d700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2558108740 0x7f255819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.237+0000 7f2555d9b700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558107df0 0x7f25581a0f00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f254000c9e0 tx=0x7f254000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:40.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.237+0000 7f254f7fe700 1 -- 172.21.15.39:0/360705896 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540007aa0 con 0x7f2558107df0 2024-05-04T03:11:40.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.237+0000 7f254f7fe700 1 -- 172.21.15.39:0/360705896 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2540007c40 con 0x7f2558107df0 2024-05-04T03:11:40.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.237+0000 7f254f7fe700 1 -- 172.21.15.39:0/360705896 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540018400 con 0x7f2558107df0 2024-05-04T03:11:40.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.237+0000 7f255ceb0700 1 -- 172.21.15.39:0/360705896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255819b4e0 con 0x7f2558107df0 2024-05-04T03:11:40.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.237+0000 7f255ceb0700 1 -- 172.21.15.39:0/360705896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25581109b0 con 0x7f2558107df0 2024-05-04T03:11:40.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.240+0000 7f254f7fe700 1 -- 172.21.15.39:0/360705896 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f2540004850 con 0x7f2558107df0 2024-05-04T03:11:40.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.241+0000 7f254f7fe700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f25440b2910 0x7f25440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.241+0000 7f255659c700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f25440b2910 0x7f25440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.241+0000 7f254f7fe700 1 -- 172.21.15.39:0/360705896 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f25400e2220 con 0x7f2558107df0 2024-05-04T03:11:40.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.241+0000 7f256b8c4b80 1 Processor -- start 2024-05-04T03:11:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.241+0000 7f256b8c4b80 1 -- start start 2024-05-04T03:11:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.241+0000 7f255659c700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f25440b2910 0x7f25440b4e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2548010860 tx=0x7f2548012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188406da0 0x5621884071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 0x562188401300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562188323810 con 0x562188406da0 2024-05-04T03:11:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5621882c2770 con 0x562188400ef0 2024-05-04T03:11:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56218831f7d0 con 0x562188401840 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188406da0 0x5621884071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188406da0 0x5621884071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39576/0 (socket says 172.21.15.39:39576) 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 -- 172.21.15.39:0/2096692655 learned_addr learned my addr 172.21.15.39:0/2096692655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f2555d9b700 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 0x562188401300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f2556d9d700 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 -- 172.21.15.39:0/2096692655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 msgr2=0x56218840d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 -- 172.21.15.39:0/2096692655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 msgr2=0x562188401300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 0x562188401300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 -- 172.21.15.39:0/2096692655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562188270880 con 0x562188406da0 2024-05-04T03:11:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.242+0000 7f255659c700 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188406da0 0x5621884071b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2548003a00 tx=0x7f2548004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e3971a081e5379 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:40.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.243+0000 7f254cff9700 1 -- 172.21.15.39:0/2096692655 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548004bb0 con 0x562188406da0 2024-05-04T03:11:40.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.243+0000 7f254cff9700 1 -- 172.21.15.39:0/2096692655 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2548004800 con 0x562188406da0 2024-05-04T03:11:40.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.243+0000 7f254cff9700 1 -- 172.21.15.39:0/2096692655 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548008b90 con 0x562188406da0 2024-05-04T03:11:40.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 -- 172.21.15.39:0/2096692655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188406da0 msgr2=0x5621884071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188406da0 0x5621884071b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2548003a00 tx=0x7f2548004b40 comp rx=0 tx=0).stop 2024-05-04T03:11:40.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 -- 172.21.15.39:0/2096692655 shutdown_connections 2024-05-04T03:11:40.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188406da0 0x5621884071b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/2096692655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 0x562188401300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 -- 172.21.15.39:0/2096692655 >> 172.21.15.39:0/2096692655 conn(0x562188283df0 msgr2=0x5621883fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 -- 172.21.15.39:0/2096692655 shutdown_connections 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 -- 172.21.15.39:0/2096692655 wait complete. 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 -- start start 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.244+0000 7f256b8c4b80 1 Processor -- start 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188400ef0 0x56218840aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56218840e170 con 0x562188400ef0 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562188289be0 con 0x562188406da0 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56218840e2f0 con 0x562188401840 2024-05-04T03:11:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f256b8c4b80 10 client.0 _ll_register_callbacks cb 0x5621882bb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f255659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188400ef0 0x56218840aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f255659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188400ef0 0x56218840aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39588/0 (socket says 172.21.15.39:39588) 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f255659c700 1 -- 172.21.15.39:0/3369791447 learned_addr learned my addr 172.21.15.39:0/3369791447 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f2556d9d700 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f2555d9b700 1 -- 172.21.15.39:0/3369791447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 msgr2=0x56218840b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f2555d9b700 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f2555d9b700 1 -- 172.21.15.39:0/3369791447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188400ef0 msgr2=0x56218840aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f2555d9b700 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188400ef0 0x56218840aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f2555d9b700 1 -- 172.21.15.39:0/3369791447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621882bf640 con 0x562188401840 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f2555d9b700 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840b410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25400dfbc0 tx=0x7f25400a91e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a407aae356c3587 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.245+0000 7f253dffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:11:40.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f253dffb700 1 -- 172.21.15.39:0/3369791447 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254006d070 con 0x562188401840 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3369791447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2548004800 con 0x562188401840 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f253dffb700 1 -- 172.21.15.39:0/3369791447 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f25400b0e00 con 0x562188401840 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f253dffb700 1 -- 172.21.15.39:0/3369791447 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25400a9dd0 con 0x562188401840 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 10 client.25075 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3369791447 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562188418e20 con 0x562188401840 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f253dffb700 1 -- 172.21.15.39:0/3369791447 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2548004800 con 0x562188401840 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f253dffb700 1 -- 172.21.15.39:0/3369791447 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f25400b20a0 con 0x562188401840 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 10 client.25075 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 10 client.25075 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 20 client.25075 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x56218841b3d0 0x56218841c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 4 client.25075 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.246+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3369791447 --> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621883311f0 con 0x56218841b3d0 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.247+0000 7f255659c700 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x56218841b3d0 0x56218841c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.247+0000 7f255659c700 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x56218841b3d0 0x56218841c840 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23274bbe451ec647 server_cookie=b0c165ca8f00ce88 in_seq=0 out_seq=0 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.247+0000 7f253dffb700 10 client.25075 ms_handle_connect on v2:172.21.15.167:6836/2270685849 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f253dffb700 1 -- 172.21.15.39:0/3369791447 <== mds.2 v2:172.21.15.167:6836/2270685849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5621883311f0 con 0x56218841b3d0 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f253dffb700 10 client.25075 handle_command_reply: tid=0 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 client.25075 shutdown 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3369791447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 msgr2=0x56218840b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840b410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25400dfbc0 tx=0x7f25400a91e0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3369791447 shutdown_connections 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x56218841b3d0 0x56218841c840 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188400ef0 0x56218840aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f253dffb700 0 client.25075 ms_handle_reset on v2:172.21.15.167:6836/2270685849 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3369791447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188401840 0x56218840b410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3369791447 >> 172.21.15.39:0/3369791447 conn(0x562188283df0 msgr2=0x5621883ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:40.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3369791447 shutdown_connections 2024-05-04T03:11:40.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3369791447 wait complete. 2024-05-04T03:11:40.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.248+0000 7f256b8c4b80 20 client.25075 trim_cache size 0 max 16384 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.269+0000 7f256b8c4b80 1 Processor -- start 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.269+0000 7f256b8c4b80 1 -- start start 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188400ef0 0x562188417590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x56218840d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56218841ab80 con 0x562188401840 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56218841b250 con 0x562188406da0 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56218841b920 con 0x562188400ef0 2024-05-04T03:11:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x56218840d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x56218840d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39600/0 (socket says 172.21.15.39:39600) 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2555d9b700 1 -- 172.21.15.39:0/1734762598 learned_addr learned my addr 172.21.15.39:0/1734762598 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2556d9d700 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f255659c700 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188400ef0 0x562188417590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2555d9b700 1 -- 172.21.15.39:0/1734762598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188400ef0 msgr2=0x562188417590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2555d9b700 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188400ef0 0x562188417590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2555d9b700 1 -- 172.21.15.39:0/1734762598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 msgr2=0x56218840b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2555d9b700 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2555d9b700 1 -- 172.21.15.39:0/1734762598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218840d980 con 0x562188401840 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f255659c700 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188400ef0 0x562188417590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.270+0000 7f2556d9d700 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.271+0000 7f2555d9b700 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x56218840d440 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2540000c00 tx=0x7f25400a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b040f2572ecb5421 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.271+0000 7f253ffff700 1 -- 172.21.15.39:0/1734762598 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254006d070 con 0x562188401840 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.271+0000 7f253ffff700 1 -- 172.21.15.39:0/1734762598 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25400a96d0 con 0x562188401840 2024-05-04T03:11:40.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.271+0000 7f253ffff700 1 -- 172.21.15.39:0/1734762598 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25400ada80 con 0x562188401840 2024-05-04T03:11:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 -- 172.21.15.39:0/1734762598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 msgr2=0x56218840d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x56218840d440 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2540000c00 tx=0x7f25400a56c0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 -- 172.21.15.39:0/1734762598 shutdown_connections 2024-05-04T03:11:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188406da0 0x56218840b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188400ef0 0x562188417590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/1734762598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x56218840d440 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 -- 172.21.15.39:0/1734762598 >> 172.21.15.39:0/1734762598 conn(0x562188283df0 msgr2=0x5621883f5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 -- 172.21.15.39:0/1734762598 shutdown_connections 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 -- 172.21.15.39:0/1734762598 wait complete. 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.272+0000 7f256b8c4b80 1 -- start start 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f256b8c4b80 1 Processor -- start 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 0x562188425d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x5621884262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f256b8c4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188406da0 0x562188420d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56218841c6c0 con 0x562188401840 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56218841bff0 con 0x562188400ef0 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f256b8c4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56218841cd90 con 0x562188406da0 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x5621884262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f256b8c4b80 10 client.0 _ll_register_callbacks cb 0x5621882bb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2555d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x5621884262c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39608/0 (socket says 172.21.15.39:39608) 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2555d9b700 1 -- 172.21.15.39:0/3100270936 learned_addr learned my addr 172.21.15.39:0/3100270936 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2556d9d700 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188406da0 0x562188420d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f255659c700 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 0x562188425d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2555d9b700 1 -- 172.21.15.39:0/3100270936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188406da0 msgr2=0x562188420d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2555d9b700 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188406da0 0x562188420d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2555d9b700 1 -- 172.21.15.39:0/3100270936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 msgr2=0x562188425d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2555d9b700 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 0x562188425d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.273+0000 7f2555d9b700 1 -- 172.21.15.39:0/3100270936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562188421370 con 0x562188401840 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f2555d9b700 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x5621884262c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f25400dfbc0 tx=0x7f254000f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d87a651f376922 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f253dffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f253dffb700 1 -- 172.21.15.39:0/3100270936 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254006d070 con 0x562188401840 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3100270936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25400a96d0 con 0x562188401840 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f253dffb700 1 -- 172.21.15.39:0/3100270936 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f25400b0e00 con 0x562188401840 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f253dffb700 1 -- 172.21.15.39:0/3100270936 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25400a9b40 con 0x562188401840 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f256b8c4b80 10 client.15384 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3100270936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562188487160 con 0x562188401840 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f253dffb700 1 -- 172.21.15.39:0/3100270936 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25400a96d0 con 0x562188401840 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f253dffb700 1 -- 172.21.15.39:0/3100270936 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f25400a5040 con 0x562188401840 2024-05-04T03:11:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.274+0000 7f256b8c4b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f256b8c4b80 10 client.15384 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f256b8c4b80 20 client.15384 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x562188489050 0x56218848a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f256b8c4b80 4 client.15384 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3100270936 --> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] -- command(tid 0: {"prefix": "status"}) -- 0x56218832b290 con 0x562188489050 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f255659c700 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x562188489050 0x56218848a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f253dffb700 10 client.15384 ms_handle_connect on v2:172.21.15.167:6836/2270685849 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f255659c700 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x562188489050 0x56218848a4c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c65f2c6c0b417730 server_cookie=40823993491c2642 in_seq=0 out_seq=0 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f253dffb700 1 -- 172.21.15.39:0/3100270936 <== mds.2 v2:172.21.15.167:6836/2270685849 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56218832b290 con 0x562188489050 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f253dffb700 10 client.15384 handle_command_reply: tid=0 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.275+0000 7f256b8c4b80 1 client.15384 shutdown 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3100270936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 msgr2=0x5621884262c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x5621884262c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f25400dfbc0 tx=0x7f254000f8f0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3100270936 shutdown_connections 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.167:6836/2270685849,v1:172.21.15.167:6837/2270685849] conn(0x562188489050 0x56218848a4c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562188406da0 0x562188420d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562188400ef0 0x562188425d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 --2- 172.21.15.39:0/3100270936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562188401840 0x5621884262c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3100270936 >> 172.21.15.39:0/3100270936 conn(0x562188283df0 msgr2=0x5621883ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f253dffb700 0 client.15384 ms_handle_reset on v2:172.21.15.167:6836/2270685849 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3100270936 shutdown_connections 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 1 -- 172.21.15.39:0/3100270936 wait complete. 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f256b8c4b80 20 client.15384 trim_cache size 0 max 16384 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24472, 2024-05-04T03:11:40.281 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:11:40.282 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:11:40.282 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:11:40.282 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 225.14329718900001, 2024-05-04T03:11:40.282 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-05-04T03:11:40.282 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:11:40.282 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 80, 2024-05-04T03:11:40.282 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 276.24505119200001 2024-05-04T03:11:40.282 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f255ceb0700 1 -- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f25440b2910 msgr2=0x7f25440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f255ceb0700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f25440b2910 0x7f25440b4e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2548010860 tx=0x7f2548012040 comp rx=0 tx=0).stop 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f255ceb0700 1 -- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558107df0 msgr2=0x7f25581a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.276+0000 7f255ceb0700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558107df0 0x7f25581a0f00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f254000c9e0 tx=0x7f254000ccf0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.277+0000 7f255ceb0700 1 -- 172.21.15.39:0/360705896 shutdown_connections 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.277+0000 7f255ceb0700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f25440b2910 0x7f25440b4e20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.277+0000 7f255ceb0700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2558108740 0x7f255819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.277+0000 7f255ceb0700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25581034c0 0x7f25581a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.277+0000 7f255ceb0700 1 --2- 172.21.15.39:0/360705896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2558107df0 0x7f25581a0f00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.277+0000 7f255ceb0700 1 -- 172.21.15.39:0/360705896 >> 172.21.15.39:0/360705896 conn(0x7f25580fedc0 msgr2=0x7f2558101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.277+0000 7f255ceb0700 1 -- 172.21.15.39:0/360705896 shutdown_connections 2024-05-04T03:11:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:40.277+0000 7f255ceb0700 1 -- 172.21.15.39:0/360705896 wait complete. 2024-05-04T03:11:40.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:40 smithi167 ceph-mon[33614]: pgmap v412: 97 pgs: 97 active+clean; 28 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 71 op/s 2024-05-04T03:11:40.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:40 smithi066 ceph-mon[33264]: pgmap v412: 97 pgs: 97 active+clean; 28 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 71 op/s 2024-05-04T03:11:40.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-04T03:11:41.241 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:42.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:42 smithi167 ceph-mon[33614]: pgmap v413: 97 pgs: 97 active+clean; 33 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.7 MiB/s wr, 88 op/s 2024-05-04T03:11:42.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:42 smithi039 ceph-mon[29715]: pgmap v413: 97 pgs: 97 active+clean; 33 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.7 MiB/s wr, 88 op/s 2024-05-04T03:11:42.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:42 smithi066 ceph-mon[33264]: pgmap v413: 97 pgs: 97 active+clean; 33 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 3.7 MiB/s wr, 88 op/s 2024-05-04T03:11:43.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 -- 172.21.15.39:0/2460855433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d4107df0 msgr2=0x7fa0d4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 --2- 172.21.15.39:0/2460855433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d4107df0 0x7fa0d4108200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40099c0 tx=0x7fa0c4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 -- 172.21.15.39:0/2460855433 shutdown_connections 2024-05-04T03:11:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 --2- 172.21.15.39:0/2460855433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d4108740 0x7fa0d41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 --2- 172.21.15.39:0/2460855433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d41034c0 0x7fa0d41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 --2- 172.21.15.39:0/2460855433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d4107df0 0x7fa0d4108200 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 -- 172.21.15.39:0/2460855433 >> 172.21.15.39:0/2460855433 conn(0x7fa0d40fedf0 msgr2=0x7fa0d4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 -- 172.21.15.39:0/2460855433 shutdown_connections 2024-05-04T03:11:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 -- 172.21.15.39:0/2460855433 wait complete. 2024-05-04T03:11:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.052+0000 7fa0d975b700 1 Processor -- start 2024-05-04T03:11:43.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d975b700 1 -- start start 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d975b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d41034c0 0x7fa0d419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d975b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d4107df0 0x7fa0d419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d975b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d4108740 0x7fa0d419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d975b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4113200 con 0x7fa0d41034c0 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d975b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4113080 con 0x7fa0d4107df0 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d975b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4113380 con 0x7fa0d4108740 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d41034c0 0x7fa0d419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d41034c0 0x7fa0d419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39624/0 (socket says 172.21.15.39:39624) 2024-05-04T03:11:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d2ffd700 1 -- 172.21.15.39:0/1608225792 learned_addr learned my addr 172.21.15.39:0/1608225792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d4107df0 0x7fa0d419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d37fe700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d4108740 0x7fa0d419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d2ffd700 1 -- 172.21.15.39:0/1608225792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d4108740 msgr2=0x7fa0d419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d4108740 0x7fa0d419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d2ffd700 1 -- 172.21.15.39:0/1608225792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d4107df0 msgr2=0x7fa0d419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d4107df0 0x7fa0d419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.053+0000 7fa0d2ffd700 1 -- 172.21.15.39:0/1608225792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d419fc80 con 0x7fa0d41034c0 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.054+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d41034c0 0x7fa0d419ae60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00d950 tx=0x7fa0bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.054+0000 7fa0cbfff700 1 -- 172.21.15.39:0/1608225792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc016070 con 0x7fa0d41034c0 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.054+0000 7fa0d975b700 1 -- 172.21.15.39:0/1608225792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d419ff70 con 0x7fa0d41034c0 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.054+0000 7fa0cbfff700 1 -- 172.21.15.39:0/1608225792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0bc00fd20 con 0x7fa0d41034c0 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.054+0000 7fa0cbfff700 1 -- 172.21.15.39:0/1608225792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00e3b0 con 0x7fa0d41034c0 2024-05-04T03:11:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.054+0000 7fa0d975b700 1 -- 172.21.15.39:0/1608225792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d41a04d0 con 0x7fa0d41034c0 2024-05-04T03:11:43.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.056+0000 7fa0cbfff700 1 -- 172.21.15.39:0/1608225792 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fa0bc019070 con 0x7fa0d41034c0 2024-05-04T03:11:43.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.057+0000 7fa0cbfff700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0c00b2910 0x7fa0c00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.057+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0c00b2910 0x7fa0c00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.057+0000 7fa0cbfff700 1 -- 172.21.15.39:0/1608225792 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa0bc0e1ea0 con 0x7fa0d41034c0 2024-05-04T03:11:43.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0c00b2910 0x7fa0c00b4e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009990 tx=0x7fa0c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0e816fb80 1 Processor -- start 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0e816fb80 1 -- start start 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 0x55688fcce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcc8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55688fbea810 con 0x55688fcc7ef0 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55688fb89770 con 0x55688fccdda0 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55688fbe67d0 con 0x55688fcc8840 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.058+0000 7fa0d27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcc8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcc8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39634/0 (socket says 172.21.15.39:39634) 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d27fc700 1 -- 172.21.15.39:0/1975220360 learned_addr learned my addr 172.21.15.39:0/1975220360 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 0x55688fcce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d37fe700 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d27fc700 1 -- 172.21.15.39:0/1975220360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 msgr2=0x55688fcd49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d27fc700 1 -- 172.21.15.39:0/1975220360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 msgr2=0x55688fcce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 0x55688fcce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d27fc700 1 -- 172.21.15.39:0/1975220360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688fb37880 con 0x55688fcc7ef0 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.059+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcc8300 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009750 tx=0x7fa0c401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8d524dc5f86799 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.060+0000 7fa0c97fa700 1 -- 172.21.15.39:0/1975220360 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400be50 con 0x55688fcc7ef0 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.060+0000 7fa0c97fa700 1 -- 172.21.15.39:0/1975220360 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0c4005690 con 0x55688fcc7ef0 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.060+0000 7fa0c97fa700 1 -- 172.21.15.39:0/1975220360 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400f950 con 0x55688fcc7ef0 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 -- 172.21.15.39:0/1975220360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 msgr2=0x55688fcc8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcc8300 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009750 tx=0x7fa0c401e000 comp rx=0 tx=0).stop 2024-05-04T03:11:43.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 -- 172.21.15.39:0/1975220360 shutdown_connections 2024-05-04T03:11:43.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 0x55688fcce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/1975220360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcc8300 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 -- 172.21.15.39:0/1975220360 >> 172.21.15.39:0/1975220360 conn(0x55688fb4adf0 msgr2=0x55688fcc2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:43.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 -- 172.21.15.39:0/1975220360 shutdown_connections 2024-05-04T03:11:43.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 -- 172.21.15.39:0/1975220360 wait complete. 2024-05-04T03:11:43.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 -- start start 2024-05-04T03:11:43.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.061+0000 7fa0e816fb80 1 Processor -- start 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcd1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 0x55688fcd28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55688fcd5110 con 0x55688fcc7ef0 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55688fb50be0 con 0x55688fccdda0 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55688fcd5290 con 0x55688fcc8840 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0e816fb80 10 client.0 _ll_register_callbacks cb 0x55688fb828d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcd1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcd1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39646/0 (socket says 172.21.15.39:39646) 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d2ffd700 1 -- 172.21.15.39:0/603157153 learned_addr learned my addr 172.21.15.39:0/603157153 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d37fe700 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 0x55688fcd28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d2ffd700 1 -- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 msgr2=0x55688fcd23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d2ffd700 1 -- 172.21.15.39:0/603157153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 msgr2=0x55688fcd28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 0x55688fcd28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d2ffd700 1 -- 172.21.15.39:0/603157153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688fb86640 con 0x55688fcc7ef0 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.062+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcd1e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0a9070 tx=0x7fa0bc0dfce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5091eac7f5d0b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/603157153 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc020040 con 0x55688fcc7ef0 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/603157153 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa0bc0b1d40 con 0x55688fcc7ef0 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0e816fb80 1 -- 172.21.15.39:0/603157153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c4005690 con 0x55688fcc7ef0 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/603157153 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0bc0ad720 con 0x55688fcc7ef0 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0e816fb80 10 client.15396 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:43.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0e816fb80 1 -- 172.21.15.39:0/603157153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55688fcdf280 con 0x55688fcc7ef0 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/603157153 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0c4005690 con 0x55688fcc7ef0 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/603157153 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa0bc0a85d0 con 0x55688fcc7ef0 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.063+0000 7fa0e816fb80 10 client.15396 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.064+0000 7fa0e816fb80 10 client.15396 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.064+0000 7fa0e816fb80 20 client.15396 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.064+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55688fce18f0 0x55688fce2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.064+0000 7fa0e816fb80 4 client.15396 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.064+0000 7fa0e816fb80 1 -- 172.21.15.39:0/603157153 --> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55688fbf81f0 con 0x55688fce18f0 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.064+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55688fce18f0 0x55688fce2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.065+0000 7fa0ba7fc700 10 client.15396 ms_handle_connect on v2:172.21.15.167:6838/975309166 2024-05-04T03:11:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.065+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55688fce18f0 0x55688fce2d60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82bda396d5899ea7 server_cookie=5f4e26608a3c2366 in_seq=0 out_seq=0 2024-05-04T03:11:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.065+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/603157153 <== mds.1 v2:172.21.15.167:6838/975309166 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55688fbf81f0 con 0x55688fce18f0 2024-05-04T03:11:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.065+0000 7fa0ba7fc700 10 client.15396 handle_command_reply: tid=0 2024-05-04T03:11:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.065+0000 7fa0e816fb80 1 client.15396 shutdown 2024-05-04T03:11:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 -- 172.21.15.39:0/603157153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 msgr2=0x55688fcd1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcd1e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0a9070 tx=0x7fa0bc0dfce0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 -- 172.21.15.39:0/603157153 shutdown_connections 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55688fce18f0 0x55688fce2d60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc7ef0 0x55688fcd1e70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fccdda0 0x55688fcd28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0ba7fc700 0 client.15396 ms_handle_reset on v2:172.21.15.167:6838/975309166 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/603157153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fcd23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 -- 172.21.15.39:0/603157153 >> 172.21.15.39:0/603157153 conn(0x55688fb4adf0 msgr2=0x55688fcc6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 -- 172.21.15.39:0/603157153 shutdown_connections 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 1 -- 172.21.15.39:0/603157153 wait complete. 2024-05-04T03:11:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.066+0000 7fa0e816fb80 20 client.15396 trim_cache size 0 max 16384 2024-05-04T03:11:43.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0e816fb80 1 Processor -- start 2024-05-04T03:11:43.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0e816fb80 1 -- start start 2024-05-04T03:11:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 0x55688fcde590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fccf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fccdda0 0x55688fcd2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55688fce10a0 con 0x55688fccdda0 2024-05-04T03:11:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55688fce1770 con 0x55688fcc7ef0 2024-05-04T03:11:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55688fce1e40 con 0x55688fcc8840 2024-05-04T03:11:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0d37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fccdda0 0x55688fcd2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0d37fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fccdda0 0x55688fcd2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39660/0 (socket says 172.21.15.39:39660) 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.087+0000 7fa0d37fe700 1 -- 172.21.15.39:0/796181140 learned_addr learned my addr 172.21.15.39:0/796181140 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 0x55688fcde590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fccf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0d37fe700 1 -- 172.21.15.39:0/796181140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 msgr2=0x55688fccf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0d37fe700 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fccf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0d37fe700 1 -- 172.21.15.39:0/796181140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 msgr2=0x55688fcde590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0d37fe700 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 0x55688fcde590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0d37fe700 1 -- 172.21.15.39:0/796181140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688fcbf470 con 0x55688fccdda0 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0d37fe700 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fccdda0 0x55688fcd2730 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc014700 tx=0x7fa0cc007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4e9b2ae82140b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0c8ff9700 1 -- 172.21.15.39:0/796181140 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc016630 con 0x55688fccdda0 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0c8ff9700 1 -- 172.21.15.39:0/796181140 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0cc016c60 con 0x55688fccdda0 2024-05-04T03:11:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.088+0000 7fa0c8ff9700 1 -- 172.21.15.39:0/796181140 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc01ed40 con 0x55688fccdda0 2024-05-04T03:11:43.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 -- 172.21.15.39:0/796181140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fccdda0 msgr2=0x55688fcd2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fccdda0 0x55688fcd2730 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc014700 tx=0x7fa0cc007950 comp rx=0 tx=0).stop 2024-05-04T03:11:43.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 -- 172.21.15.39:0/796181140 shutdown_connections 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fccdda0 0x55688fcd2730 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 0x55688fcde590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/796181140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fcc8840 0x55688fccf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 -- 172.21.15.39:0/796181140 >> 172.21.15.39:0/796181140 conn(0x55688fb4adf0 msgr2=0x55688fcbca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 -- 172.21.15.39:0/796181140 shutdown_connections 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.089+0000 7fa0e816fb80 1 -- 172.21.15.39:0/796181140 wait complete. 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 1 -- start start 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 1 Processor -- start 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 0x55688fcecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc8840 0x55688fced280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fccdda0 0x55688fce7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55688fce2be0 con 0x55688fcc8840 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55688fce2510 con 0x55688fcc7ef0 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55688fce32b0 con 0x55688fccdda0 2024-05-04T03:11:43.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0e816fb80 10 client.0 _ll_register_callbacks cb 0x55688fb828d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc8840 0x55688fced280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d27fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc8840 0x55688fced280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39666/0 (socket says 172.21.15.39:39666) 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d27fc700 1 -- 172.21.15.39:0/556747251 learned_addr learned my addr 172.21.15.39:0/556747251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 0x55688fcecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d37fe700 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fccdda0 0x55688fce7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d27fc700 1 -- 172.21.15.39:0/556747251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fccdda0 msgr2=0x55688fce7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fccdda0 0x55688fce7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d27fc700 1 -- 172.21.15.39:0/556747251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 msgr2=0x55688fcecd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 0x55688fcecd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.090+0000 7fa0d27fc700 1 -- 172.21.15.39:0/556747251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688fce8470 con 0x55688fcc8840 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0d27fc700 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc8840 0x55688fced280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55688fcc43f0 tx=0x7fa0c4005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506c18310fa16a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/556747251 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400f670 con 0x55688fcc8840 2024-05-04T03:11:43.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/556747251 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa0c402b040 con 0x55688fcc8840 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0e816fb80 1 -- 172.21.15.39:0/556747251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc016c60 con 0x55688fcc8840 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/556747251 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0c400f810 con 0x55688fcc8840 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0e816fb80 10 client.15402 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0e816fb80 1 -- 172.21.15.39:0/556747251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55688fd4e2a0 con 0x55688fcc8840 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/556747251 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0cc016c60 con 0x55688fcc8840 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/556747251 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa0c4022070 con 0x55688fcc8840 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.091+0000 7fa0e816fb80 10 client.15402 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0e816fb80 10 client.15402 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0e816fb80 20 client.15402 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55688fce18f0 0x55688fd515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0e816fb80 4 client.15402 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0e816fb80 1 -- 172.21.15.39:0/556747251 --> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] -- command(tid 0: {"prefix": "status"}) -- 0x55688fbf2290 con 0x55688fce18f0 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55688fce18f0 0x55688fd515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:43.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0ba7fc700 10 client.15402 ms_handle_connect on v2:172.21.15.167:6838/975309166 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0d2ffd700 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55688fce18f0 0x55688fd515f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1b08dacb58afcd1 server_cookie=23d16d22056299de in_seq=0 out_seq=0 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0ba7fc700 1 -- 172.21.15.39:0/556747251 <== mds.1 v2:172.21.15.167:6838/975309166 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55688fbf2290 con 0x55688fce18f0 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0ba7fc700 10 client.15402 handle_command_reply: tid=0 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.092+0000 7fa0e816fb80 1 client.15402 shutdown 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 -- 172.21.15.39:0/556747251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc8840 msgr2=0x55688fced280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc8840 0x55688fced280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55688fcc43f0 tx=0x7fa0c4005ed0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 -- 172.21.15.39:0/556747251 shutdown_connections 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.167:6838/975309166,v1:172.21.15.167:6839/975309166] conn(0x55688fce18f0 0x55688fd515f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55688fccdda0 0x55688fce7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55688fcc7ef0 0x55688fcecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 --2- 172.21.15.39:0/556747251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55688fcc8840 0x55688fced280 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 -- 172.21.15.39:0/556747251 >> 172.21.15.39:0/556747251 conn(0x55688fb4adf0 msgr2=0x55688fcc6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0ba7fc700 0 client.15402 ms_handle_reset on v2:172.21.15.167:6838/975309166 2024-05-04T03:11:43.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 -- 172.21.15.39:0/556747251 shutdown_connections 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 1 -- 172.21.15.39:0/556747251 wait complete. 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0e816fb80 20 client.15402 trim_cache size 0 max 16384 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24490, 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:11:43.099 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 228.96885989699999, 2024-05-04T03:11:43.100 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-05-04T03:11:43.100 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:11:43.100 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 80, 2024-05-04T03:11:43.100 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 273.43200610299999 2024-05-04T03:11:43.100 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:11:43.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 -- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0c00b2910 msgr2=0x7fa0c00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0c00b2910 0x7fa0c00b4e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009990 tx=0x7fa0c4005fb0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 -- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d41034c0 msgr2=0x7fa0d419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d41034c0 0x7fa0d419ae60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00d950 tx=0x7fa0bc00dc60 comp rx=0 tx=0).stop 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 -- 172.21.15.39:0/1608225792 shutdown_connections 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0c00b2910 0x7fa0c00b4e20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0d41034c0 0x7fa0d419ae60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0d4108740 0x7fa0d419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 --2- 172.21.15.39:0/1608225792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0d4107df0 0x7fa0d419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.093+0000 7fa0d975b700 1 -- 172.21.15.39:0/1608225792 >> 172.21.15.39:0/1608225792 conn(0x7fa0d40fedf0 msgr2=0x7fa0d4075ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.094+0000 7fa0d975b700 1 -- 172.21.15.39:0/1608225792 shutdown_connections 2024-05-04T03:11:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:43.094+0000 7fa0d975b700 1 -- 172.21.15.39:0/1608225792 wait complete. 2024-05-04T03:11:43.939 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-04T03:11:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:11:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:44 smithi167 ceph-mon[33614]: pgmap v414: 97 pgs: 97 active+clean; 37 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 3.4 MiB/s wr, 100 op/s 2024-05-04T03:11:44.440 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:44 smithi039 ceph-mon[29715]: pgmap v414: 97 pgs: 97 active+clean; 37 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 3.4 MiB/s wr, 100 op/s 2024-05-04T03:11:44.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:44 smithi066 ceph-mon[33264]: pgmap v414: 97 pgs: 97 active+clean; 37 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 3.4 MiB/s wr, 100 op/s 2024-05-04T03:11:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 -- 172.21.15.39:0/1543888335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc05010e8d0 msgr2=0x7fc050110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 --2- 172.21.15.39:0/1543888335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc05010e8d0 0x7fc050110de0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc04800b0a0 tx=0x7fc04800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 -- 172.21.15.39:0/1543888335 shutdown_connections 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 --2- 172.21.15.39:0/1543888335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc05010e8d0 0x7fc050110de0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 --2- 172.21.15.39:0/1543888335 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc050105460 0x7fc050105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 --2- 172.21.15.39:0/1543888335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc050105db0 0x7fc05010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 -- 172.21.15.39:0/1543888335 >> 172.21.15.39:0/1543888335 conn(0x7fc0500fedf0 msgr2=0x7fc050101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 -- 172.21.15.39:0/1543888335 shutdown_connections 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.950+0000 7fc0560f8700 1 -- 172.21.15.39:0/1543888335 wait complete. 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.951+0000 7fc0560f8700 1 Processor -- start 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.951+0000 7fc0560f8700 1 -- start start 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc0560f8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc050105460 0x7fc0501a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc0560f8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc050105db0 0x7fc0501a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc0560f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05010e8d0 0x7fc05019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc0560f8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc050114330 con 0x7fc050105db0 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc0560f8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0501141b0 con 0x7fc050105460 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc0560f8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0501144b0 con 0x7fc05010e8d0 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc050105db0 0x7fc0501a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05010e8d0 0x7fc05019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc050105460 0x7fc0501a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc050105db0 0x7fc0501a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39214/0 (socket says 172.21.15.39:39214) 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04effd700 1 -- 172.21.15.39:0/2673134338 learned_addr learned my addr 172.21.15.39:0/2673134338 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04ffff700 1 -- 172.21.15.39:0/2673134338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc050105460 msgr2=0x7fc0501a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04ffff700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc050105460 0x7fc0501a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04ffff700 1 -- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc050105db0 msgr2=0x7fc0501a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04ffff700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc050105db0 0x7fc0501a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.952+0000 7fc04ffff700 1 -- 172.21.15.39:0/2673134338 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05019b260 con 0x7fc05010e8d0 2024-05-04T03:11:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.953+0000 7fc04ffff700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05010e8d0 0x7fc05019ab30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc04800b0a0 tx=0x7fc048004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.953+0000 7fc04cff9700 1 -- 172.21.15.39:0/2673134338 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048022070 con 0x7fc05010e8d0 2024-05-04T03:11:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.953+0000 7fc04cff9700 1 -- 172.21.15.39:0/2673134338 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc048007b80 con 0x7fc05010e8d0 2024-05-04T03:11:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.953+0000 7fc04cff9700 1 -- 172.21.15.39:0/2673134338 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0480052a0 con 0x7fc05010e8d0 2024-05-04T03:11:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.953+0000 7fc0560f8700 1 -- 172.21.15.39:0/2673134338 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05019b4f0 con 0x7fc05010e8d0 2024-05-04T03:11:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.953+0000 7fc0560f8700 1 -- 172.21.15.39:0/2673134338 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05019b920 con 0x7fc05010e8d0 2024-05-04T03:11:45.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.957+0000 7fc04cff9700 1 -- 172.21.15.39:0/2673134338 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fc04802d020 con 0x7fc05010e8d0 2024-05-04T03:11:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.957+0000 7fc04cff9700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0380b2910 0x7fc0380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0380b2910 0x7fc0380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc04cff9700 1 -- 172.21.15.39:0/2673134338 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc04800f020 con 0x7fc05010e8d0 2024-05-04T03:11:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc064b0cb80 1 Processor -- start 2024-05-04T03:11:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc064b0cb80 1 -- start start 2024-05-04T03:11:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e669d0d0 0x5643e669d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 0x5643e66a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5643e65b98f0 con 0x5643e669d0d0 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.958+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5643e651ed50 con 0x5643e66971c0 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5643e65b58b0 con 0x5643e6697b10 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0380b2910 0x7fc0380b4e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc05006d150 tx=0x7fc040009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e669d0d0 0x5643e669d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e669d0d0 0x5643e669d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39226/0 (socket says 172.21.15.39:39226) 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:57346/0 (socket says 172.21.15.39:57346) 2024-05-04T03:11:45.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04f7fe700 1 -- 172.21.15.39:0/602714764 learned_addr learned my addr 172.21.15.39:0/602714764 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04ffff700 1 --2- 172.21.15.39:0/602714764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 0x5643e66a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04effd700 1 -- 172.21.15.39:0/602714764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 msgr2=0x5643e66a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04effd700 1 --2- 172.21.15.39:0/602714764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 0x5643e66a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04effd700 1 -- 172.21.15.39:0/602714764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e669d0d0 msgr2=0x5643e669d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04effd700 1 --2- 172.21.15.39:0/602714764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e669d0d0 0x5643e669d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04effd700 1 -- 172.21.15.39:0/602714764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e6506880 con 0x5643e66971c0 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.959+0000 7fc04effd700 1 --2- 172.21.15.39:0/602714764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66975d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc05019c260 tx=0x7fc044009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff57f435d16b672 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.960+0000 7fc035ffb700 1 -- 172.21.15.39:0/602714764 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044011ce0 con 0x5643e66971c0 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.960+0000 7fc035ffb700 1 -- 172.21.15.39:0/602714764 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0440044e0 con 0x5643e66971c0 2024-05-04T03:11:45.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.960+0000 7fc035ffb700 1 -- 172.21.15.39:0/602714764 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0440193e0 con 0x5643e66971c0 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 -- 172.21.15.39:0/602714764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 msgr2=0x5643e66975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/602714764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66975d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc05019c260 tx=0x7fc044009f70 comp rx=0 tx=0).stop 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 -- 172.21.15.39:0/602714764 shutdown_connections 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/602714764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e669d0d0 0x5643e669d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/602714764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 0x5643e66a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/602714764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66975d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 -- 172.21.15.39:0/602714764 >> 172.21.15.39:0/602714764 conn(0x5643e6519df0 msgr2=0x5643e6692210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 -- 172.21.15.39:0/602714764 shutdown_connections 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.961+0000 7fc064b0cb80 1 -- 172.21.15.39:0/602714764 wait complete. 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.962+0000 7fc064b0cb80 1 -- start start 2024-05-04T03:11:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.962+0000 7fc064b0cb80 1 Processor -- start 2024-05-04T03:11:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.962+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.962+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 0x5643e66a1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.962+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e669d0d0 0x5643e66a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.962+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5643e66a44a0 con 0x5643e66971c0 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.962+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5643e6572e80 con 0x5643e669d0d0 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.962+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5643e66a4620 con 0x5643e6697b10 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc064b0cb80 10 client.0 _ll_register_callbacks cb 0x5643e6551900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39242/0 (socket says 172.21.15.39:39242) 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04f7fe700 1 -- 172.21.15.39:0/2431520302 learned_addr learned my addr 172.21.15.39:0/2431520302 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e669d0d0 0x5643e66a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04effd700 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 0x5643e66a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04ffff700 1 -- 172.21.15.39:0/2431520302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 msgr2=0x5643e66a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04ffff700 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 0x5643e66a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04ffff700 1 -- 172.21.15.39:0/2431520302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 msgr2=0x5643e66a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04ffff700 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04ffff700 1 -- 172.21.15.39:0/2431520302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e6570710 con 0x5643e669d0d0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc04ffff700 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e669d0d0 0x5643e66a1cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0480e4840 tx=0x7fc048004f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8204afcefb540fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc02effd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.963+0000 7fc02effd700 1 -- 172.21.15.39:0/2431520302 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0480ae3e0 con 0x5643e669d0d0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.964+0000 7fc02effd700 1 -- 172.21.15.39:0/2431520302 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc0480ae640 con 0x5643e669d0d0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.964+0000 7fc064b0cb80 1 -- 172.21.15.39:0/2431520302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0440044e0 con 0x5643e669d0d0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.964+0000 7fc02effd700 1 -- 172.21.15.39:0/2431520302 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc0480b2790 con 0x5643e669d0d0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.964+0000 7fc064b0cb80 10 client.25109 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.964+0000 7fc064b0cb80 1 -- 172.21.15.39:0/2431520302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5643e66af050 con 0x5643e669d0d0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.964+0000 7fc02effd700 1 -- 172.21.15.39:0/2431520302 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0440044e0 con 0x5643e669d0d0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.964+0000 7fc02effd700 1 -- 172.21.15.39:0/2431520302 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc048011070 con 0x5643e669d0d0 2024-05-04T03:11:45.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.964+0000 7fc064b0cb80 10 client.25109 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.965+0000 7fc064b0cb80 10 client.25109 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.965+0000 7fc064b0cb80 20 client.25109 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.965+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5643e66b1630 0x5643e66b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.965+0000 7fc064b0cb80 4 client.25109 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.965+0000 7fc064b0cb80 1 -- 172.21.15.39:0/2431520302 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643e65c72d0 con 0x5643e66b1630 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.965+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5643e66b1630 0x5643e66b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.965+0000 7fc02effd700 10 client.25109 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.965+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5643e66b1630 0x5643e66b2aa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccdde8e2c6549b9a server_cookie=90c625d0bea34176 in_seq=0 out_seq=0 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.966+0000 7fc02effd700 1 -- 172.21.15.39:0/2431520302 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5643e65c72d0 con 0x5643e66b1630 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.966+0000 7fc02effd700 10 client.25109 handle_command_reply: tid=0 2024-05-04T03:11:45.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.966+0000 7fc064b0cb80 1 client.25109 shutdown 2024-05-04T03:11:45.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 -- 172.21.15.39:0/2431520302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e669d0d0 msgr2=0x5643e66a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:45.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e669d0d0 0x5643e66a1cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0480e4840 tx=0x7fc048004f30 comp rx=0 tx=0).stop 2024-05-04T03:11:45.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 -- 172.21.15.39:0/2431520302 shutdown_connections 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5643e66b1630 0x5643e66b2aa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc02effd700 0 client.25109 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e669d0d0 0x5643e66a1cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/2431520302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e6697b10 0x5643e66a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 -- 172.21.15.39:0/2431520302 >> 172.21.15.39:0/2431520302 conn(0x5643e6519df0 msgr2=0x5643e66961b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 -- 172.21.15.39:0/2431520302 shutdown_connections 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 1 -- 172.21.15.39:0/2431520302 wait complete. 2024-05-04T03:11:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.967+0000 7fc064b0cb80 20 client.25109 trim_cache size 0 max 16384 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc064b0cb80 1 Processor -- start 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc064b0cb80 1 -- start start 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e6697b10 0x5643e668c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e669ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5643e66b0de0 con 0x5643e6697b10 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5643e66b14b0 con 0x5643e66971c0 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5643e66b1b80 con 0x5643e669d0d0 2024-05-04T03:11:45.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc04effd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e6697b10 0x5643e668c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:57364/0 (socket says 172.21.15.39:57364) 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc04f7fe700 1 -- 172.21.15.39:0/4090561121 learned_addr learned my addr 172.21.15.39:0/4090561121 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.993+0000 7fc04ffff700 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e669ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc04f7fe700 1 -- 172.21.15.39:0/4090561121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 msgr2=0x5643e669ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e669ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc04f7fe700 1 -- 172.21.15.39:0/4090561121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e6697b10 msgr2=0x5643e668c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e6697b10 0x5643e668c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc04f7fe700 1 -- 172.21.15.39:0/4090561121 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e66a0560 con 0x5643e66971c0 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc04ffff700 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e669ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:45.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66ad8c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc04001d490 tx=0x7fc040014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9fb9b58833d03a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:45.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc0357fa700 1 -- 172.21.15.39:0/4090561121 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040031070 con 0x5643e66971c0 2024-05-04T03:11:45.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc0357fa700 1 -- 172.21.15.39:0/4090561121 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc040003680 con 0x5643e66971c0 2024-05-04T03:11:45.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.994+0000 7fc0357fa700 1 -- 172.21.15.39:0/4090561121 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04002c3c0 con 0x5643e66971c0 2024-05-04T03:11:45.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.995+0000 7fc064b0cb80 1 -- 172.21.15.39:0/4090561121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 msgr2=0x5643e66ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:45.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.995+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66ad8c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc04001d490 tx=0x7fc040014000 comp rx=0 tx=0).stop 2024-05-04T03:11:45.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.995+0000 7fc064b0cb80 1 -- 172.21.15.39:0/4090561121 shutdown_connections 2024-05-04T03:11:45.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.995+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e669ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.995+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e66971c0 0x5643e66ad8c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:45.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.995+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/4090561121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e6697b10 0x5643e668c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.995+0000 7fc064b0cb80 1 -- 172.21.15.39:0/4090561121 >> 172.21.15.39:0/4090561121 conn(0x5643e6519df0 msgr2=0x5643e66a28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.996+0000 7fc064b0cb80 1 -- 172.21.15.39:0/4090561121 shutdown_connections 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.996+0000 7fc064b0cb80 1 -- 172.21.15.39:0/4090561121 wait complete. 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.996+0000 7fc064b0cb80 1 -- start start 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.996+0000 7fc064b0cb80 1 Processor -- start 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e6697b10 0x5643e66a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc064b0cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e66ba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5643e66b2920 con 0x5643e66971c0 2024-05-04T03:11:46.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5643e66b2250 con 0x5643e6697b10 2024-05-04T03:11:46.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc064b0cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5643e66b2ff0 con 0x5643e669d0d0 2024-05-04T03:11:46.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc064b0cb80 10 client.0 _ll_register_callbacks cb 0x5643e6551900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:46.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e66ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e66ba720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:58562/0 (socket says 172.21.15.39:58562) 2024-05-04T03:11:46.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e6697b10 0x5643e66a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04ffff700 1 -- 172.21.15.39:0/3130175326 learned_addr learned my addr 172.21.15.39:0/3130175326 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04ffff700 1 -- 172.21.15.39:0/3130175326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e6697b10 msgr2=0x5643e66a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04ffff700 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e6697b10 0x5643e66a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04ffff700 1 -- 172.21.15.39:0/3130175326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 msgr2=0x5643e66a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04ffff700 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.997+0000 7fc04ffff700 1 -- 172.21.15.39:0/3130175326 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e66a2300 con 0x5643e669d0d0 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc04ffff700 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e66ba720 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0480ae370 tx=0x7fc0480095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a12014a0002666 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc02effd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc02effd700 1 -- 172.21.15.39:0/3130175326 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0480ae3e0 con 0x5643e669d0d0 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc02effd700 1 -- 172.21.15.39:0/3130175326 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc0480a9d50 con 0x5643e669d0d0 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc064b0cb80 1 -- 172.21.15.39:0/3130175326 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc040003680 con 0x5643e669d0d0 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc02effd700 1 -- 172.21.15.39:0/3130175326 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc0480b71b0 con 0x5643e669d0d0 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc064b0cb80 10 client.25114 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:46.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc064b0cb80 1 -- 172.21.15.39:0/3130175326 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5643e671d120 con 0x5643e669d0d0 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc02effd700 1 -- 172.21.15.39:0/3130175326 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc040003680 con 0x5643e669d0d0 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.998+0000 7fc02effd700 1 -- 172.21.15.39:0/3130175326 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc048011040 con 0x5643e669d0d0 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc064b0cb80 10 client.25114 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc064b0cb80 10 client.25114 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc064b0cb80 20 client.25114 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5643e66b1630 0x5643e6720880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc064b0cb80 4 client.25114 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc064b0cb80 1 -- 172.21.15.39:0/3130175326 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "status"}) -- 0x5643e65c1370 con 0x5643e66b1630 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5643e66b1630 0x5643e6720880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc02effd700 10 client.25114 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:45.999+0000 7fc04f7fe700 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5643e66b1630 0x5643e6720880 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e4d14f49408aab server_cookie=c1acb2445b27aace in_seq=0 out_seq=0 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc02effd700 1 -- 172.21.15.39:0/3130175326 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5643e65c1370 con 0x5643e66b1630 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc02effd700 10 client.25114 handle_command_reply: tid=0 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 client.25114 shutdown 2024-05-04T03:11:46.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 -- 172.21.15.39:0/3130175326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 msgr2=0x5643e66ba720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e66ba720 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0480ae370 tx=0x7fc0480095a0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 -- 172.21.15.39:0/3130175326 shutdown_connections 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5643e66b1630 0x5643e6720880 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643e66971c0 0x5643e66a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643e669d0d0 0x5643e66ba720 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 --2- 172.21.15.39:0/3130175326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5643e6697b10 0x5643e66a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 -- 172.21.15.39:0/3130175326 >> 172.21.15.39:0/3130175326 conn(0x5643e6519df0 msgr2=0x5643e66a2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc02effd700 0 client.25114 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 -- 172.21.15.39:0/3130175326 shutdown_connections 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 1 -- 172.21.15.39:0/3130175326 wait complete. 2024-05-04T03:11:46.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.000+0000 7fc064b0cb80 20 client.25114 trim_cache size 0 max 16384 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "1e913ae0-09c2-11ef-bc97-c7b262605968", 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24506, 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 245.932269835, 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 80, 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 80, 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 264.784732228 2024-05-04T03:11:46.010 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 -- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0380b2910 msgr2=0x7fc0380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0380b2910 0x7fc0380b4e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc05006d150 tx=0x7fc040009260 comp rx=0 tx=0).stop 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 -- 172.21.15.39:0/2673134338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05010e8d0 msgr2=0x7fc05019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05010e8d0 0x7fc05019ab30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc04800b0a0 tx=0x7fc048004d00 comp rx=0 tx=0).stop 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 -- 172.21.15.39:0/2673134338 shutdown_connections 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0380b2910 0x7fc0380b4e20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc05010e8d0 0x7fc05019ab30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc050105460 0x7fc0501a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 --2- 172.21.15.39:0/2673134338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc050105db0 0x7fc0501a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 -- 172.21.15.39:0/2673134338 >> 172.21.15.39:0/2673134338 conn(0x7fc0500fedf0 msgr2=0x7fc050100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 -- 172.21.15.39:0/2673134338 shutdown_connections 2024-05-04T03:11:46.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:11:46.001+0000 7fc0560f8700 1 -- 172.21.15.39:0/2673134338 wait complete. 2024-05-04T03:11:46.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:46 smithi039 ceph-mon[29715]: pgmap v415: 97 pgs: 97 active+clean; 39 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 3.2 MiB/s wr, 101 op/s 2024-05-04T03:11:46.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:46 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:46.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:46 smithi066 ceph-mon[33264]: pgmap v415: 97 pgs: 97 active+clean; 39 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 3.2 MiB/s wr, 101 op/s 2024-05-04T03:11:46.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:46 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 -- 172.21.15.39:0/3701263000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9001045e0 msgr2=0x7fe90010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 --2- 172.21.15.39:0/3701263000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9001045e0 0x7fe90010ad30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009ca0 tx=0x7fe8f802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 -- 172.21.15.39:0/3701263000 shutdown_connections 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 --2- 172.21.15.39:0/3701263000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9001045e0 0x7fe90010ad30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 --2- 172.21.15.39:0/3701263000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe900103a80 0x7fe900103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 --2- 172.21.15.39:0/3701263000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9001030a0 0x7fe9001034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 -- 172.21.15.39:0/3701263000 >> 172.21.15.39:0/3701263000 conn(0x7fe9000fd4c0 msgr2=0x7fe9000ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 -- 172.21.15.39:0/3701263000 shutdown_connections 2024-05-04T03:11:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.581+0000 7fe904ac8640 1 -- 172.21.15.39:0/3701263000 wait complete. 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.582+0000 7fe904ac8640 1 Processor -- start 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.582+0000 7fe904ac8640 1 -- start start 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe904ac8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9001030a0 0x7fe900199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe904ac8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe900103a80 0x7fe90019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe904ac8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe90019f300 0x7fe9001a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe904ac8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010e5b0 con 0x7fe900103a80 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe904ac8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010e430 con 0x7fe9001030a0 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe904ac8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010e730 con 0x7fe90019f300 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe900103a80 0x7fe90019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8fed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe90019f300 0x7fe9001a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe900103a80 0x7fe90019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39278/0 (socket says 172.21.15.39:39278) 2024-05-04T03:11:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8f7fff640 1 -- 172.21.15.39:0/760892152 learned_addr learned my addr 172.21.15.39:0/760892152 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9001030a0 0x7fe900199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8fed9d640 1 -- 172.21.15.39:0/760892152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9001030a0 msgr2=0x7fe900199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8fed9d640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9001030a0 0x7fe900199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8fed9d640 1 -- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe900103a80 msgr2=0x7fe90019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.583+0000 7fe8fed9d640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe900103a80 0x7fe90019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.584+0000 7fe8fed9d640 1 -- 172.21.15.39:0/760892152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9001a1d90 con 0x7fe90019f300 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.584+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe900103a80 0x7fe90019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.584+0000 7fe8fed9d640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe90019f300 0x7fe9001a1720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8031920 tx=0x7fe8f8009870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.584+0000 7fe8f77fe640 1 -- 172.21.15.39:0/760892152 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8003bf0 con 0x7fe90019f300 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.584+0000 7fe904ac8640 1 -- 172.21.15.39:0/760892152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9001a2020 con 0x7fe90019f300 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.584+0000 7fe8f77fe640 1 -- 172.21.15.39:0/760892152 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8f8003d90 con 0x7fe90019f300 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.584+0000 7fe8f77fe640 1 -- 172.21.15.39:0/760892152 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8030b20 con 0x7fe90019f300 2024-05-04T03:11:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.584+0000 7fe904ac8640 1 -- 172.21.15.39:0/760892152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9001a2570 con 0x7fe90019f300 2024-05-04T03:11:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.587+0000 7fe90b97c740 1 Processor -- start 2024-05-04T03:11:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.587+0000 7fe8f77fe640 1 -- 172.21.15.39:0/760892152 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fe8f8030cc0 con 0x7fe90019f300 2024-05-04T03:11:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe90b97c740 1 -- start start 2024-05-04T03:11:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c70d1b0 0x55d70caaea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c6e42f0 0x55d70caaefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cab5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe90b97c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d70ca135d0 con 0x55d70c6e42f0 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe90b97c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d70c9cc890 con 0x55d70c6ec4d0 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe90b97c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d70ca0f590 con 0x55d70c70d1b0 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c70d1b0 0x55d70caaea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe8fe59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c6e42f0 0x55d70caaefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c70d1b0 0x55d70caaea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:58608/0 (socket says 172.21.15.39:58608) 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe8f7fff640 1 -- 172.21.15.39:0/2337450535 learned_addr learned my addr 172.21.15.39:0/2337450535 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.588+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cab5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8f77fe640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe8ec0b2870 0x7fe8ec0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8fe59c640 1 -- 172.21.15.39:0/2337450535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c70d1b0 msgr2=0x55d70caaea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c70d1b0 0x55d70caaea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8fe59c640 1 -- 172.21.15.39:0/2337450535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 msgr2=0x55d70cab5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cab5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8fe59c640 1 -- 172.21.15.39:0/2337450535 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d70c725210 con 0x55d70c6e42f0 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8fed9d640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe8ec0b2870 0x7fe8ec0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c70d1b0 0x55d70caaea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.589+0000 7fe8f77fe640 1 -- 172.21.15.39:0/760892152 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe8f8103510 con 0x7fe90019f300 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.590+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c6e42f0 0x55d70caaefe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe90006a480 tx=0x7fe8f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f913fd8f3b3a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.590+0000 7fe8f4ff9640 1 -- 172.21.15.39:0/2337450535 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0009e60 con 0x55d70c6e42f0 2024-05-04T03:11:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.590+0000 7fe8fed9d640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe8ec0b2870 0x7fe8ec0b4d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80ca7f0 tx=0x7fe8f80bf3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.590+0000 7fe8f4ff9640 1 -- 172.21.15.39:0/2337450535 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8f0004230 con 0x55d70c6e42f0 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.590+0000 7fe8f4ff9640 1 -- 172.21.15.39:0/2337450535 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0019680 con 0x55d70c6e42f0 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 -- 172.21.15.39:0/2337450535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c6e42f0 msgr2=0x55d70caaefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c6e42f0 0x55d70caaefe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe90006a480 tx=0x7fe8f0011810 comp rx=0 tx=0).stop 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 -- 172.21.15.39:0/2337450535 shutdown_connections 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cab5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c6e42f0 0x55d70caaefe0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 --2- 172.21.15.39:0/2337450535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c70d1b0 0x55d70caaea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 -- 172.21.15.39:0/2337450535 >> 172.21.15.39:0/2337450535 conn(0x55d70c701300 msgr2=0x55d70caaa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 -- 172.21.15.39:0/2337450535 shutdown_connections 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.591+0000 7fe90b97c740 1 -- 172.21.15.39:0/2337450535 wait complete. 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 1 -- start start 2024-05-04T03:11:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 1 Processor -- start 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cab8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d70cab6930 con 0x55d70c70d1b0 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d70c9d0370 con 0x55d70c6ec4d0 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d70cab6ab0 con 0x55d70c6e42f0 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe90b97c740 10 client.0 _ll_register_callbacks cb 0x55d70c724a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:58614/0 (socket says 172.21.15.39:58614) 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe8f7fff640 1 -- 172.21.15.39:0/115990930 learned_addr learned my addr 172.21.15.39:0/115990930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.592+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cab8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8f7fff640 1 -- 172.21.15.39:0/115990930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 msgr2=0x55d70cab8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cab8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8f7fff640 1 -- 172.21.15.39:0/115990930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 msgr2=0x55d70cabfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8f7fff640 1 -- 172.21.15.39:0/115990930 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70c7f1620 con 0x55d70c6e42f0 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab86e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800f550 tx=0x7fe8e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f274bfe553f5b3ad server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/115990930 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8004460 con 0x55d70c6e42f0 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe90b97c740 1 -- 172.21.15.39:0/115990930 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f0004230 con 0x55d70c6e42f0 2024-05-04T03:11:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/115990930 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe8e80035f0 con 0x55d70c6e42f0 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/115990930 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8e80037f0 con 0x55d70c6e42f0 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe90b97c740 10 client.25126 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.593+0000 7fe90b97c740 1 -- 172.21.15.39:0/115990930 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d70cac2b70 con 0x55d70c6e42f0 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/115990930 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8f0004230 con 0x55d70c6e42f0 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/115990930 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe8e800e800 con 0x55d70c6e42f0 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe90b97c740 10 client.25126 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe90b97c740 10 client.25126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe90b97c740 20 client.25126 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe90b97c740 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d70c6cbd80 0x55d70cac62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe90b97c740 4 client.25126 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe90b97c740 1 -- 172.21.15.39:0/115990930 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d70ca211e0 con 0x55d70c6cbd80 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.594+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d70c6cbd80 0x55d70cac62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.595+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d70c6cbd80 0x55d70cac62d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f66b82023ea3cc2 server_cookie=d7b29c013f794aad in_seq=0 out_seq=0 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.595+0000 7fe8e5ffb640 10 client.25126 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.595+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/115990930 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d70ca211e0 con 0x55d70c6cbd80 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.595+0000 7fe8e5ffb640 10 client.25126 handle_command_reply: tid=0 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.595+0000 7fe90b97c740 1 client.25126 shutdown 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.595+0000 7fe90b97c740 1 -- 172.21.15.39:0/115990930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 msgr2=0x55d70cab86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.595+0000 7fe90b97c740 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab86e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800f550 tx=0x7fe8e80046c0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 1 -- 172.21.15.39:0/115990930 shutdown_connections 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d70c6cbd80 0x55d70cac62d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cab8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 1 --2- 172.21.15.39:0/115990930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab86e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe8e5ffb640 0 client.25126 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 1 -- 172.21.15.39:0/115990930 >> 172.21.15.39:0/115990930 conn(0x55d70c701300 msgr2=0x55d70caad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 1 -- 172.21.15.39:0/115990930 shutdown_connections 2024-05-04T03:11:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 1 -- 172.21.15.39:0/115990930 wait complete. 2024-05-04T03:11:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.596+0000 7fe90b97c740 20 client.25126 trim_cache size 0 max 16384 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe90b97c740 1 Processor -- start 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe90b97c740 1 -- start start 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cabfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cac0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe90b97c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9001a8fd0 con 0x55d70c70d1b0 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe90b97c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9001aabd0 con 0x55d70c6ec4d0 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe90b97c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9001a96a0 con 0x55d70c6e42f0 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cac0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cac0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39310/0 (socket says 172.21.15.39:39310) 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.618+0000 7fe8f7fff640 1 -- 172.21.15.39:0/598602963 learned_addr learned my addr 172.21.15.39:0/598602963 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cabfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8fe59c640 1 -- 172.21.15.39:0/598602963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 msgr2=0x55d70cab7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8fe59c640 1 -- 172.21.15.39:0/598602963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 msgr2=0x55d70cac0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cac0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8fe59c640 1 -- 172.21.15.39:0/598602963 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d70cab62a0 con 0x55d70c6ec4d0 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cabfb10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8f001b040 tx=0x7fe8f0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4b21a845f1144ef server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8e7fff640 1 -- 172.21.15.39:0/598602963 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0003fe0 con 0x55d70c6ec4d0 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8e7fff640 1 -- 172.21.15.39:0/598602963 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8f0004180 con 0x55d70c6ec4d0 2024-05-04T03:11:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.619+0000 7fe8e7fff640 1 -- 172.21.15.39:0/598602963 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0005530 con 0x55d70c6ec4d0 2024-05-04T03:11:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.620+0000 7fe90b97c740 1 -- 172.21.15.39:0/598602963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 msgr2=0x55d70cabfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cabfb10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8f001b040 tx=0x7fe8f0005f90 comp rx=0 tx=0).stop 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 -- 172.21.15.39:0/598602963 shutdown_connections 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cac0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6ec4d0 0x55d70cabfb10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 --2- 172.21.15.39:0/598602963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6e42f0 0x55d70cab7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 -- 172.21.15.39:0/598602963 >> 172.21.15.39:0/598602963 conn(0x55d70c701300 msgr2=0x55d70cab6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 -- 172.21.15.39:0/598602963 shutdown_connections 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 -- 172.21.15.39:0/598602963 wait complete. 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 -- start start 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.621+0000 7fe90b97c740 1 Processor -- start 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6e42f0 0x55d70cabff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6ec4d0 0x55d70cabcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe90b97c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe90b97c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9001aa440 con 0x55d70c70d1b0 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe90b97c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9001a9d70 con 0x55d70c6e42f0 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe90b97c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9001a6fb0 con 0x55d70c6ec4d0 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe90b97c740 10 client.0 _ll_register_callbacks cb 0x7fe8e8009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8fe59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6ec4d0 0x55d70cabcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39318/0 (socket says 172.21.15.39:39318) 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 -- 172.21.15.39:0/3082238826 learned_addr learned my addr 172.21.15.39:0/3082238826 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6e42f0 0x55d70cabff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 -- 172.21.15.39:0/3082238826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6ec4d0 msgr2=0x55d70cabcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6ec4d0 0x55d70cabcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 -- 172.21.15.39:0/3082238826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6e42f0 msgr2=0x55d70cabff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6e42f0 0x55d70cabff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 -- 172.21.15.39:0/3082238826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70cb2c630 con 0x55d70c70d1b0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6ec4d0 0x55d70cabcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8f7fff640 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabd100 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8011630 tx=0x7fe8e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de9f84edfb2942c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.622+0000 7fe8e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.623+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/3082238826 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8003ba0 con 0x55d70c70d1b0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.623+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/3082238826 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe8e8036070 con 0x55d70c70d1b0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.623+0000 7fe90b97c740 1 -- 172.21.15.39:0/3082238826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8f0004180 con 0x55d70c70d1b0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.623+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/3082238826 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8e800e450 con 0x55d70c70d1b0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.623+0000 7fe90b97c740 10 client.15438 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.623+0000 7fe90b97c740 1 -- 172.21.15.39:0/3082238826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d70cb2d870 con 0x55d70c70d1b0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.623+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/3082238826 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8f0004180 con 0x55d70c70d1b0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.624+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/3082238826 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe8e802d8a0 con 0x55d70c70d1b0 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.624+0000 7fe90b97c740 10 client.15438 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.624+0000 7fe90b97c740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:11:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.624+0000 7fe90b97c740 20 client.15438 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.624+0000 7fe90b97c740 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d70c6a11c0 0x55d70cb301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.624+0000 7fe90b97c740 4 client.15438 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:11:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.624+0000 7fe90b97c740 1 -- 172.21.15.39:0/3082238826 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d70ca1b050 con 0x55d70c6a11c0 2024-05-04T03:11:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.624+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d70c6a11c0 0x55d70cb301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.625+0000 7fe8fe59c640 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d70c6a11c0 0x55d70cb301e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428e5033102e929c server_cookie=c0650b4751fded5c in_seq=0 out_seq=0 2024-05-04T03:11:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.625+0000 7fe8e5ffb640 10 client.15438 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.635+0000 7fe8e5ffb640 1 -- 172.21.15.39:0/3082238826 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d70ca1b050 con 0x55d70c6a11c0 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.635+0000 7fe8e5ffb640 10 client.15438 handle_command_reply: tid=0 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.635+0000 7fe90b97c740 1 client.15438 shutdown 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.635+0000 7fe90b97c740 1 -- 172.21.15.39:0/3082238826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 msgr2=0x55d70cabd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.635+0000 7fe90b97c740 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabd100 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8011630 tx=0x7fe8e8000bc0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 1 -- 172.21.15.39:0/3082238826 shutdown_connections 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d70c6a11c0 0x55d70cb301e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d70c70d1b0 0x55d70cabd100 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d70c6ec4d0 0x55d70cabcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 1 --2- 172.21.15.39:0/3082238826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d70c6e42f0 0x55d70cabff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 1 -- 172.21.15.39:0/3082238826 >> 172.21.15.39:0/3082238826 conn(0x55d70c701300 msgr2=0x55d70cab6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe8e5ffb640 0 client.15438 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 1 -- 172.21.15.39:0/3082238826 shutdown_connections 2024-05-04T03:11:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 1 -- 172.21.15.39:0/3082238826 wait complete. 2024-05-04T03:11:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.636+0000 7fe90b97c740 20 client.15438 trim_cache size 0 max 16384 2024-05-04T03:11:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:11:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:11:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:11:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:11:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 -- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe8ec0b2870 msgr2=0x7fe8ec0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe8ec0b2870 0x7fe8ec0b4d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80ca7f0 tx=0x7fe8f80bf3c0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 -- 172.21.15.39:0/760892152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe90019f300 msgr2=0x7fe9001a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe90019f300 0x7fe9001a1720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8031920 tx=0x7fe8f8009870 comp rx=0 tx=0).stop 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 -- 172.21.15.39:0/760892152 shutdown_connections 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe8ec0b2870 0x7fe8ec0b4d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe90019f300 0x7fe9001a1720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe900103a80 0x7fe90019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 --2- 172.21.15.39:0/760892152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9001030a0 0x7fe900199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 -- 172.21.15.39:0/760892152 >> 172.21.15.39:0/760892152 conn(0x7fe9000fd4c0 msgr2=0x7fe9000feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 -- 172.21.15.39:0/760892152 shutdown_connections 2024-05-04T03:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:46.637+0000 7fe904ac8640 1 -- 172.21.15.39:0/760892152 wait complete. 2024-05-04T03:11:46.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:46 smithi167 ceph-mon[33614]: pgmap v415: 97 pgs: 97 active+clean; 39 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 3.2 MiB/s wr, 101 op/s 2024-05-04T03:11:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:46 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:46.785 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-04T03:11:47.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:47.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:47.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:47.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:47.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:47.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:47.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:47.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:47.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:47.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:47.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:47.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:47.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:47.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:47.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:11:48.434 INFO:teuthology.orchestra.run.smithi039.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-04T03:11:48.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:48 smithi039 ceph-mon[29715]: pgmap v416: 97 pgs: 97 active+clean; 40 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 3.3 MiB/s wr, 103 op/s 2024-05-04T03:11:48.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-04T03:11:48.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:48 smithi066 ceph-mon[33264]: pgmap v416: 97 pgs: 97 active+clean; 40 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 3.3 MiB/s wr, 103 op/s 2024-05-04T03:11:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:48 smithi167 ceph-mon[33614]: pgmap v416: 97 pgs: 97 active+clean; 40 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 3.3 MiB/s wr, 103 op/s 2024-05-04T03:11:48.755 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-05-04T03:11:48.755 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-04T03:11:48.755 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 25356883 May 4 03:11 . 2024-05-04T03:11:48.755 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 25356883 May 4 03:11 .. 2024-05-04T03:11:48.755 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25356883 May 4 03:11 data 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:total 45 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25356883 May 4 03:11 . 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 25356883 May 4 03:11 .. 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 4 03:11 base 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 4 03:11 global 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 log 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_commit_ts 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_dynshmem 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 4 03:11 pg_hba.conf 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 4 03:11 pg_ident.conf 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:11 pg_logical 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 pg_multixact 2024-05-04T03:11:48.756 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_notify 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_replslot 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_serial 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_snapshots 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_stat 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_stat_tmp 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 pg_subtrans 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_tblspc 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_twophase 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 4 03:11 PG_VERSION 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:11 pg_wal 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 pg_xact 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 4 03:11 postgresql.auto.conf 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 4 03:11 postgresql.conf 2024-05-04T03:11:48.757 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base: 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 4 03:11 . 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25356883 May 4 03:11 .. 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 1 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 13493 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 13494 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:total 7778 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 . 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 4 03:11 .. 2024-05-04T03:11:48.758 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 112 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 113 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 1247 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1247_fsm 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1247_vm 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 4 03:11 1249 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1249_fsm 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1249_vm 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 4 03:11 1255 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1255_fsm 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1255_vm 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 4 03:11 1259 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1259_fsm 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1259_vm 2024-05-04T03:11:48.759 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 4 03:11 13344 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13344_fsm 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13344_vm 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13346 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13348 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13349_fsm 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349_vm 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13351 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13353 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13354_fsm 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354_vm 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13356 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13358 2024-05-04T03:11:48.760 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13359_fsm 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359_vm 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13361 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13363 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1417 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1418 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 174 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 175 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2187 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2224 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2328 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2336 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2337 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2579 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2600 2024-05-04T03:11:48.761 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2600_fsm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2600_vm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2601_fsm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601_vm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2602 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2602_fsm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2602_vm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2603 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2603_fsm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2603_vm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2604 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2605 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2605_fsm 2024-05-04T03:11:48.762 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2605_vm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2606_fsm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606_vm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2607 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2607_fsm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2607_vm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 4 03:11 2608 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2608_fsm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2608_vm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 4 03:11 2609 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2609_fsm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2609_vm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2610 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2610_fsm 2024-05-04T03:11:48.763 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2610_vm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2611 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2612_fsm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612_vm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2613 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2615_fsm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615_vm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2616 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2616_fsm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2616_vm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 4 03:11 2617 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2617_fsm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2617_vm 2024-05-04T03:11:48.764 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 4 03:11 2618 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2618_fsm 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2618_vm 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 4 03:11 2619 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2619_fsm 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2619_vm 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2620 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2650 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2651 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2652 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2653 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2654 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2655 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2656 2024-05-04T03:11:48.765 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2657 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 4 03:11 2658 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 2659 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2660 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2661 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2662 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2663 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2664 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2665 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2666 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2667 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2668 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2669 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2670 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 4 03:11 2673 2024-05-04T03:11:48.766 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 4 03:11 2674 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:11 2675 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2678 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2679 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2680 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2681 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2682 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2683 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2684 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2685 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2686 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2687 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2688 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2689 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 4 03:11 2690 2024-05-04T03:11:48.767 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 4 03:11 2691 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2692 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2693 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2696 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2699 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2701 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2702 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2703 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2704 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2753 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2753_fsm 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2753_vm 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2754 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2755 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2756 2024-05-04T03:11:48.768 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2757 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2830 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2831 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2832 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2833 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2834 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2835 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2836 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2837 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 4 03:11 2838 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2838_fsm 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2838_vm 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2839 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840_fsm 2024-05-04T03:11:48.769 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2840_vm 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2841 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2995 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2996 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3079_fsm 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079_vm 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3080 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3081 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3085 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3118 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3119 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 4 03:11 3164 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3256 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3257 2024-05-04T03:11:48.770 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3258 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3350 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3351 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3379 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3380 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3381 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3394 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3394_fsm 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3394_vm 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3395 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3429 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3430 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3431 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3433 2024-05-04T03:11:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3439 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3440 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3455 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 4 03:11 3456 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3456_fsm 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3456_vm 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3466 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3467 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3468 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3501 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3502 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3503 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3534 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3541_fsm 2024-05-04T03:11:48.772 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541_vm 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3542 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3574 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3575 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3576 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3596 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3597 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3598 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3599 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3600_fsm 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600_vm 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3601_fsm 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601_vm 2024-05-04T03:11:48.773 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3602_fsm 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602_vm 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603_fsm 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3603_vm 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3604 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3605 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3606 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3607 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3608 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 3609 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3712 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3764_fsm 2024-05-04T03:11:48.774 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764_vm 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3766 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3767 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3997 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4143 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4144 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4145 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4146 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4147 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4148 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4149 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4150 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4151 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4152 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4153 2024-05-04T03:11:48.775 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4154 2024-05-04T03:11:48.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4155 2024-05-04T03:11:48.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4156 2024-05-04T03:11:48.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4157 2024-05-04T03:11:48.776 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4158 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4159 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4160 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4161 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4162 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4163 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4164 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4165 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4166 2024-05-04T03:11:49.014 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4167 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4168 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4169 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4170 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4171 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4172 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4173 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4174 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 5002 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 548 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 549 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6102 2024-05-04T03:11:49.015 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6104 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6106 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6110 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6111 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6112 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6113 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6117 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 826 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 827 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 828 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 4 03:11 pg_filenode.map 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 4 03:11 PG_VERSION 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:total 7778 2024-05-04T03:11:49.016 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 . 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 4 03:11 .. 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 112 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 113 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 1247 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1247_fsm 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1247_vm 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 4 03:11 1249 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1249_fsm 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1249_vm 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 4 03:11 1255 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1255_fsm 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1255_vm 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 4 03:11 1259 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1259_fsm 2024-05-04T03:11:49.017 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1259_vm 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 4 03:11 13344 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13344_fsm 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13344_vm 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13346 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13348 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13349_fsm 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349_vm 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13351 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13353 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13354_fsm 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354_vm 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13356 2024-05-04T03:11:49.018 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13358 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13359_fsm 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359_vm 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13361 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13363 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1417 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1418 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 174 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 175 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2187 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2224 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2328 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2336 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2337 2024-05-04T03:11:49.019 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2579 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2600 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2600_fsm 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2600_vm 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2601_fsm 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601_vm 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2602 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2602_fsm 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2602_vm 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2603 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2603_fsm 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2603_vm 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2604 2024-05-04T03:11:49.020 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2605 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2605_fsm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2605_vm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2606_fsm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606_vm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2607 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2607_fsm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2607_vm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 4 03:11 2608 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2608_fsm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2608_vm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 4 03:11 2609 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2609_fsm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2609_vm 2024-05-04T03:11:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2610 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2610_fsm 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2610_vm 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2611 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2612_fsm 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612_vm 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2613 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2615_fsm 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615_vm 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2616 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2616_fsm 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2616_vm 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 4 03:11 2617 2024-05-04T03:11:49.022 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2617_fsm 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2617_vm 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 4 03:11 2618 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2618_fsm 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2618_vm 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 4 03:11 2619 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2619_fsm 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2619_vm 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2620 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2650 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2651 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2652 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2653 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2654 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2655 2024-05-04T03:11:49.023 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2656 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2657 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 4 03:11 2658 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 2659 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2660 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2661 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2662 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2663 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2664 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2665 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2666 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2667 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2668 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2669 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2670 2024-05-04T03:11:49.024 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 4 03:11 2673 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 4 03:11 2674 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:11 2675 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2678 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2679 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2680 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2681 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2682 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2683 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2684 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2685 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2686 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2687 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2688 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2689 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 4 03:11 2690 2024-05-04T03:11:49.025 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 4 03:11 2691 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2692 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2693 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2696 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2699 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2701 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2702 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2703 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2704 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2753 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2753_fsm 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2753_vm 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2754 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2755 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2756 2024-05-04T03:11:49.026 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2757 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2830 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2831 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2832 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2833 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2834 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2835 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2836 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2837 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 4 03:11 2838 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2838_fsm 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2838_vm 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2839 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840 2024-05-04T03:11:49.027 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840_fsm 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2840_vm 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2841 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2995 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2996 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3079_fsm 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079_vm 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3080 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3081 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3085 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3118 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3119 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 4 03:11 3164 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3256 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3257 2024-05-04T03:11:49.028 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3258 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3350 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3351 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3379 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3380 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3381 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3394 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3394_fsm 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3394_vm 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3395 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3429 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3430 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3431 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3433 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3439 2024-05-04T03:11:49.029 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3440 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3455 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 4 03:11 3456 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3456_fsm 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3456_vm 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3466 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3467 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3468 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3501 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3502 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3503 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3534 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3541_fsm 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541_vm 2024-05-04T03:11:49.030 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3542 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3574 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3575 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3576 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3596 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3597 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3598 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3599 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3600_fsm 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600_vm 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3601_fsm 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601_vm 2024-05-04T03:11:49.031 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3602_fsm 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602_vm 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603_fsm 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3603_vm 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3604 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3605 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3606 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3607 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3608 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 3609 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3712 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3764_fsm 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764_vm 2024-05-04T03:11:49.032 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3766 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3767 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3997 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4143 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4144 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4145 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4146 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4147 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4148 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4149 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4150 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4151 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4152 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4153 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4154 2024-05-04T03:11:49.033 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4155 2024-05-04T03:11:49.034 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4156 2024-05-04T03:11:49.034 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4157 2024-05-04T03:11:49.034 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4158 2024-05-04T03:11:49.034 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4159 2024-05-04T03:11:49.034 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4160 2024-05-04T03:11:49.034 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4161 2024-05-04T03:11:49.034 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4162 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4163 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4164 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4165 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4166 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4167 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4168 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4169 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4170 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4171 2024-05-04T03:11:49.272 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4172 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4173 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4174 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 5002 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 548 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 549 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6102 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6104 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6106 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6110 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6111 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6112 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6113 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6117 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 826 2024-05-04T03:11:49.273 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 827 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 828 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 4 03:11 pg_filenode.map 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 4 03:11 PG_VERSION 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:total 7778 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 . 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 4 03:11 .. 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 112 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 113 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 1247 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1247_fsm 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1247_vm 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 4 03:11 1249 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1249_fsm 2024-05-04T03:11:49.274 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1249_vm 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 4 03:11 1255 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1255_fsm 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1255_vm 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 4 03:11 1259 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1259_fsm 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1259_vm 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 4 03:11 13344 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13344_fsm 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13344_vm 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13346 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13348 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13349_fsm 2024-05-04T03:11:49.275 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349_vm 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13351 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13353 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13354_fsm 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354_vm 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13356 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13358 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359 2024-05-04T03:11:49.276 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13359_fsm 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359_vm 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13361 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13363 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1417 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1418 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 174 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 175 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2187 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2224 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2328 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2336 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2337 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2579 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2600 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2600_fsm 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2600_vm 2024-05-04T03:11:49.277 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2601_fsm 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601_vm 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2602 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2602_fsm 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2602_vm 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2603 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2603_fsm 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2603_vm 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2604 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2605 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2605_fsm 2024-05-04T03:11:49.278 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2605_vm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2606_fsm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606_vm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2607 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2607_fsm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2607_vm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 4 03:11 2608 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2608_fsm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2608_vm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 4 03:11 2609 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2609_fsm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2609_vm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2610 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2610_fsm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2610_vm 2024-05-04T03:11:49.279 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2611 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2612_fsm 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612_vm 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2613 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2615_fsm 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615_vm 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2616 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2616_fsm 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2616_vm 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 4 03:11 2617 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2617_fsm 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2617_vm 2024-05-04T03:11:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 4 03:11 2618 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2618_fsm 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2618_vm 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 4 03:11 2619 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2619_fsm 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2619_vm 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2620 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2650 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2651 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2652 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2653 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2654 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2655 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2656 2024-05-04T03:11:49.281 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2657 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 4 03:11 2658 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 2659 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2660 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2661 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2662 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2663 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2664 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2665 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2666 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2667 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2668 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2669 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2670 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 4 03:11 2673 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 4 03:11 2674 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:11 2675 2024-05-04T03:11:49.282 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2678 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2679 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2680 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2681 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2682 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2683 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2684 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2685 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2686 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2687 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2688 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2689 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 4 03:11 2690 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 4 03:11 2691 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2692 2024-05-04T03:11:49.283 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2693 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2696 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2699 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2701 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2702 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2703 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2704 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2753 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2753_fsm 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2753_vm 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2754 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2755 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2756 2024-05-04T03:11:49.284 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2757 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2830 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2831 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2832 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2833 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2834 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2835 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2836 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2837 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 4 03:11 2838 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2838_fsm 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2838_vm 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2839 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840_fsm 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2840_vm 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2841 2024-05-04T03:11:49.285 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2995 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2996 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3079_fsm 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079_vm 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3080 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3081 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3085 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3118 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3119 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 4 03:11 3164 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3256 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3257 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3258 2024-05-04T03:11:49.286 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3350 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3351 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3379 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3380 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3381 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3394 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3394_fsm 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3394_vm 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3395 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3429 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3430 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3431 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3433 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3439 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3440 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3455 2024-05-04T03:11:49.287 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 4 03:11 3456 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3456_fsm 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3456_vm 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3466 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3467 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3468 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3501 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3502 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3503 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3534 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3541_fsm 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541_vm 2024-05-04T03:11:49.288 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3542 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3574 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3575 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3576 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3596 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3597 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3598 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3599 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3600_fsm 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600_vm 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3601_fsm 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601_vm 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602 2024-05-04T03:11:49.289 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3602_fsm 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602_vm 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603_fsm 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3603_vm 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3604 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3605 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3606 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3607 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3608 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 3609 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3712 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3764_fsm 2024-05-04T03:11:49.290 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764_vm 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3766 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3767 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3997 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4143 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4144 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4145 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4146 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4147 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4148 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4149 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4150 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4151 2024-05-04T03:11:49.291 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4152 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4153 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4154 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4155 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4156 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4157 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4158 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4159 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4160 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4161 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4162 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4163 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4164 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4165 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4166 2024-05-04T03:11:49.292 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4167 2024-05-04T03:11:49.316 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4168 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4169 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4170 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4171 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4172 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4173 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4174 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 5002 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 548 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 549 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6102 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6104 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6106 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6110 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6111 2024-05-04T03:11:49.317 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6112 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6113 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6117 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 826 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 827 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 828 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 4 03:11 pg_filenode.map 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 4 03:11 PG_VERSION 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/global: 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:total 538 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 4 03:11 . 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1213 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1213_fsm 2024-05-04T03:11:49.318 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1213_vm 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1214 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1214_fsm 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1214_vm 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 1232 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 1233 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1260 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1260_fsm 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1260_vm 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1261 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1261_fsm 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1261_vm 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1262 2024-05-04T03:11:49.319 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1262_fsm 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1262_vm 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2396 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2396_fsm 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2396_vm 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2397 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2671 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2672 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2676 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2677 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2694 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2695 2024-05-04T03:11:49.320 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2697 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2698 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2846 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2847 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2964 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2965 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2966 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2967 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3592 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3593 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4060 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4061 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4175 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4176 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4177 2024-05-04T03:11:49.321 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4178 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4181 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4182 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4183 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4184 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4185 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4186 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6000 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6001 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6002 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6100 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6114 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6115 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 pg_control 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 4 03:11 pg_filenode.map 2024-05-04T03:11:49.322 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.323 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/log: 2024-05-04T03:11:49.323 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.323 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.373 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.373 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.373 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-04T03:11:49.373 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.373 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.373 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.373 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:11 . 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 mappings 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:11 replorigin_checkpoint 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 snapshots 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-04T03:11:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:11 .. 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:11 .. 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 . 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 members 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 offsets 2024-05-04T03:11:49.375 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 . 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 .. 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 0000 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 . 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 .. 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 0000 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-04T03:11:49.376 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.377 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.378 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.379 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 . 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 0000 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-04T03:11:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:total 16386 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:11 . 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:11 000000010000000000000001 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 archive_status 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:11 .. 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:11:49.393 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-04T03:11:49.394 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-04T03:11:49.394 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 . 2024-05-04T03:11:49.394 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 4 03:11 .. 2024-05-04T03:11:49.394 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 0000 2024-05-04T03:11:49.395 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-04T03:11:49.845 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-04T03:11:49.934 INFO:teuthology.orchestra.run.smithi039.stderr:dropping old tables... 2024-05-04T03:11:49.942 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-04T03:11:49.942 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-04T03:11:49.942 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-04T03:11:49.942 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-04T03:11:49.942 INFO:teuthology.orchestra.run.smithi039.stderr:creating tables... 2024-05-04T03:11:49.999 INFO:teuthology.orchestra.run.smithi039.stderr:generating data (client-side)... 2024-05-04T03:11:50.537 INFO:teuthology.orchestra.run.smithi039.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.51 s, remaining 15.83 s) 2024-05-04T03:11:50.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:50 smithi066 ceph-mon[33264]: pgmap v417: 97 pgs: 97 active+clean; 45 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 3.0 MiB/s wr, 106 op/s 2024-05-04T03:11:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:50 smithi167 ceph-mon[33614]: pgmap v417: 97 pgs: 97 active+clean; 45 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 3.0 MiB/s wr, 106 op/s 2024-05-04T03:11:50.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:50 smithi039 ceph-mon[29715]: pgmap v417: 97 pgs: 97 active+clean; 45 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 3.0 MiB/s wr, 106 op/s 2024-05-04T03:11:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:200000 of 3200000 tuples (6%) done (elapsed 1.71 s, remaining 25.67 s) 2024-05-04T03:11:52.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:52 smithi066 ceph-mon[33264]: pgmap v418: 97 pgs: 97 active+clean; 48 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-05-04T03:11:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:52 smithi167 ceph-mon[33614]: pgmap v418: 97 pgs: 97 active+clean; 48 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-05-04T03:11:52.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:52 smithi039 ceph-mon[29715]: pgmap v418: 97 pgs: 97 active+clean; 48 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-05-04T03:11:53.002 INFO:teuthology.orchestra.run.smithi039.stderr:300000 of 3200000 tuples (9%) done (elapsed 2.98 s, remaining 28.77 s) 2024-05-04T03:11:53.769 INFO:teuthology.orchestra.run.smithi039.stderr:400000 of 3200000 tuples (12%) done (elapsed 3.74 s, remaining 26.20 s) 2024-05-04T03:11:54.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:54 smithi066 ceph-mon[33264]: pgmap v419: 97 pgs: 97 active+clean; 82 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 6.9 MiB/s wr, 123 op/s 2024-05-04T03:11:54.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:54 smithi167 ceph-mon[33614]: pgmap v419: 97 pgs: 97 active+clean; 82 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 6.9 MiB/s wr, 123 op/s 2024-05-04T03:11:54.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:54 smithi039 ceph-mon[29715]: pgmap v419: 97 pgs: 97 active+clean; 82 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 6.9 MiB/s wr, 123 op/s 2024-05-04T03:11:55.012 INFO:teuthology.orchestra.run.smithi039.stderr:500000 of 3200000 tuples (15%) done (elapsed 4.99 s, remaining 26.93 s) 2024-05-04T03:11:55.894 INFO:teuthology.orchestra.run.smithi039.stderr:600000 of 3200000 tuples (18%) done (elapsed 5.87 s, remaining 25.43 s) 2024-05-04T03:11:56.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:56 smithi066 ceph-mon[33264]: pgmap v420: 97 pgs: 97 active+clean; 106 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 10 MiB/s wr, 109 op/s 2024-05-04T03:11:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:56 smithi167 ceph-mon[33614]: pgmap v420: 97 pgs: 97 active+clean; 106 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 10 MiB/s wr, 109 op/s 2024-05-04T03:11:56.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:56 smithi039 ceph-mon[29715]: pgmap v420: 97 pgs: 97 active+clean; 106 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 10 MiB/s wr, 109 op/s 2024-05-04T03:11:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:700000 of 3200000 tuples (21%) done (elapsed 7.06 s, remaining 25.22 s) 2024-05-04T03:11:58.321 INFO:teuthology.orchestra.run.smithi039.stderr:800000 of 3200000 tuples (25%) done (elapsed 8.29 s, remaining 24.88 s) 2024-05-04T03:11:58.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:11:58 smithi039 ceph-mon[29715]: pgmap v421: 97 pgs: 97 active+clean; 133 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 14 MiB/s wr, 109 op/s 2024-05-04T03:11:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:11:58 smithi066 ceph-mon[33264]: pgmap v421: 97 pgs: 97 active+clean; 133 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 14 MiB/s wr, 109 op/s 2024-05-04T03:11:58.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:11:58 smithi167 ceph-mon[33614]: pgmap v421: 97 pgs: 97 active+clean; 133 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 14 MiB/s wr, 109 op/s 2024-05-04T03:11:59.218 INFO:teuthology.orchestra.run.smithi039.stderr:900000 of 3200000 tuples (28%) done (elapsed 9.19 s, remaining 23.49 s) 2024-05-04T03:11:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 -- 172.21.15.39:0/375686454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 msgr2=0x7f183810aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 --2- 172.21.15.39:0/375686454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 0x7f183810aed0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1830009ca0 tx=0x7f183002a8c0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 -- 172.21.15.39:0/375686454 shutdown_connections 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 --2- 172.21.15.39:0/375686454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 0x7f183810aed0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 --2- 172.21.15.39:0/375686454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103db0 0x7f18381041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 --2- 172.21.15.39:0/375686454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102b70 0x7f1838103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 -- 172.21.15.39:0/375686454 >> 172.21.15.39:0/375686454 conn(0x7f18380fdd60 msgr2=0x7f1838100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 -- 172.21.15.39:0/375686454 shutdown_connections 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.766+0000 7f183f21e640 1 -- 172.21.15.39:0/375686454 wait complete. 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183f21e640 1 Processor -- start 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183f21e640 1 -- start start 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183f21e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102b70 0x7f183819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183f21e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103db0 0x7f183819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183f21e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 0x7f18381a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183f21e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f183810e5a0 con 0x7f1838108ab0 2024-05-04T03:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183f21e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f183810e420 con 0x7f1838103db0 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183f21e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f183810e720 con 0x7f1838102b70 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183d7bb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 0x7f18381a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183d7bb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 0x7f18381a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60822/0 (socket says 172.21.15.39:60822) 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183cfba640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102b70 0x7f183819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f183d7bb640 1 -- 172.21.15.39:0/3452657396 learned_addr learned my addr 172.21.15.39:0/3452657396 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.767+0000 7f182ffff640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103db0 0x7f183819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f183d7bb640 1 -- 172.21.15.39:0/3452657396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102b70 msgr2=0x7f183819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f183d7bb640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102b70 0x7f183819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f183d7bb640 1 -- 172.21.15.39:0/3452657396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103db0 msgr2=0x7f183819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f183d7bb640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103db0 0x7f183819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f183d7bb640 1 -- 172.21.15.39:0/3452657396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18381a1d70 con 0x7f1838108ab0 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f182ffff640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103db0 0x7f183819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f183d7bb640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 0x7f18381a1700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1830031040 tx=0x7f1830003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f182dffb640 1 -- 172.21.15.39:0/3452657396 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830004040 con 0x7f1838108ab0 2024-05-04T03:11:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f182dffb640 1 -- 172.21.15.39:0/3452657396 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1830004670 con 0x7f1838108ab0 2024-05-04T03:11:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f183f21e640 1 -- 172.21.15.39:0/3452657396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18381a2000 con 0x7f1838108ab0 2024-05-04T03:11:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f182dffb640 1 -- 172.21.15.39:0/3452657396 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830030d50 con 0x7f1838108ab0 2024-05-04T03:11:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.768+0000 7f183f21e640 1 -- 172.21.15.39:0/3452657396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18381a2550 con 0x7f1838108ab0 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f18461a8740 1 Processor -- start 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f182dffb640 1 -- 172.21.15.39:0/3452657396 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f18300305c0 con 0x7f1838108ab0 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f18461a8740 1 -- start start 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f18461a8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e9b1b0 0x558df223ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f18461a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e722f0 0x558df223cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f18461a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df2243ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f18461a8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558df21a15d0 con 0x558df1e9b1b0 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f18461a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558df215a890 con 0x558df1e7a4d0 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f18461a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558df219d590 con 0x558df1e722f0 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.771+0000 7f182ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e9b1b0 0x558df223ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e9b1b0 0x558df223ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60828/0 (socket says 172.21.15.39:60828) 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 -- 172.21.15.39:0/2335223231 learned_addr learned my addr 172.21.15.39:0/2335223231 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f183cfba640 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e722f0 0x558df223cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df2243ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 -- 172.21.15.39:0/2335223231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e722f0 msgr2=0x558df223cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e722f0 0x558df223cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 -- 172.21.15.39:0/2335223231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 msgr2=0x558df2243ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df2243ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 -- 172.21.15.39:0/2335223231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df1eb3210 con 0x558df1e9b1b0 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182dffb640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18200b2870 0x7f18200b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182ffff640 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e9b1b0 0x558df223ca60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f18280135e0 tx=0x7f182800b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142a75320e3eccca server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f183cfba640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18200b2870 0x7f18200b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f181effd640 1 -- 172.21.15.39:0/2335223231 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828019070 con 0x558df1e9b1b0 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f182dffb640 1 -- 172.21.15.39:0/3452657396 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1830103860 con 0x7f1838108ab0 2024-05-04T03:11:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f181effd640 1 -- 172.21.15.39:0/2335223231 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18280047a0 con 0x558df1e9b1b0 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.772+0000 7f181effd640 1 -- 172.21.15.39:0/2335223231 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828011720 con 0x558df1e9b1b0 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f183cfba640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18200b2870 0x7f18200b4d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x558df223de40 tx=0x7f183400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 -- 172.21.15.39:0/2335223231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e9b1b0 msgr2=0x558df223ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e9b1b0 0x558df223ca60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f18280135e0 tx=0x7f182800b6f0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 -- 172.21.15.39:0/2335223231 shutdown_connections 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df2243ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e722f0 0x558df223cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 --2- 172.21.15.39:0/2335223231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e9b1b0 0x558df223ca60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 -- 172.21.15.39:0/2335223231 >> 172.21.15.39:0/2335223231 conn(0x558df1e8f300 msgr2=0x558df22383a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 -- 172.21.15.39:0/2335223231 shutdown_connections 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.773+0000 7f18461a8740 1 -- 172.21.15.39:0/2335223231 wait complete. 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 1 -- start start 2024-05-04T03:11:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 1 Processor -- start 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2246820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df2246d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558df2244930 con 0x558df1e722f0 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558df215e370 con 0x558df1e7a4d0 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558df2244ab0 con 0x558df1e9b1b0 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f18461a8740 10 client.0 _ll_register_callbacks cb 0x558df1eb2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f183cfba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2246820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f183cfba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2246820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60838/0 (socket says 172.21.15.39:60838) 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.774+0000 7f183cfba640 1 -- 172.21.15.39:0/1417661928 learned_addr learned my addr 172.21.15.39:0/1417661928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f183d7bb640 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f182ffff640 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df2246d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f183cfba640 1 -- 172.21.15.39:0/1417661928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 msgr2=0x558df224dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f183cfba640 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f183cfba640 1 -- 172.21.15.39:0/1417661928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 msgr2=0x558df2246d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f183cfba640 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df2246d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f183cfba640 1 -- 172.21.15.39:0/1417661928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df1f7f620 con 0x558df1e722f0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f183cfba640 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2246820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f183400bfb0 tx=0x7f1834004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d1b6ceab2b2b3b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f1817fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f1817fff640 1 -- 172.21.15.39:0/1417661928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183402c060 con 0x558df1e722f0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f18461a8740 1 -- 172.21.15.39:0/1417661928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18280047a0 con 0x558df1e722f0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f1817fff640 1 -- 172.21.15.39:0/1417661928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1834017030 con 0x558df1e722f0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f1817fff640 1 -- 172.21.15.39:0/1417661928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f183401c600 con 0x558df1e722f0 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f18461a8740 10 client.15450 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.775+0000 7f18461a8740 1 -- 172.21.15.39:0/1417661928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558df2250c50 con 0x558df1e722f0 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f1817fff640 1 -- 172.21.15.39:0/1417661928 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18280047a0 con 0x558df1e722f0 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f1817fff640 1 -- 172.21.15.39:0/1417661928 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f18340277a0 con 0x558df1e722f0 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f18461a8740 10 client.15450 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f18461a8740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f18461a8740 20 client.15450 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f18461a8740 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558df1e59d80 0x558df2254480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f18461a8740 4 client.15450 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f18461a8740 1 -- 172.21.15.39:0/1417661928 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df21af1e0 con 0x558df1e59d80 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.776+0000 7f182ffff640 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558df1e59d80 0x558df2254480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f182ffff640 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558df1e59d80 0x558df2254480 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1c2ae0c015e580 server_cookie=5df180273c64a8e1 in_seq=0 out_seq=0 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f1817fff640 10 client.15450 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f1817fff640 1 -- 172.21.15.39:0/1417661928 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558df21af1e0 con 0x558df1e59d80 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f1817fff640 10 client.15450 handle_command_reply: tid=0 2024-05-04T03:11:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f18461a8740 1 client.15450 shutdown 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f18461a8740 1 -- 172.21.15.39:0/1417661928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 msgr2=0x558df2246820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f18461a8740 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2246820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f183400bfb0 tx=0x7f1834004730 comp rx=0 tx=0).stop 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f18461a8740 1 -- 172.21.15.39:0/1417661928 shutdown_connections 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f18461a8740 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558df1e59d80 0x558df2254480 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f18461a8740 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f18461a8740 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df2246d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f1817fff640 0 client.15450 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.777+0000 7f18461a8740 1 --2- 172.21.15.39:0/1417661928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2246820 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.778+0000 7f18461a8740 1 -- 172.21.15.39:0/1417661928 >> 172.21.15.39:0/1417661928 conn(0x558df1e8f300 msgr2=0x558df223b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.778+0000 7f18461a8740 1 -- 172.21.15.39:0/1417661928 shutdown_connections 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.778+0000 7f18461a8740 1 -- 172.21.15.39:0/1417661928 wait complete. 2024-05-04T03:11:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.778+0000 7f18461a8740 20 client.15450 trim_cache size 0 max 16384 2024-05-04T03:11:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f18461a8740 1 Processor -- start 2024-05-04T03:11:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f18461a8740 1 -- start start 2024-05-04T03:11:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f18461a8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2241280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f18461a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df223b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f18461a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f18461a8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18381ab1d0 con 0x558df1e722f0 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f18461a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18381acdd0 con 0x558df1e7a4d0 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f18461a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18381ab8a0 con 0x558df1e9b1b0 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f183cfba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2241280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f183d7bb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.796+0000 7f183cfba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2241280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60850/0 (socket says 172.21.15.39:60850) 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f183cfba640 1 -- 172.21.15.39:0/149516623 learned_addr learned my addr 172.21.15.39:0/149516623 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f182ffff640 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df223b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f183d7bb640 1 -- 172.21.15.39:0/149516623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 msgr2=0x558df223b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f183d7bb640 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df223b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f183d7bb640 1 -- 172.21.15.39:0/149516623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 msgr2=0x558df2241280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f183d7bb640 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2241280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f183d7bb640 1 -- 172.21.15.39:0/149516623 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df223d7c0 con 0x558df1e9b1b0 2024-05-04T03:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f183cfba640 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2241280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f183d7bb640 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dbe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1830101330 tx=0x7f18300ca5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42416a1e2037513b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f181e7fc640 1 -- 172.21.15.39:0/149516623 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300cd070 con 0x558df1e9b1b0 2024-05-04T03:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f181e7fc640 1 -- 172.21.15.39:0/149516623 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1830036040 con 0x558df1e9b1b0 2024-05-04T03:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.797+0000 7f181e7fc640 1 -- 172.21.15.39:0/149516623 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300cb0a0 con 0x558df1e9b1b0 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 -- 172.21.15.39:0/149516623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 msgr2=0x558df224dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dbe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1830101330 tx=0x7f18300ca5b0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 -- 172.21.15.39:0/149516623 shutdown_connections 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224dbe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df223b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 --2- 172.21.15.39:0/149516623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df2241280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 -- 172.21.15.39:0/149516623 >> 172.21.15.39:0/149516623 conn(0x558df1e8f300 msgr2=0x558df223b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 -- 172.21.15.39:0/149516623 shutdown_connections 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.798+0000 7f18461a8740 1 -- 172.21.15.39:0/149516623 wait complete. 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 1 -- start start 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 1 Processor -- start 2024-05-04T03:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df224b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df224bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18381ac640 con 0x558df1e722f0 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18381abf70 con 0x558df1e7a4d0 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18381a91b0 con 0x558df1e9b1b0 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f18461a8740 10 client.0 _ll_register_callbacks cb 0x7f183400b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f183cfba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df224b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f183cfba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df224b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60854/0 (socket says 172.21.15.39:60854) 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.799+0000 7f183cfba640 1 -- 172.21.15.39:0/2732799917 learned_addr learned my addr 172.21.15.39:0/2732799917 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f183d7bb640 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f183cfba640 1 -- 172.21.15.39:0/2732799917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 msgr2=0x558df224ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f183cfba640 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f183cfba640 1 -- 172.21.15.39:0/2732799917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 msgr2=0x558df224bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f183cfba640 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df224bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f183cfba640 1 -- 172.21.15.39:0/2732799917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df224d280 con 0x558df1e722f0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f183cfba640 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df224b850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f183400b880 tx=0x7f1834005820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7afc7672bdcb617 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f1817fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f1817fff640 1 -- 172.21.15.39:0/2732799917 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834037070 con 0x558df1e722f0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f1817fff640 1 -- 172.21.15.39:0/2732799917 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f18340277f0 con 0x558df1e722f0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f18461a8740 1 -- 172.21.15.39:0/2732799917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1830036040 con 0x558df1e722f0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f1817fff640 1 -- 172.21.15.39:0/2732799917 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18340279f0 con 0x558df1e722f0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f18461a8740 10 client.15456 fetch_fsmap learned FSMap version 30 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.800+0000 7f18461a8740 1 -- 172.21.15.39:0/2732799917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558df224e580 con 0x558df1e722f0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f1817fff640 1 -- 172.21.15.39:0/2732799917 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1830036040 con 0x558df1e722f0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f1817fff640 1 -- 172.21.15.39:0/2732799917 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1834010a40 con 0x558df1e722f0 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f18461a8740 10 client.15456 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f18461a8740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f18461a8740 20 client.15456 populate_metadata read hostname 'smithi039' 2024-05-04T03:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f18461a8740 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558df1e2f1c0 0x558df22beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f18461a8740 4 client.15456 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f18461a8740 1 -- 172.21.15.39:0/2732799917 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558df21a9050 con 0x558df1e2f1c0 2024-05-04T03:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.801+0000 7f182ffff640 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558df1e2f1c0 0x558df22beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.802+0000 7f1817fff640 10 client.15456 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.802+0000 7f182ffff640 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558df1e2f1c0 0x558df22beea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35e622aa586e4f6 server_cookie=d877f367743dc0f3 in_seq=0 out_seq=0 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f1817fff640 1 -- 172.21.15.39:0/2732799917 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558df21a9050 con 0x558df1e2f1c0 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f1817fff640 10 client.15456 handle_command_reply: tid=0 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 client.15456 shutdown 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 -- 172.21.15.39:0/2732799917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 msgr2=0x558df224b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df224b850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f183400b880 tx=0x7f1834005820 comp rx=0 tx=0).stop 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 -- 172.21.15.39:0/2732799917 shutdown_connections 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558df1e2f1c0 0x558df22beea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558df1e9b1b0 0x558df224ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558df1e7a4d0 0x558df224bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f1817fff640 0 client.15456 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 --2- 172.21.15.39:0/2732799917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558df1e722f0 0x558df224b850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 -- 172.21.15.39:0/2732799917 >> 172.21.15.39:0/2732799917 conn(0x558df1e8f300 msgr2=0x558df2246d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 -- 172.21.15.39:0/2732799917 shutdown_connections 2024-05-04T03:11:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.811+0000 7f18461a8740 1 -- 172.21.15.39:0/2732799917 wait complete. 2024-05-04T03:11:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f18461a8740 20 client.15456 trim_cache size 0 max 16384 2024-05-04T03:11:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:11:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:11:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:11:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 -- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18200b2870 msgr2=0x7f18200b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18200b2870 0x7f18200b4d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x558df223de40 tx=0x7f183400b390 comp rx=0 tx=0).stop 2024-05-04T03:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 -- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 msgr2=0x7f18381a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 0x7f18381a1700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1830031040 tx=0x7f1830003ba0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 -- 172.21.15.39:0/3452657396 shutdown_connections 2024-05-04T03:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18200b2870 0x7f18200b4d80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1838108ab0 0x7f18381a1700 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1838103db0 0x7f183819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 --2- 172.21.15.39:0/3452657396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1838102b70 0x7f183819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 -- 172.21.15.39:0/3452657396 >> 172.21.15.39:0/3452657396 conn(0x7f18380fdd60 msgr2=0x7f1838109f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 -- 172.21.15.39:0/3452657396 shutdown_connections 2024-05-04T03:11:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:11:59.812+0000 7f183f21e640 1 -- 172.21.15.39:0/3452657396 wait complete. 2024-05-04T03:12:00.419 INFO:teuthology.orchestra.run.smithi039.stderr:1000000 of 3200000 tuples (31%) done (elapsed 10.39 s, remaining 22.87 s) 2024-05-04T03:12:00.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:00 smithi066 ceph-mon[33264]: pgmap v422: 97 pgs: 97 active+clean; 193 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 24 MiB/s wr, 109 op/s 2024-05-04T03:12:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:00 smithi167 ceph-mon[33614]: pgmap v422: 97 pgs: 97 active+clean; 193 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 24 MiB/s wr, 109 op/s 2024-05-04T03:12:00.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:00 smithi039 ceph-mon[29715]: pgmap v422: 97 pgs: 97 active+clean; 193 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 24 MiB/s wr, 109 op/s 2024-05-04T03:12:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:1100000 of 3200000 tuples (34%) done (elapsed 11.63 s, remaining 22.19 s) 2024-05-04T03:12:01.785 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-04T03:12:01.786 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.25 2024-05-04T03:12:01.786 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:12:02.074 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:12:02.466 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:02 smithi039 ceph-mon[29715]: pgmap v423: 97 pgs: 97 active+clean; 229 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 30 MiB/s wr, 95 op/s 2024-05-04T03:12:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:1200000 of 3200000 tuples (37%) done (elapsed 12.49 s, remaining 20.82 s) 2024-05-04T03:12:02.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:02 smithi066 ceph-mon[33264]: pgmap v423: 97 pgs: 97 active+clean; 229 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 30 MiB/s wr, 95 op/s 2024-05-04T03:12:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:02 smithi167 ceph-mon[33614]: pgmap v423: 97 pgs: 97 active+clean; 229 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 30 MiB/s wr, 95 op/s 2024-05-04T03:12:03.870 INFO:teuthology.orchestra.run.smithi039.stderr:1300000 of 3200000 tuples (40%) done (elapsed 13.84 s, remaining 20.23 s) 2024-05-04T03:12:04.566 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:04 smithi039 ceph-mon[29715]: pgmap v424: 97 pgs: 97 active+clean; 293 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 40 MiB/s wr, 37 op/s 2024-05-04T03:12:04.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:04 smithi066 ceph-mon[33264]: pgmap v424: 97 pgs: 97 active+clean; 293 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 40 MiB/s wr, 37 op/s 2024-05-04T03:12:04.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:04 smithi167 ceph-mon[33614]: pgmap v424: 97 pgs: 97 active+clean; 293 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 40 MiB/s wr, 37 op/s 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 -- 172.21.15.39:0/4210830179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac1034c0 msgr2=0x7f18ac1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 --2- 172.21.15.39:0/4210830179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac1034c0 0x7f18ac1038d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f189c0099c0 tx=0x7f189c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 -- 172.21.15.39:0/4210830179 shutdown_connections 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 --2- 172.21.15.39:0/4210830179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac1034c0 0x7f18ac1038d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 --2- 172.21.15.39:0/4210830179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac108740 0x7f18ac1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 --2- 172.21.15.39:0/4210830179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac107df0 0x7f18ac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 -- 172.21.15.39:0/4210830179 >> 172.21.15.39:0/4210830179 conn(0x7f18ac0fedc0 msgr2=0x7f18ac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 -- 172.21.15.39:0/4210830179 shutdown_connections 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.852+0000 7f18b1043700 1 -- 172.21.15.39:0/4210830179 wait complete. 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.853+0000 7f18b1043700 1 Processor -- start 2024-05-04T03:12:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.853+0000 7f18b1043700 1 -- start start 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.853+0000 7f18b1043700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac1034c0 0x7f18ac1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.853+0000 7f18b1043700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac107df0 0x7f18ac1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.853+0000 7f18b1043700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac108740 0x7f18ac19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.853+0000 7f18b1043700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac113220 con 0x7f18ac107df0 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.853+0000 7f18b1043700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1130a0 con 0x7f18ac108740 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.853+0000 7f18b1043700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18ac1133a0 con 0x7f18ac1034c0 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac107df0 0x7f18ac1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac107df0 0x7f18ac1a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60864/0 (socket says 172.21.15.39:60864) 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 -- 172.21.15.39:0/1754881561 learned_addr learned my addr 172.21.15.39:0/1754881561 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18ab59e700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac108740 0x7f18ac19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aad9d700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac1034c0 0x7f18ac1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 -- 172.21.15.39:0/1754881561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac1034c0 msgr2=0x7f18ac1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac1034c0 0x7f18ac1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 -- 172.21.15.39:0/1754881561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac108740 msgr2=0x7f18ac19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac108740 0x7f18ac19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 -- 172.21.15.39:0/1754881561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ac19b1d0 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f18aa59c700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac107df0 0x7f18ac1a0ee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f18a000c9e0 tx=0x7f18a000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.854+0000 7f1893fff700 1 -- 172.21.15.39:0/1754881561 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0007aa0 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.855+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ac19b4c0 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.855+0000 7f1893fff700 1 -- 172.21.15.39:0/1754881561 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a0007c40 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.855+0000 7f1893fff700 1 -- 172.21.15.39:0/1754881561 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0018400 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.855+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ac1a7750 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.856+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ac104970 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.858+0000 7f1893fff700 1 -- 172.21.15.39:0/1754881561 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f18a000f450 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.859+0000 7f1893fff700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18940b2910 0x7f18940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.859+0000 7f18aad9d700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18940b2910 0x7f18940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.859+0000 7f1893fff700 1 -- 172.21.15.39:0/1754881561 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f18a00e24c0 con 0x7f18ac107df0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.859+0000 7f18aad9d700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18940b2910 0x7f18940b4e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f189c000c00 tx=0x7f189c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:04.863+0000 7f1893fff700 1 -- 172.21.15.39:0/1754881561 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18a0014030 con 0x7f18ac107df0 2024-05-04T03:12:05.082 INFO:teuthology.orchestra.run.smithi039.stderr:1400000 of 3200000 tuples (43%) done (elapsed 15.06 s, remaining 19.36 s) 2024-05-04T03:12:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.206+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18ac19bfd0 con 0x7f18ac107df0 2024-05-04T03:12:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.207+0000 7f1893fff700 1 -- 172.21.15.39:0/1754881561 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11937 (secure 0 0 0) 0x7f18a00aa120 con 0x7f18ac107df0 2024-05-04T03:12:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:12:05.208 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:11:36.224039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24472,24506,24466,24490,24455]},"id":1}]} 2024-05-04T03:12:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18940b2910 msgr2=0x7f18940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18940b2910 0x7f18940b4e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f189c000c00 tx=0x7f189c01a040 comp rx=0 tx=0).stop 2024-05-04T03:12:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac107df0 msgr2=0x7f18ac1a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac107df0 0x7f18ac1a0ee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f18a000c9e0 tx=0x7f18a000ccf0 comp rx=0 tx=0).stop 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 shutdown_connections 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f18940b2910 0x7f18940b4e20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac108740 0x7f18ac19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18ac1034c0 0x7f18ac1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 --2- 172.21.15.39:0/1754881561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18ac107df0 0x7f18ac1a0ee0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 >> 172.21.15.39:0/1754881561 conn(0x7f18ac0fedc0 msgr2=0x7f18ac06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 shutdown_connections 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:05.209+0000 7f18b1043700 1 -- 172.21.15.39:0/1754881561 wait complete. 2024-05-04T03:12:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-05-04T03:12:05.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:05 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1754881561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:05.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:05 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1754881561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:05.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:05 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1754881561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:1500000 of 3200000 tuples (46%) done (elapsed 15.89 s, remaining 18.00 s) 2024-05-04T03:12:06.129 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 89.25 --await-for 42.0' 2024-05-04T03:12:06.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 89.25 --await-for 42.0 2024-05-04T03:12:06.416 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:12:06.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:06 smithi039 ceph-mon[29715]: pgmap v425: 97 pgs: 97 active+clean; 321 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 39 MiB/s wr, 33 op/s 2024-05-04T03:12:06.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:06 smithi066 ceph-mon[33264]: pgmap v425: 97 pgs: 97 active+clean; 321 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 39 MiB/s wr, 33 op/s 2024-05-04T03:12:06.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:06 smithi167 ceph-mon[33614]: pgmap v425: 97 pgs: 97 active+clean; 321 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 39 MiB/s wr, 33 op/s 2024-05-04T03:12:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:1600000 of 3200000 tuples (50%) done (elapsed 17.19 s, remaining 17.19 s) 2024-05-04T03:12:08.002 INFO:teuthology.orchestra.run.smithi039.stderr:1700000 of 3200000 tuples (53%) done (elapsed 17.98 s, remaining 15.86 s) 2024-05-04T03:12:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 -- 172.21.15.39:0/3669518860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c1034c0 msgr2=0x7f151c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 --2- 172.21.15.39:0/3669518860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c1034c0 0x7f151c1038d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f15100099c0 tx=0x7f1510009cd0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 -- 172.21.15.39:0/3669518860 shutdown_connections 2024-05-04T03:12:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 --2- 172.21.15.39:0/3669518860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c1034c0 0x7f151c1038d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 --2- 172.21.15.39:0/3669518860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f151c108740 0x7f151c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 --2- 172.21.15.39:0/3669518860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c107df0 0x7f151c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 -- 172.21.15.39:0/3669518860 >> 172.21.15.39:0/3669518860 conn(0x7f151c0fedc0 msgr2=0x7f151c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 -- 172.21.15.39:0/3669518860 shutdown_connections 2024-05-04T03:12:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.181+0000 7f1522ccc700 1 -- 172.21.15.39:0/3669518860 wait complete. 2024-05-04T03:12:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.182+0000 7f1522ccc700 1 Processor -- start 2024-05-04T03:12:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.182+0000 7f1522ccc700 1 -- start start 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f1522ccc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f151c1034c0 0x7f151c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f1522ccc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107df0 0x7f151c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f1522ccc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c108740 0x7f151c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f1522ccc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f151c113220 con 0x7f151c107df0 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f1522ccc700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1130a0 con 0x7f151c1034c0 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f1522ccc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1133a0 con 0x7f151c108740 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f151bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107df0 0x7f151c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f151bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107df0 0x7f151c19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51772/0 (socket says 172.21.15.39:51772) 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f151bfff700 1 -- 172.21.15.39:0/4246475180 learned_addr learned my addr 172.21.15.39:0/4246475180 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f1520a68700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f151c1034c0 0x7f151c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f1521269700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c108740 0x7f151c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f151bfff700 1 -- 172.21.15.39:0/4246475180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c108740 msgr2=0x7f151c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f151bfff700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c108740 0x7f151c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f151bfff700 1 -- 172.21.15.39:0/4246475180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f151c1034c0 msgr2=0x7f151c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f151bfff700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f151c1034c0 0x7f151c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.183+0000 7f151bfff700 1 -- 172.21.15.39:0/4246475180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c1a1df0 con 0x7f151c107df0 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.184+0000 7f151bfff700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107df0 0x7f151c19d4c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f150800d950 tx=0x7f150800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.184+0000 7f1519ffb700 1 -- 172.21.15.39:0/4246475180 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508016070 con 0x7f151c107df0 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.184+0000 7f1519ffb700 1 -- 172.21.15.39:0/4246475180 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1508004830 con 0x7f151c107df0 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.184+0000 7f1519ffb700 1 -- 172.21.15.39:0/4246475180 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508005420 con 0x7f151c107df0 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.184+0000 7f1522ccc700 1 -- 172.21.15.39:0/4246475180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c1a20e0 con 0x7f151c107df0 2024-05-04T03:12:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.185+0000 7f1522ccc700 1 -- 172.21.15.39:0/4246475180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c1a9a00 con 0x7f151c107df0 2024-05-04T03:12:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.187+0000 7f1519ffb700 1 -- 172.21.15.39:0/4246475180 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7f1508004220 con 0x7f151c107df0 2024-05-04T03:12:08.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.188+0000 7f1519ffb700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f150c0b2a60 0x7f150c0b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.188+0000 7f1519ffb700 1 -- 172.21.15.39:0/4246475180 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f15080e1fa0 con 0x7f151c107df0 2024-05-04T03:12:08.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.188+0000 7f1520a68700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f150c0b2a60 0x7f150c0b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.189+0000 7f1520a68700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f150c0b2a60 0x7f150c0b4f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15100058e0 tx=0x7f1510005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.189+0000 7f15316e0b80 1 Processor -- start 2024-05-04T03:12:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.189+0000 7f15316e0b80 1 -- start start 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af42a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42fb0 0x55db7af4d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af4d970 0x55db7af4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f15316e0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55db7ae659f0 con 0x55db7af4d970 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f15316e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55db7ae04950 con 0x55db7af42660 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f15316e0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55db7ae619b0 con 0x55db7af42fb0 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f1520a68700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af42a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f151bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42fb0 0x55db7af4d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f1520a68700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af42a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43634/0 (socket says 172.21.15.39:43634) 2024-05-04T03:12:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f1520a68700 1 -- 172.21.15.39:0/3102405027 learned_addr learned my addr 172.21.15.39:0/3102405027 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f1521269700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af4d970 0x55db7af4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f151bfff700 1 -- 172.21.15.39:0/3102405027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 msgr2=0x55db7af42a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f151bfff700 1 --2- 172.21.15.39:0/3102405027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af42a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f151bfff700 1 -- 172.21.15.39:0/3102405027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af4d970 msgr2=0x55db7af4fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f151bfff700 1 --2- 172.21.15.39:0/3102405027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af4d970 0x55db7af4fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f151bfff700 1 -- 172.21.15.39:0/3102405027 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db7adb2a60 con 0x55db7af42fb0 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.190+0000 7f1520a68700 1 --2- 172.21.15.39:0/3102405027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af42a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.191+0000 7f151bfff700 1 --2- 172.21.15.39:0/3102405027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42fb0 0x55db7af4d430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15080df960 tx=0x7f15080a9000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9808f2ef86e92f39 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.191+0000 7f1506ffd700 1 -- 172.21.15.39:0/3102405027 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080a9470 con 0x55db7af42fb0 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.191+0000 7f1506ffd700 1 -- 172.21.15.39:0/3102405027 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15080ad820 con 0x55db7af42fb0 2024-05-04T03:12:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.191+0000 7f1506ffd700 1 -- 172.21.15.39:0/3102405027 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080b0ba0 con 0x55db7af42fb0 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 -- 172.21.15.39:0/3102405027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42fb0 msgr2=0x55db7af4d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3102405027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42fb0 0x55db7af4d430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15080df960 tx=0x7f15080a9000 comp rx=0 tx=0).stop 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 -- 172.21.15.39:0/3102405027 shutdown_connections 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3102405027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af4d970 0x55db7af4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3102405027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af42a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3102405027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42fb0 0x55db7af4d430 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 -- 172.21.15.39:0/3102405027 >> 172.21.15.39:0/3102405027 conn(0x55db7adc5fd0 msgr2=0x55db7af3e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 -- 172.21.15.39:0/3102405027 shutdown_connections 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.192+0000 7f15316e0b80 1 -- 172.21.15.39:0/3102405027 wait complete. 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.193+0000 7f15316e0b80 1 -- start start 2024-05-04T03:12:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.193+0000 7f15316e0b80 1 Processor -- start 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.193+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af4a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.193+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af4ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 0x55db7af4b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f15316e0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55db7af50700 con 0x55db7af42fb0 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f15316e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55db7adcbdc0 con 0x55db7af42660 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af4ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f15316e0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55db7af50880 con 0x55db7af4d970 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af4ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51790/0 (socket says 172.21.15.39:51790) 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 -- 172.21.15.39:0/3065213474 learned_addr learned my addr 172.21.15.39:0/3065213474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f15316e0b80 10 client.0 _ll_register_callbacks cb 0x55db7adfdab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f1520a68700 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af4a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f1521269700 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 0x55db7af4b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 -- 172.21.15.39:0/3065213474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 msgr2=0x55db7af4b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 0x55db7af4b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 -- 172.21.15.39:0/3065213474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 msgr2=0x55db7af4a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af4a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 -- 172.21.15.39:0/3065213474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db7ae01820 con 0x55db7af42fb0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f151bfff700 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af4ae80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f15080a66e0 tx=0x7f15080dfde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45440186d2719206 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f14fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f14fbfff700 1 -- 172.21.15.39:0/3065213474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080c7070 con 0x55db7af42fb0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f14fbfff700 1 -- 172.21.15.39:0/3065213474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f15080ca2d0 con 0x55db7af42fb0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.194+0000 7f15316e0b80 1 -- 172.21.15.39:0/3065213474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15080ad820 con 0x55db7af42fb0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f14fbfff700 1 -- 172.21.15.39:0/3065213474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f15080b2050 con 0x55db7af42fb0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f15316e0b80 10 client.15474 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f15316e0b80 1 -- 172.21.15.39:0/3065213474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55db7af63b40 con 0x55db7af42fb0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f14fbfff700 1 -- 172.21.15.39:0/3065213474 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15080ad820 con 0x55db7af42fb0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f14fbfff700 1 -- 172.21.15.39:0/3065213474 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f15080a3b90 con 0x55db7af42fb0 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f15316e0b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f15316e0b80 10 client.15474 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f15316e0b80 20 client.15474 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:08.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55db7af66010 0x55db7af68430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f15316e0b80 4 client.15474 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:12:08.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f15316e0b80 1 -- 172.21.15.39:0/3065213474 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db7ae733d0 con 0x55db7af66010 2024-05-04T03:12:08.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.195+0000 7f1520a68700 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55db7af66010 0x55db7af68430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.196+0000 7f14fbfff700 10 client.15474 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:08.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.196+0000 7f1520a68700 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55db7af66010 0x55db7af68430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceba91aeba7dcc7b server_cookie=6b5a535640c2cf38 in_seq=0 out_seq=0 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f14fbfff700 1 -- 172.21.15.39:0/3065213474 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db7ae733d0 con 0x55db7af66010 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f14fbfff700 10 client.15474 handle_command_reply: tid=0 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 client.15474 shutdown 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 -- 172.21.15.39:0/3065213474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 msgr2=0x55db7af4ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af4ae80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f15080a66e0 tx=0x7f15080dfde0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 -- 172.21.15.39:0/3065213474 shutdown_connections 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55db7af66010 0x55db7af68430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 0x55db7af4b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42660 0x55db7af4a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f14fbfff700 0 client.15474 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 --2- 172.21.15.39:0/3065213474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af4ae80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 -- 172.21.15.39:0/3065213474 >> 172.21.15.39:0/3065213474 conn(0x55db7adc5fd0 msgr2=0x55db7af44250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 -- 172.21.15.39:0/3065213474 shutdown_connections 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.197+0000 7f15316e0b80 1 -- 172.21.15.39:0/3065213474 wait complete. 2024-05-04T03:12:08.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.198+0000 7f15316e0b80 20 client.15474 trim_cache size 0 max 16384 2024-05-04T03:12:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.221+0000 7f15316e0b80 1 Processor -- start 2024-05-04T03:12:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.221+0000 7f15316e0b80 1 -- start start 2024-05-04T03:12:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42660 0x55db7af44d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af4d970 0x55db7af4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f15316e0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1ac8a0 con 0x55db7af42fb0 2024-05-04T03:12:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f15316e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1ae4a0 con 0x55db7af4d970 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f15316e0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1acf70 con 0x55db7af42660 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f151bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f151bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51804/0 (socket says 172.21.15.39:51804) 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f151bfff700 1 -- 172.21.15.39:0/1634446324 learned_addr learned my addr 172.21.15.39:0/1634446324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f1521269700 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af4d970 0x55db7af4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f1520a68700 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42660 0x55db7af44d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f151bfff700 1 -- 172.21.15.39:0/1634446324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42660 msgr2=0x55db7af44d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f151bfff700 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42660 0x55db7af44d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f151bfff700 1 -- 172.21.15.39:0/1634446324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af4d970 msgr2=0x55db7af4b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f151bfff700 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af4d970 0x55db7af4b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.222+0000 7f151bfff700 1 -- 172.21.15.39:0/1634446324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db7af43570 con 0x55db7af42fb0 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.223+0000 7f1521269700 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af4d970 0x55db7af4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:12:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.223+0000 7f151bfff700 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af623b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1508022f00 tx=0x7f15080a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a774039cc8afc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:08.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.223+0000 7f15067fc700 1 -- 172.21.15.39:0/1634446324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080c7070 con 0x55db7af42fb0 2024-05-04T03:12:08.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.223+0000 7f15067fc700 1 -- 172.21.15.39:0/1634446324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15080ad820 con 0x55db7af42fb0 2024-05-04T03:12:08.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.223+0000 7f15067fc700 1 -- 172.21.15.39:0/1634446324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080a5160 con 0x55db7af42fb0 2024-05-04T03:12:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 -- 172.21.15.39:0/1634446324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 msgr2=0x55db7af623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af623b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1508022f00 tx=0x7f15080a9000 comp rx=0 tx=0).stop 2024-05-04T03:12:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 -- 172.21.15.39:0/1634446324 shutdown_connections 2024-05-04T03:12:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af4d970 0x55db7af4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af42660 0x55db7af44d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 --2- 172.21.15.39:0/1634446324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42fb0 0x55db7af623b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 -- 172.21.15.39:0/1634446324 >> 172.21.15.39:0/1634446324 conn(0x55db7adc5fd0 msgr2=0x55db7af3d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:08.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 -- 172.21.15.39:0/1634446324 shutdown_connections 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.224+0000 7f15316e0b80 1 -- 172.21.15.39:0/1634446324 wait complete. 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 1 -- start start 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 1 Processor -- start 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42660 0x55db7af69830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42fb0 0x55db7af69d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 0x55db7afcf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1add10 con 0x55db7af42660 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1ad640 con 0x55db7af42fb0 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f151c10e710 con 0x55db7af4d970 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f15316e0b80 10 client.0 _ll_register_callbacks cb 0x55db7adfdab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f1520a68700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42660 0x55db7af69830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f1520a68700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42660 0x55db7af69830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51808/0 (socket says 172.21.15.39:51808) 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f1520a68700 1 -- 172.21.15.39:0/876780814 learned_addr learned my addr 172.21.15.39:0/876780814 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:08.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.225+0000 7f151bfff700 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42fb0 0x55db7af69d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f1521269700 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 0x55db7afcf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f1520a68700 1 -- 172.21.15.39:0/876780814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 msgr2=0x55db7afcf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f1520a68700 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 0x55db7afcf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f1520a68700 1 -- 172.21.15.39:0/876780814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42fb0 msgr2=0x55db7af69d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f1520a68700 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42fb0 0x55db7af69d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f1520a68700 1 -- 172.21.15.39:0/876780814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db7af500f0 con 0x55db7af42660 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f1520a68700 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42660 0x55db7af69830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1510009750 tx=0x7f1510009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4d913015b92dc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f14fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f14fbfff700 1 -- 172.21.15.39:0/876780814 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510003e70 con 0x55db7af42660 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f14fbfff700 1 -- 172.21.15.39:0/876780814 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1510036150 con 0x55db7af42660 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.226+0000 7f15316e0b80 1 -- 172.21.15.39:0/876780814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15080ad820 con 0x55db7af42660 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f14fbfff700 1 -- 172.21.15.39:0/876780814 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f151000be50 con 0x55db7af42660 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f15316e0b80 10 client.15480 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:08.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f15316e0b80 1 -- 172.21.15.39:0/876780814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55db7afd0af0 con 0x55db7af42660 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f14fbfff700 1 -- 172.21.15.39:0/876780814 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15100367e0 con 0x55db7af42660 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f14fbfff700 1 -- 172.21.15.39:0/876780814 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1510022070 con 0x55db7af42660 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f15316e0b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f15316e0b80 10 client.15480 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f15316e0b80 20 client.15480 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f15316e0b80 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55db7afd1310 0x55db7afd3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f15316e0b80 4 client.15480 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.25, "await_for": 42.0}] 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.227+0000 7f15316e0b80 1 -- 172.21.15.39:0/876780814 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.25, "await_for": 42.0}) -- 0x7f15080ad820 con 0x55db7afd1310 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.228+0000 7f151bfff700 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55db7afd1310 0x55db7afd3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.228+0000 7f151bfff700 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55db7afd1310 0x55db7afd3730 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aafe5fbd9891dee server_cookie=4f166a1754c95e8 in_seq=0 out_seq=0 2024-05-04T03:12:08.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.228+0000 7f14fbfff700 10 client.15480 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:08.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:08 smithi039 ceph-mon[29715]: pgmap v426: 97 pgs: 97 active+clean; 365 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 42 MiB/s wr, 36 op/s 2024-05-04T03:12:08.567 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:08 smithi066 ceph-mon[33264]: pgmap v426: 97 pgs: 97 active+clean; 365 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 42 MiB/s wr, 36 op/s 2024-05-04T03:12:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:08 smithi167 ceph-mon[33614]: pgmap v426: 97 pgs: 97 active+clean; 365 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 42 MiB/s wr, 36 op/s 2024-05-04T03:12:08.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f14fbfff700 1 -- 172.21.15.39:0/876780814 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7f15080ad820 con 0x55db7afd1310 2024-05-04T03:12:08.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f14fbfff700 10 client.15480 handle_command_reply: tid=0 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 client.15480 shutdown 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 -- 172.21.15.39:0/876780814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42660 msgr2=0x55db7af69830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42660 0x55db7af69830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1510009750 tx=0x7f1510009cd0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 -- 172.21.15.39:0/876780814 shutdown_connections 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55db7afd1310 0x55db7afd3730 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db7af42660 0x55db7af69830 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55db7af4d970 0x55db7afcf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f14fbfff700 0 client.15480 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 --2- 172.21.15.39:0/876780814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55db7af42fb0 0x55db7af69d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.733+0000 7f15316e0b80 1 -- 172.21.15.39:0/876780814 >> 172.21.15.39:0/876780814 conn(0x55db7adc5fd0 msgr2=0x55db7af3e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f15316e0b80 1 -- 172.21.15.39:0/876780814 shutdown_connections 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f15316e0b80 1 -- 172.21.15.39:0/876780814 wait complete. 2024-05-04T03:12:08.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f15316e0b80 20 client.15480 trim_cache size 0 max 16384 2024-05-04T03:12:08.740 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:12:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-05-04T03:12:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:12:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 2, 2024-05-04T03:12:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:12:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "80d64dad": { 2024-05-04T03:12:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 2, 2024-05-04T03:12:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 89.2, 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:08.741 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:08.742 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:08.742 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:12:08.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 -- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f150c0b2a60 msgr2=0x7f150c0b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f150c0b2a60 0x7f150c0b4f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15100058e0 tx=0x7f1510005fb0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 -- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107df0 msgr2=0x7f151c19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107df0 0x7f151c19d4c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f150800d950 tx=0x7f150800dc60 comp rx=0 tx=0).stop 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 -- 172.21.15.39:0/4246475180 shutdown_connections 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f150c0b2a60 0x7f150c0b4f70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c108740 0x7f151c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f151c1034c0 0x7f151c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 --2- 172.21.15.39:0/4246475180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107df0 0x7f151c19d4c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.734+0000 7f1522ccc700 1 -- 172.21.15.39:0/4246475180 >> 172.21.15.39:0/4246475180 conn(0x7f151c0fedc0 msgr2=0x7f151c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.735+0000 7f1522ccc700 1 -- 172.21.15.39:0/4246475180 shutdown_connections 2024-05-04T03:12:08.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:08.735+0000 7f1522ccc700 1 -- 172.21.15.39:0/4246475180 wait complete. 2024-05-04T03:12:09.533 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '80d64dad', quiesce took 7.7 seconds. Will release after: 29.25 2024-05-04T03:12:10.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:10 smithi066 ceph-mon[33264]: pgmap v427: 97 pgs: 97 active+clean; 402 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 44 MiB/s wr, 38 op/s 2024-05-04T03:12:10.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:10 smithi167 ceph-mon[33614]: pgmap v427: 97 pgs: 97 active+clean; 402 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 44 MiB/s wr, 38 op/s 2024-05-04T03:12:10.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:10 smithi039 ceph-mon[29715]: pgmap v427: 97 pgs: 97 active+clean; 402 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 44 MiB/s wr, 38 op/s 2024-05-04T03:12:12.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:12 smithi066 ceph-mon[33264]: pgmap v428: 97 pgs: 97 active+clean; 426 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 MiB/s wr, 33 op/s 2024-05-04T03:12:12.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:12 smithi167 ceph-mon[33614]: pgmap v428: 97 pgs: 97 active+clean; 426 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 MiB/s wr, 33 op/s 2024-05-04T03:12:12.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:12 smithi039 ceph-mon[29715]: pgmap v428: 97 pgs: 97 active+clean; 426 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 MiB/s wr, 33 op/s 2024-05-04T03:12:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3331944405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 msgr2=0x7fa7ec10aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3331944405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 0x7fa7ec10aeb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009ca0 tx=0x7fa7e402a8c0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3331944405 shutdown_connections 2024-05-04T03:12:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3331944405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 0x7fa7ec10aeb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3331944405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7ec103d90 0x7fa7ec1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3331944405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7ec102b50 0x7fa7ec103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3331944405 >> 172.21.15.39:0/3331944405 conn(0x7fa7ec0fdd40 msgr2=0x7fa7ec1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3331944405 shutdown_connections 2024-05-04T03:12:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.926+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3331944405 wait complete. 2024-05-04T03:12:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.927+0000 7fa7f27ea640 1 Processor -- start 2024-05-04T03:12:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.927+0000 7fa7f27ea640 1 -- start start 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.927+0000 7fa7f27ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7ec102b50 0x7fa7ec199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.927+0000 7fa7f27ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7ec103d90 0x7fa7ec199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.927+0000 7fa7f27ea640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 0x7fa7ec1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f27ea640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec10e580 con 0x7fa7ec108a90 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f27ea640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec10e400 con 0x7fa7ec102b50 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f27ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec10e700 con 0x7fa7ec103d90 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 0x7fa7ec1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 0x7fa7ec1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51820/0 (socket says 172.21.15.39:51820) 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 -- 172.21.15.39:0/3026120819 learned_addr learned my addr 172.21.15.39:0/3026120819 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7ec102b50 0x7fa7ec199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7ec103d90 0x7fa7ec199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 -- 172.21.15.39:0/3026120819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7ec103d90 msgr2=0x7fa7ec199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7ec103d90 0x7fa7ec199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 -- 172.21.15.39:0/3026120819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7ec102b50 msgr2=0x7fa7ec199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7ec102b50 0x7fa7ec199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 -- 172.21.15.39:0/3026120819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec06a4f0 con 0x7fa7ec108a90 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7ec103d90 0x7fa7ec199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7f0d87640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 0x7fa7ec1a05f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4005e30 tx=0x7fa7e4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.928+0000 7fa7e97fa640 1 -- 172.21.15.39:0/3026120819 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4032e90 con 0x7fa7ec108a90 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.929+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3026120819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ec06a780 con 0x7fa7ec108a90 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.929+0000 7fa7e97fa640 1 -- 172.21.15.39:0/3026120819 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7e40044d0 con 0x7fa7ec108a90 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.929+0000 7fa7e97fa640 1 -- 172.21.15.39:0/3026120819 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4030ce0 con 0x7fa7ec108a90 2024-05-04T03:12:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.929+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3026120819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ec06acd0 con 0x7fa7ec108a90 2024-05-04T03:12:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.931+0000 7fa7f9775740 1 Processor -- start 2024-05-04T03:12:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.931+0000 7fa7e97fa640 1 -- 172.21.15.39:0/3026120819 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149703+0+0 (secure 0 0 0) 0x7fa7e4004040 con 0x7fa7ec108a90 2024-05-04T03:12:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7f9775740 1 -- start start 2024-05-04T03:12:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613bd1b0 0x55686175ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686175efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55686139c4d0 0x556861765ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7f9775740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5568616c35d0 con 0x5568613942f0 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7f9775740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55686167c890 con 0x5568613bd1b0 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7f9775740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5568616bf590 con 0x55686139c4d0 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686175efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686175efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51822/0 (socket says 172.21.15.39:51822) 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 -- 172.21.15.39:0/3774693753 learned_addr learned my addr 172.21.15.39:0/3774693753 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613bd1b0 0x55686175ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55686139c4d0 0x556861765ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 -- 172.21.15.39:0/3774693753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55686139c4d0 msgr2=0x556861765ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55686139c4d0 0x556861765ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 -- 172.21.15.39:0/3774693753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613bd1b0 msgr2=0x55686175ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613bd1b0 0x55686175ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 -- 172.21.15.39:0/3774693753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568613d5210 con 0x5568613942f0 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55686139c4d0 0x556861765ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7e97fa640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7d40b2870 0x7fa7d40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.932+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686175efe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009c60 tx=0x7fa7dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630a67f3743554b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.933+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7d40b2870 0x7fa7d40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.933+0000 7fa7e97fa640 1 -- 172.21.15.39:0/3026120819 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa7e4103770 con 0x7fa7ec108a90 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.933+0000 7fa7d27fc640 1 -- 172.21.15.39:0/3774693753 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc004220 con 0x5568613942f0 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.933+0000 7fa7d27fc640 1 -- 172.21.15.39:0/3774693753 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7dc0043c0 con 0x5568613942f0 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.933+0000 7fa7d27fc640 1 -- 172.21.15.39:0/3774693753 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc004c60 con 0x5568613942f0 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 -- 172.21.15.39:0/3774693753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 msgr2=0x55686175efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7d40b2870 0x7fa7d40b4d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00058c0 tx=0x7fa7e000c510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686175efe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009c60 tx=0x7fa7dc011810 comp rx=0 tx=0).stop 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 -- 172.21.15.39:0/3774693753 shutdown_connections 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55686139c4d0 0x556861765ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686175efe0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 --2- 172.21.15.39:0/3774693753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613bd1b0 0x55686175ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 -- 172.21.15.39:0/3774693753 >> 172.21.15.39:0/3774693753 conn(0x5568613b1300 msgr2=0x55686175a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 -- 172.21.15.39:0/3774693753 shutdown_connections 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 -- 172.21.15.39:0/3774693753 wait complete. 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.934+0000 7fa7f9775740 1 -- start start 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f9775740 1 Processor -- start 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613942f0 0x556861768640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 0x556861768b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613bd1b0 0x55686176fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f9775740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556861766930 con 0x5568613bd1b0 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f9775740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556861680370 con 0x55686139c4d0 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f9775740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556861766ab0 con 0x5568613942f0 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f9775740 10 client.0 _ll_register_callbacks cb 0x5568613d4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f0d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613bd1b0 0x55686176fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f0d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613bd1b0 0x55686176fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51824/0 (socket says 172.21.15.39:51824) 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f0d87640 1 -- 172.21.15.39:0/1813914283 learned_addr learned my addr 172.21.15.39:0/1813914283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 0x556861768b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613942f0 0x556861768640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f0d87640 1 -- 172.21.15.39:0/1813914283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613942f0 msgr2=0x556861768640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f0d87640 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613942f0 0x556861768640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f0d87640 1 -- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 msgr2=0x556861768b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f0d87640 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 0x556861768b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7f0d87640 1 -- 172.21.15.39:0/1813914283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568614a1620 con 0x5568613bd1b0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.935+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 0x556861768b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f0d87640 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613bd1b0 0x55686176fae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40c6dd0 tx=0x7fa7e40ca580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fc0552dd9576be server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1813914283 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40cad50 con 0x5568613bd1b0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1813914283 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa7e4101380 con 0x5568613bd1b0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 1 -- 172.21.15.39:0/1813914283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7dc0043c0 con 0x5568613bd1b0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1813914283 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7e40cf100 con 0x5568613bd1b0 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 10 client.15492 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 1 -- 172.21.15.39:0/1813914283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556861766930 con 0x5568613bd1b0 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1813914283 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7dc0043c0 con 0x5568613bd1b0 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1813914283 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa7e40cafb0 con 0x5568613bd1b0 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 10 client.15492 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 20 client.15492 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55686137bd80 0x556861776300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 4 client.15492 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.936+0000 7fa7f9775740 1 -- 172.21.15.39:0/1813914283 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568616d11e0 con 0x55686137bd80 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.937+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55686137bd80 0x556861776300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.937+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55686137bd80 0x556861776300 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5b5417187176c8 server_cookie=31efc18c1f5ad6f in_seq=0 out_seq=0 2024-05-04T03:12:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.937+0000 7fa7cb7fe640 10 client.15492 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1813914283 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568616d11e0 con 0x55686137bd80 2024-05-04T03:12:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7cb7fe640 10 client.15492 handle_command_reply: tid=0 2024-05-04T03:12:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 client.15492 shutdown 2024-05-04T03:12:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 -- 172.21.15.39:0/1813914283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613bd1b0 msgr2=0x55686176fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613bd1b0 0x55686176fae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40c6dd0 tx=0x7fa7e40ca580 comp rx=0 tx=0).stop 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 -- 172.21.15.39:0/1813914283 shutdown_connections 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55686137bd80 0x556861776300 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613bd1b0 0x55686176fae0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 0x556861768b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1813914283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613942f0 0x556861768640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 -- 172.21.15.39:0/1813914283 >> 172.21.15.39:0/1813914283 conn(0x5568613b1300 msgr2=0x55686175d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7cb7fe640 0 client.15492 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 -- 172.21.15.39:0/1813914283 shutdown_connections 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 1 -- 172.21.15.39:0/1813914283 wait complete. 2024-05-04T03:12:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.938+0000 7fa7f9775740 20 client.15492 trim_cache size 0 max 16384 2024-05-04T03:12:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f9775740 1 Processor -- start 2024-05-04T03:12:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f9775740 1 -- start start 2024-05-04T03:12:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613942f0 0x556861762a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55686139c4d0 0x55686176fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 0x556861762f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f9775740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1ab1b0 con 0x55686139c4d0 2024-05-04T03:12:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f9775740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1acdb0 con 0x5568613942f0 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f9775740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1ab880 con 0x5568613bd1b0 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7eb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55686139c4d0 0x55686176fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7eb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55686139c4d0 0x55686176fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51830/0 (socket says 172.21.15.39:51830) 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7eb7fe640 1 -- 172.21.15.39:0/3713379418 learned_addr learned my addr 172.21.15.39:0/3713379418 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613942f0 0x556861762a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.954+0000 7fa7f0d87640 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 0x556861762f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7eb7fe640 1 -- 172.21.15.39:0/3713379418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 msgr2=0x556861762f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 0x556861762f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7eb7fe640 1 -- 172.21.15.39:0/3713379418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613942f0 msgr2=0x556861762a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613942f0 0x556861762a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7eb7fe640 1 -- 172.21.15.39:0/3713379418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55686175fc60 con 0x55686139c4d0 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613942f0 0x556861762a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55686139c4d0 0x55686176fa00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0011650 tx=0x7fa7e0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec34ed2d5cc29cb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7d1ffb640 1 -- 172.21.15.39:0/3713379418 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e001f750 con 0x55686139c4d0 2024-05-04T03:12:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7d1ffb640 1 -- 172.21.15.39:0/3713379418 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7e001fd80 con 0x55686139c4d0 2024-05-04T03:12:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.955+0000 7fa7d1ffb640 1 -- 172.21.15.39:0/3713379418 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0020d20 con 0x55686139c4d0 2024-05-04T03:12:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 -- 172.21.15.39:0/3713379418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55686139c4d0 msgr2=0x55686176fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55686139c4d0 0x55686176fa00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0011650 tx=0x7fa7e0014040 comp rx=0 tx=0).stop 2024-05-04T03:12:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 -- 172.21.15.39:0/3713379418 shutdown_connections 2024-05-04T03:12:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 0x556861762f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55686139c4d0 0x55686176fa00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 --2- 172.21.15.39:0/3713379418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5568613942f0 0x556861762a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 -- 172.21.15.39:0/3713379418 >> 172.21.15.39:0/3713379418 conn(0x5568613b1300 msgr2=0x556861754f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 -- 172.21.15.39:0/3713379418 shutdown_connections 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 -- 172.21.15.39:0/3713379418 wait complete. 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 -- start start 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.956+0000 7fa7f9775740 1 Processor -- start 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686176c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 0x55686176cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f9775740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 0x5568617dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f9775740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1ac620 con 0x5568613942f0 2024-05-04T03:12:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f9775740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1abf50 con 0x55686139c4d0 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f9775740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec1a9190 con 0x5568613bd1b0 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f9775740 10 client.0 _ll_register_callbacks cb 0x7fa7e4030f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686176c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686176c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51832/0 (socket says 172.21.15.39:51832) 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 -- 172.21.15.39:0/1706101344 learned_addr learned my addr 172.21.15.39:0/1706101344 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f0d87640 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 0x5568617dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 -- 172.21.15.39:0/1706101344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 msgr2=0x5568617dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 0x5568617dc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 -- 172.21.15.39:0/1706101344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 msgr2=0x55686176cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 0x55686176cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 -- 172.21.15.39:0/1706101344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55686176fb50 con 0x5568613942f0 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7ebfff640 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686176c650 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009250 tx=0x7fa7dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a98878d5d754665 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1706101344 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc032060 con 0x5568613942f0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1706101344 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa7dc0054c0 con 0x5568613942f0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.957+0000 7fa7f9775740 1 -- 172.21.15.39:0/1706101344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e001fd80 con 0x5568613942f0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1706101344 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7dc01d040 con 0x5568613942f0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7f9775740 10 client.15498 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7f9775740 1 -- 172.21.15.39:0/1706101344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568617ddaf0 con 0x5568613942f0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1706101344 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7e001fd80 con 0x5568613942f0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1706101344 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa7dc02cce0 con 0x5568613942f0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7f9775740 10 client.15498 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7f9775740 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7f9775740 20 client.15498 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5568613511c0 0x5568617e0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7f9775740 4 client.15498 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7f9775740 1 -- 172.21.15.39:0/1706101344 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568616cb050 con 0x5568613511c0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.958+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5568613511c0 0x5568617e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.959+0000 7fa7cb7fe640 10 client.15498 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.959+0000 7fa7eb7fe640 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5568613511c0 0x5568617e0420 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c9dba63fc21aec server_cookie=edf34c6848725c9 in_seq=0 out_seq=0 2024-05-04T03:12:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7cb7fe640 1 -- 172.21.15.39:0/1706101344 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568616cb050 con 0x5568613511c0 2024-05-04T03:12:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7cb7fe640 10 client.15498 handle_command_reply: tid=0 2024-05-04T03:12:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 client.15498 shutdown 2024-05-04T03:12:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 -- 172.21.15.39:0/1706101344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 msgr2=0x55686176c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686176c650 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009250 tx=0x7fa7dc009310 comp rx=0 tx=0).stop 2024-05-04T03:12:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 -- 172.21.15.39:0/1706101344 shutdown_connections 2024-05-04T03:12:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5568613511c0 0x5568617e0420 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5568613bd1b0 0x5568617dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55686139c4d0 0x55686176cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7cb7fe640 0 client.15498 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 --2- 172.21.15.39:0/1706101344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568613942f0 0x55686176c650 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.967+0000 7fa7f9775740 1 -- 172.21.15.39:0/1706101344 >> 172.21.15.39:0/1706101344 conn(0x5568613b1300 msgr2=0x556861762a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f9775740 1 -- 172.21.15.39:0/1706101344 shutdown_connections 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f9775740 1 -- 172.21.15.39:0/1706101344 wait complete. 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f9775740 20 client.15498 trim_cache size 0 max 16384 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:12:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:12:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:12:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:12:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7d40b2870 msgr2=0x7fa7d40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7d40b2870 0x7fa7d40b4d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00058c0 tx=0x7fa7e000c510 comp rx=0 tx=0).stop 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 msgr2=0x7fa7ec1a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 0x7fa7ec1a05f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4005e30 tx=0x7fa7e4003ba0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3026120819 shutdown_connections 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa7d40b2870 0x7fa7d40b4d80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7ec108a90 0x7fa7ec1a05f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7ec103d90 0x7fa7ec199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 --2- 172.21.15.39:0/3026120819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7ec102b50 0x7fa7ec199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3026120819 >> 172.21.15.39:0/3026120819 conn(0x7fa7ec0fdd40 msgr2=0x7fa7ec104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.968+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3026120819 shutdown_connections 2024-05-04T03:12:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:12.969+0000 7fa7f27ea640 1 -- 172.21.15.39:0/3026120819 wait complete. 2024-05-04T03:12:14.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:14 smithi066 ceph-mon[33264]: pgmap v429: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 31 op/s 2024-05-04T03:12:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:14 smithi167 ceph-mon[33614]: pgmap v429: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 31 op/s 2024-05-04T03:12:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:14 smithi039 ceph-mon[29715]: pgmap v429: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 31 op/s 2024-05-04T03:12:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:16 smithi066 ceph-mon[33264]: pgmap v430: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-04T03:12:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:16 smithi066 ceph-mon[33264]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-05-04T03:12:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:16 smithi167 ceph-mon[33614]: pgmap v430: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-04T03:12:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:16 smithi167 ceph-mon[33614]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-05-04T03:12:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:16 smithi039 ceph-mon[29715]: pgmap v430: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-05-04T03:12:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:16 smithi039 ceph-mon[29715]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-05-04T03:12:18.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:18 smithi066 ceph-mon[33264]: pgmap v431: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 24 op/s 2024-05-04T03:12:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:18 smithi167 ceph-mon[33614]: pgmap v431: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 24 op/s 2024-05-04T03:12:18.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:18 smithi039 ceph-mon[29715]: pgmap v431: 97 pgs: 97 active+clean; 431 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 24 op/s 2024-05-04T03:12:20.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:20 smithi066 ceph-mon[33264]: pgmap v432: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-05-04T03:12:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:20 smithi167 ceph-mon[33614]: pgmap v432: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-05-04T03:12:20.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:20 smithi039 ceph-mon[29715]: pgmap v432: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-05-04T03:12:22.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:22 smithi066 ceph-mon[33264]: pgmap v433: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-04T03:12:22.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:22 smithi167 ceph-mon[33614]: pgmap v433: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-04T03:12:22.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:22 smithi039 ceph-mon[29715]: pgmap v433: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-04T03:12:24.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:24 smithi167 ceph-mon[33614]: pgmap v434: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-05-04T03:12:24.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:24 smithi039 ceph-mon[29715]: pgmap v434: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-05-04T03:12:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:24 smithi066 ceph-mon[33264]: pgmap v434: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 MiB/s wr, 15 op/s 2024-05-04T03:12:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 -- 172.21.15.39:0/2466127778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180104eb0 msgr2=0x7f81801052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 --2- 172.21.15.39:0/2466127778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180104eb0 0x7f81801052c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8174009900 tx=0x7f817402a410 comp rx=0 tx=0).stop 2024-05-04T03:12:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 -- 172.21.15.39:0/2466127778 shutdown_connections 2024-05-04T03:12:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 --2- 172.21.15.39:0/2466127778 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8180109bb0 0x7f818010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 --2- 172.21.15.39:0/2466127778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180104eb0 0x7f81801052c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 --2- 172.21.15.39:0/2466127778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180103c70 0x7f8180104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 -- 172.21.15.39:0/2466127778 >> 172.21.15.39:0/2466127778 conn(0x7f81800fee40 msgr2=0x7f8180101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 -- 172.21.15.39:0/2466127778 shutdown_connections 2024-05-04T03:12:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 -- 172.21.15.39:0/2466127778 wait complete. 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 Processor -- start 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.082+0000 7f81875c0640 1 -- start start 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f81875c0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180103c70 0x7f818019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f81875c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180104eb0 0x7f818019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f81875c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8180109bb0 0x7f81801a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f81875c0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f818010f640 con 0x7f8180103c70 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f81875c0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f818010f4c0 con 0x7f8180109bb0 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f81875c0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f818010f7c0 con 0x7f8180104eb0 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180103c70 0x7f818019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180103c70 0x7f818019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50476/0 (socket says 172.21.15.39:50476) 2024-05-04T03:12:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 -- 172.21.15.39:0/1484634621 learned_addr learned my addr 172.21.15.39:0/1484634621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f8184b5b640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180104eb0 0x7f818019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f8185b5d640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8180109bb0 0x7f81801a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 -- 172.21.15.39:0/1484634621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180104eb0 msgr2=0x7f818019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180104eb0 0x7f818019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 -- 172.21.15.39:0/1484634621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8180109bb0 msgr2=0x7f81801a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8180109bb0 0x7f81801a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 -- 172.21.15.39:0/1484634621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81801a1de0 con 0x7f8180103c70 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.083+0000 7f818535c640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180103c70 0x7f818019a2a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f817c005d00 tx=0x7f817c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.084+0000 7f81727fc640 1 -- 172.21.15.39:0/1484634621 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c017070 con 0x7f8180103c70 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.084+0000 7f81727fc640 1 -- 172.21.15.39:0/1484634621 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f817c009c30 con 0x7f8180103c70 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.084+0000 7f81875c0640 1 -- 172.21.15.39:0/1484634621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81801a20d0 con 0x7f8180103c70 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.084+0000 7f81727fc640 1 -- 172.21.15.39:0/1484634621 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c00f620 con 0x7f8180103c70 2024-05-04T03:12:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.084+0000 7f81875c0640 1 -- 172.21.15.39:0/1484634621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81801a25d0 con 0x7f8180103c70 2024-05-04T03:12:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.086+0000 7f818e471740 1 Processor -- start 2024-05-04T03:12:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818e471740 1 -- start start 2024-05-04T03:12:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818e471740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771591b0 0x5582774faa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818e471740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771302f0 0x5582774fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818e471740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x558277501ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818e471740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55827745f5d0 con 0x5582771302f0 2024-05-04T03:12:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818e471740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558277418890 con 0x5582771591b0 2024-05-04T03:12:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818e471740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55827745b590 con 0x5582771384d0 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f81727fc640 1 -- 172.21.15.39:0/1484634621 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f817c01a070 con 0x7f8180103c70 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818535c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771302f0 0x5582774fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f8184b5b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x558277501ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f8184b5b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x558277501ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:55522/0 (socket says 172.21.15.39:55522) 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f8184b5b640 1 -- 172.21.15.39:0/4291100859 learned_addr learned my addr 172.21.15.39:0/4291100859 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f8185b5d640 1 --2- 172.21.15.39:0/4291100859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771591b0 0x5582774faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818535c640 1 -- 172.21.15.39:0/4291100859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 msgr2=0x558277501ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818535c640 1 --2- 172.21.15.39:0/4291100859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x558277501ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818535c640 1 -- 172.21.15.39:0/4291100859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771591b0 msgr2=0x5582774faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818535c640 1 --2- 172.21.15.39:0/4291100859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771591b0 0x5582774faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.087+0000 7f818535c640 1 -- 172.21.15.39:0/4291100859 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558277171210 con 0x5582771302f0 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.088+0000 7f818535c640 1 --2- 172.21.15.39:0/4291100859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771302f0 0x5582774fafe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f817c0df760 tx=0x7f817c0ea750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df2d3112a34c0ff server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.088+0000 7f81727fc640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f816c0b27e0 0x7f816c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.088+0000 7f816b7fe640 1 -- 172.21.15.39:0/4291100859 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c0a5900 con 0x5582771302f0 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.088+0000 7f8185b5d640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f816c0b27e0 0x7f816c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.088+0000 7f816b7fe640 1 -- 172.21.15.39:0/4291100859 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f817c0ef070 con 0x5582771302f0 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.088+0000 7f816b7fe640 1 -- 172.21.15.39:0/4291100859 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c0a8c00 con 0x5582771302f0 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.088+0000 7f81727fc640 1 -- 172.21.15.39:0/1484634621 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f817c0e1c80 con 0x7f8180103c70 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 -- 172.21.15.39:0/4291100859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771302f0 msgr2=0x5582774fafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f8185b5d640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f816c0b27e0 0x7f816c0b4cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5582774fbc30 tx=0x7f8178009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 --2- 172.21.15.39:0/4291100859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771302f0 0x5582774fafe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f817c0df760 tx=0x7f817c0ea750 comp rx=0 tx=0).stop 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 -- 172.21.15.39:0/4291100859 shutdown_connections 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 --2- 172.21.15.39:0/4291100859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x558277501ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 --2- 172.21.15.39:0/4291100859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771302f0 0x5582774fafe0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 --2- 172.21.15.39:0/4291100859 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771591b0 0x5582774faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 -- 172.21.15.39:0/4291100859 >> 172.21.15.39:0/4291100859 conn(0x55827714d300 msgr2=0x5582774f63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 -- 172.21.15.39:0/4291100859 shutdown_connections 2024-05-04T03:12:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 -- 172.21.15.39:0/4291100859 wait complete. 2024-05-04T03:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.089+0000 7f818e471740 1 -- start start 2024-05-04T03:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818e471740 1 Processor -- start 2024-05-04T03:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818e471740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277504610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818e471740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771384d0 0x558277504b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818e471740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771591b0 0x55827750bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818e471740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558277502930 con 0x5582771384d0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818e471740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55827741c370 con 0x5582771302f0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818e471740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558277502ab0 con 0x5582771591b0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818e471740 10 client.0 _ll_register_callbacks cb 0x558277170a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f8184b5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771384d0 0x558277504b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f8184b5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771384d0 0x558277504b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50494/0 (socket says 172.21.15.39:50494) 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f8184b5b640 1 -- 172.21.15.39:0/4014211483 learned_addr learned my addr 172.21.15.39:0/4014211483 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f818535c640 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277504610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f8185b5d640 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771591b0 0x55827750bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f8184b5b640 1 -- 172.21.15.39:0/4014211483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771591b0 msgr2=0x55827750bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f8184b5b640 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771591b0 0x55827750bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f8184b5b640 1 -- 172.21.15.39:0/4014211483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 msgr2=0x558277504610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.090+0000 7f8184b5b640 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277504610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f8184b5b640 1 -- 172.21.15.39:0/4014211483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55827723d620 con 0x5582771384d0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f8184b5b640 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771384d0 0x558277504b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8174005ab0 tx=0x7f8174031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb11801fe7a57b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f8168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f8168ff9640 1 -- 172.21.15.39:0/4014211483 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174035040 con 0x5582771384d0 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f8168ff9640 1 -- 172.21.15.39:0/4014211483 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8174004240 con 0x5582771384d0 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f818e471740 1 -- 172.21.15.39:0/4014211483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f817c0ef070 con 0x5582771384d0 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f8168ff9640 1 -- 172.21.15.39:0/4014211483 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8174004440 con 0x5582771384d0 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f818e471740 10 client.15510 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f818e471740 1 -- 172.21.15.39:0/4014211483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55827750ea50 con 0x5582771384d0 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f8168ff9640 1 -- 172.21.15.39:0/4014211483 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f817c0ef070 con 0x5582771384d0 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.091+0000 7f8168ff9640 1 -- 172.21.15.39:0/4014211483 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f817403c640 con 0x5582771384d0 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f818e471740 10 client.15510 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f818e471740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f818e471740 20 client.15510 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f818e471740 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558277117d80 0x558277512290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f818e471740 4 client.15510 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:12:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f818e471740 1 -- 172.21.15.39:0/4014211483 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55827746d1e0 con 0x558277117d80 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f818535c640 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558277117d80 0x558277512290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f8168ff9640 10 client.15510 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.092+0000 7f818535c640 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558277117d80 0x558277512290 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1d357de8d2a68a server_cookie=12a75950071415e9 in_seq=0 out_seq=0 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.093+0000 7f8168ff9640 1 -- 172.21.15.39:0/4014211483 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55827746d1e0 con 0x558277117d80 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.093+0000 7f8168ff9640 10 client.15510 handle_command_reply: tid=0 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.093+0000 7f818e471740 1 client.15510 shutdown 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.093+0000 7f818e471740 1 -- 172.21.15.39:0/4014211483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771384d0 msgr2=0x558277504b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.093+0000 7f818e471740 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771384d0 0x558277504b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8174005ab0 tx=0x7f8174031890 comp rx=0 tx=0).stop 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.093+0000 7f818e471740 1 -- 172.21.15.39:0/4014211483 shutdown_connections 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f818e471740 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558277117d80 0x558277512290 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f818e471740 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771591b0 0x55827750bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f818e471740 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771384d0 0x558277504b50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f8168ff9640 0 client.15510 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f818e471740 1 --2- 172.21.15.39:0/4014211483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277504610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f818e471740 1 -- 172.21.15.39:0/4014211483 >> 172.21.15.39:0/4014211483 conn(0x55827714d300 msgr2=0x5582774f9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f818e471740 1 -- 172.21.15.39:0/4014211483 shutdown_connections 2024-05-04T03:12:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f818e471740 1 -- 172.21.15.39:0/4014211483 wait complete. 2024-05-04T03:12:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.094+0000 7f818e471740 20 client.15510 trim_cache size 0 max 16384 2024-05-04T03:12:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.109+0000 7f818e471740 1 Processor -- start 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.109+0000 7f818e471740 1 -- start start 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.109+0000 7f818e471740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771302f0 0x5582774f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.109+0000 7f818e471740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771384d0 0x55827750b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.109+0000 7f818e471740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x55827750bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.109+0000 7f818e471740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f81801ab2f0 con 0x5582771591b0 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.109+0000 7f818e471740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f81801acef0 con 0x5582771384d0 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.109+0000 7f818e471740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81801ab9c0 con 0x5582771302f0 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x55827750bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x55827750bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50508/0 (socket says 172.21.15.39:50508) 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 -- 172.21.15.39:0/3568776573 learned_addr learned my addr 172.21.15.39:0/3568776573 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8184b5b640 1 --2- 172.21.15.39:0/3568776573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771384d0 0x55827750b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 -- 172.21.15.39:0/3568776573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771302f0 msgr2=0x5582774f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 --2- 172.21.15.39:0/3568776573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771302f0 0x5582774f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 -- 172.21.15.39:0/3568776573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771384d0 msgr2=0x55827750b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 --2- 172.21.15.39:0/3568776573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771384d0 0x55827750b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 -- 172.21.15.39:0/3568776573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582774f6d00 con 0x5582771591b0 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f8185b5d640 1 --2- 172.21.15.39:0/3568776573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x55827750bf10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f817801f5f0 tx=0x7f8178016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e95edc43003c055 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f816affd640 1 -- 172.21.15.39:0/3568776573 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817801d4d0 con 0x5582771591b0 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f816affd640 1 -- 172.21.15.39:0/3568776573 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f817801db00 con 0x5582771591b0 2024-05-04T03:12:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.110+0000 7f816affd640 1 -- 172.21.15.39:0/3568776573 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178011960 con 0x5582771591b0 2024-05-04T03:12:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 -- 172.21.15.39:0/3568776573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 msgr2=0x55827750bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 --2- 172.21.15.39:0/3568776573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x55827750bf10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f817801f5f0 tx=0x7f8178016000 comp rx=0 tx=0).stop 2024-05-04T03:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 -- 172.21.15.39:0/3568776573 shutdown_connections 2024-05-04T03:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 --2- 172.21.15.39:0/3568776573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x55827750bf10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 --2- 172.21.15.39:0/3568776573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771384d0 0x55827750b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 --2- 172.21.15.39:0/3568776573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771302f0 0x5582774f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 -- 172.21.15.39:0/3568776573 >> 172.21.15.39:0/3568776573 conn(0x55827714d300 msgr2=0x5582774f5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 -- 172.21.15.39:0/3568776573 shutdown_connections 2024-05-04T03:12:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 -- 172.21.15.39:0/3568776573 wait complete. 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 -- start start 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.111+0000 7f818e471740 1 Processor -- start 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818e471740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277509360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818e471740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x5582775098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818e471740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x558277578630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818e471740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f81801ac760 con 0x5582771591b0 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818e471740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f81801ac090 con 0x5582771302f0 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818e471740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81801a92d0 con 0x5582771384d0 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818e471740 10 client.0 _ll_register_callbacks cb 0x7f8174009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x558277578630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x558277578630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50514/0 (socket says 172.21.15.39:50514) 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 -- 172.21.15.39:0/2135703118 learned_addr learned my addr 172.21.15.39:0/2135703118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8184b5b640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x5582775098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 -- 172.21.15.39:0/2135703118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 msgr2=0x5582775098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x5582775098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 -- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 msgr2=0x558277509360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818535c640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277509360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277509360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 -- 172.21.15.39:0/2135703118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558277578b70 con 0x5582771591b0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8184b5b640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x5582775098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f818535c640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277509360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8185b5d640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x558277578630 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81780092f0 tx=0x7f8178004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727a841f49080d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.112+0000 7f8168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f8168ff9640 1 -- 172.21.15.39:0/2135703118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817801d4d0 con 0x5582771591b0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f8168ff9640 1 -- 172.21.15.39:0/2135703118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8178025040 con 0x5582771591b0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 1 -- 172.21.15.39:0/2135703118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f817801db00 con 0x5582771591b0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f8168ff9640 1 -- 172.21.15.39:0/2135703118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f817801d670 con 0x5582771591b0 2024-05-04T03:12:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 10 client.15516 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 1 -- 172.21.15.39:0/2135703118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558277579de0 con 0x5582771591b0 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f8168ff9640 1 -- 172.21.15.39:0/2135703118 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f817801db00 con 0x5582771591b0 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f8168ff9640 1 -- 172.21.15.39:0/2135703118 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f817803a030 con 0x5582771591b0 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 10 client.15516 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 20 client.15516 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5582770ed1c0 0x55827757c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 4 client.15516 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.113+0000 7f818e471740 1 -- 172.21.15.39:0/2135703118 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558277467050 con 0x5582770ed1c0 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.114+0000 7f818535c640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5582770ed1c0 0x55827757c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.114+0000 7f818535c640 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5582770ed1c0 0x55827757c8c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b7fcc737890e3ce server_cookie=d68092a44c37fd91 in_seq=0 out_seq=0 2024-05-04T03:12:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.114+0000 7f8168ff9640 10 client.15516 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.122+0000 7f8168ff9640 1 -- 172.21.15.39:0/2135703118 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558277467050 con 0x5582770ed1c0 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.122+0000 7f8168ff9640 10 client.15516 handle_command_reply: tid=0 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.122+0000 7f818e471740 1 client.15516 shutdown 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.122+0000 7f818e471740 1 -- 172.21.15.39:0/2135703118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 msgr2=0x558277578630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.122+0000 7f818e471740 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x558277578630 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81780092f0 tx=0x7f8178004ac0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.122+0000 7f818e471740 1 -- 172.21.15.39:0/2135703118 shutdown_connections 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f818e471740 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5582770ed1c0 0x55827757c8c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f818e471740 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582771591b0 0x558277578630 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f8168ff9640 0 client.15516 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f818e471740 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582771384d0 0x5582775098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f818e471740 1 --2- 172.21.15.39:0/2135703118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5582771302f0 0x558277509360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f818e471740 1 -- 172.21.15.39:0/2135703118 >> 172.21.15.39:0/2135703118 conn(0x55827714d300 msgr2=0x5582775038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f818e471740 1 -- 172.21.15.39:0/2135703118 shutdown_connections 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f818e471740 1 -- 172.21.15.39:0/2135703118 wait complete. 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f818e471740 20 client.15516 trim_cache size 0 max 16384 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:12:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:12:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 -- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f816c0b27e0 msgr2=0x7f816c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f816c0b27e0 0x7f816c0b4cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5582774fbc30 tx=0x7f8178009500 comp rx=0 tx=0).stop 2024-05-04T03:12:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 -- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180103c70 msgr2=0x7f818019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180103c70 0x7f818019a2a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f817c005d00 tx=0x7f817c00dde0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 -- 172.21.15.39:0/1484634621 shutdown_connections 2024-05-04T03:12:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f816c0b27e0 0x7f816c0b4cf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8180109bb0 0x7f81801a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8180104eb0 0x7f818019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 --2- 172.21.15.39:0/1484634621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8180103c70 0x7f818019a2a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.123+0000 7f81875c0640 1 -- 172.21.15.39:0/1484634621 >> 172.21.15.39:0/1484634621 conn(0x7f81800fee40 msgr2=0x7f8180105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.124+0000 7f81875c0640 1 -- 172.21.15.39:0/1484634621 shutdown_connections 2024-05-04T03:12:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:26.124+0000 7f81875c0640 1 -- 172.21.15.39:0/1484634621 wait complete. 2024-05-04T03:12:26.620 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:26 smithi066 ceph-mon[33264]: pgmap v435: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 357 KiB/s wr, 11 op/s 2024-05-04T03:12:26.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:26 smithi167 ceph-mon[33614]: pgmap v435: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 357 KiB/s wr, 11 op/s 2024-05-04T03:12:26.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:26 smithi039 ceph-mon[29715]: pgmap v435: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 357 KiB/s wr, 11 op/s 2024-05-04T03:12:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:28 smithi167 ceph-mon[33614]: pgmap v436: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 358 KiB/s wr, 13 op/s 2024-05-04T03:12:28.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:28 smithi039 ceph-mon[29715]: pgmap v436: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 358 KiB/s wr, 13 op/s 2024-05-04T03:12:28.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:28 smithi066 ceph-mon[33264]: pgmap v436: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 358 KiB/s wr, 13 op/s 2024-05-04T03:12:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:30 smithi167 ceph-mon[33614]: pgmap v437: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 357 KiB/s wr, 12 op/s 2024-05-04T03:12:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:30 smithi039 ceph-mon[29715]: pgmap v437: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 357 KiB/s wr, 12 op/s 2024-05-04T03:12:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:30 smithi066 ceph-mon[33264]: pgmap v437: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 357 KiB/s wr, 12 op/s 2024-05-04T03:12:32.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:32 smithi167 ceph-mon[33614]: pgmap v438: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-04T03:12:32.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:32 smithi039 ceph-mon[29715]: pgmap v438: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-04T03:12:32.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:32 smithi066 ceph-mon[33264]: pgmap v438: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-04T03:12:34.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:34 smithi167 ceph-mon[33614]: pgmap v439: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-04T03:12:34.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:34 smithi039 ceph-mon[29715]: pgmap v439: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-04T03:12:34.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:34 smithi066 ceph-mon[33264]: pgmap v439: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-04T03:12:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:36 smithi167 ceph-mon[33614]: pgmap v440: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-04T03:12:36.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:36 smithi039 ceph-mon[29715]: pgmap v440: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-04T03:12:36.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:36 smithi066 ceph-mon[33264]: pgmap v440: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-05-04T03:12:38.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:38 smithi167 ceph-mon[33614]: pgmap v441: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-04T03:12:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:38 smithi039 ceph-mon[29715]: pgmap v441: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-04T03:12:38.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:12:38.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:38 smithi066 ceph-mon[33264]: pgmap v441: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-04T03:12:39.051 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:12:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/1028113895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4102b40 msgr2=0x7ff8b4103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/1028113895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4102b40 0x7ff8b4103840 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009ca0 tx=0x7ff8a402a8c0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/1028113895 shutdown_connections 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/1028113895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4108a80 0x7ff8b410aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/1028113895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4103d80 0x7ff8b4104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/1028113895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4102b40 0x7ff8b4103840 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/1028113895 >> 172.21.15.39:0/1028113895 conn(0x7ff8b40fdd30 msgr2=0x7ff8b41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/1028113895 shutdown_connections 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.225+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/1028113895 wait complete. 2024-05-04T03:12:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b9e8b640 1 Processor -- start 2024-05-04T03:12:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b9e8b640 1 -- start start 2024-05-04T03:12:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b9e8b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4102b40 0x7ff8b419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b9e8b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4103d80 0x7ff8b419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b9e8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4108a80 0x7ff8b41a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b9e8b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b410e520 con 0x7ff8b4103d80 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b9e8b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b410e3a0 con 0x7ff8b4102b40 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b9e8b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b410e6a0 con 0x7ff8b4108a80 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4103d80 0x7ff8b419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4103d80 0x7ff8b419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50274/0 (socket says 172.21.15.39:50274) 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b2ffd640 1 -- 172.21.15.39:0/277154755 learned_addr learned my addr 172.21.15.39:0/277154755 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4102b40 0x7ff8b419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.226+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4108a80 0x7ff8b41a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b2ffd640 1 -- 172.21.15.39:0/277154755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4108a80 msgr2=0x7ff8b41a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4108a80 0x7ff8b41a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b2ffd640 1 -- 172.21.15.39:0/277154755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4102b40 msgr2=0x7ff8b419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4102b40 0x7ff8b419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b2ffd640 1 -- 172.21.15.39:0/277154755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b41a1d70 con 0x7ff8b4103d80 2024-05-04T03:12:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4108a80 0x7ff8b41a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4103d80 0x7ff8b419a820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c920 tx=0x7ff89c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b0ff9640 1 -- 172.21.15.39:0/277154755 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c007c70 con 0x7ff8b4103d80 2024-05-04T03:12:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b0ff9640 1 -- 172.21.15.39:0/277154755 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff89c007e10 con 0x7ff8b4103d80 2024-05-04T03:12:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/277154755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b41a2060 con 0x7ff8b4103d80 2024-05-04T03:12:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b0ff9640 1 -- 172.21.15.39:0/277154755 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c018710 con 0x7ff8b4103d80 2024-05-04T03:12:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.227+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/277154755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b41a25b0 con 0x7ff8b4103d80 2024-05-04T03:12:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b0ff9640 1 -- 172.21.15.39:0/277154755 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff89c00f450 con 0x7ff8b4103d80 2024-05-04T03:12:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8c0e14740 1 Processor -- start 2024-05-04T03:12:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8c0e14740 1 -- start start 2024-05-04T03:12:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62fed9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb0f2f0 0x55c62fed9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb174d0 0x55c62fee0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c62fe3e5d0 con 0x55c62fb381b0 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c62fdf7890 con 0x55c62fb0f2f0 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c62fe3a590 con 0x55c62fb174d0 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62fed9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62fed9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50284/0 (socket says 172.21.15.39:50284) 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b3fff640 1 -- 172.21.15.39:0/2286329787 learned_addr learned my addr 172.21.15.39:0/2286329787 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb174d0 0x55c62fee0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b0ff9640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8a00b27e0 0x7ff8a00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb0f2f0 0x55c62fed9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b3fff640 1 -- 172.21.15.39:0/2286329787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb174d0 msgr2=0x55c62fee0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb174d0 0x55c62fee0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b3fff640 1 -- 172.21.15.39:0/2286329787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb0f2f0 msgr2=0x55c62fed9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb0f2f0 0x55c62fed9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b3fff640 1 -- 172.21.15.39:0/2286329787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62fb50210 con 0x55c62fb381b0 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb174d0 0x55c62fee0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb0f2f0 0x55c62fed9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.230+0000 7ff8b0ff9640 1 -- 172.21.15.39:0/277154755 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff89c0e23a0 con 0x7ff8b4103d80 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.231+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8a00b27e0 0x7ff8a00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.231+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62fed9a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00be50 tx=0x7ff8ac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec2f8d9e9b75479 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.231+0000 7ff8a9ffb640 1 -- 172.21.15.39:0/2286329787 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac01c600 con 0x55c62fb381b0 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.231+0000 7ff8a9ffb640 1 -- 172.21.15.39:0/2286329787 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8ac01cc30 con 0x55c62fb381b0 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.231+0000 7ff8a9ffb640 1 -- 172.21.15.39:0/2286329787 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac01b980 con 0x55c62fb381b0 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 -- 172.21.15.39:0/2286329787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 msgr2=0x55c62fed9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8a00b27e0 0x7ff8a00b4cf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac020000 tx=0x7ff8ac01d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62fed9a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00be50 tx=0x7ff8ac00f730 comp rx=0 tx=0).stop 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 -- 172.21.15.39:0/2286329787 shutdown_connections 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb174d0 0x55c62fee0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb0f2f0 0x55c62fed9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/2286329787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62fed9a60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 -- 172.21.15.39:0/2286329787 >> 172.21.15.39:0/2286329787 conn(0x55c62fb2c300 msgr2=0x55c62fed53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 -- 172.21.15.39:0/2286329787 shutdown_connections 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 -- 172.21.15.39:0/2286329787 wait complete. 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 -- start start 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.232+0000 7ff8c0e14740 1 Processor -- start 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 0x55c62fee3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62fee3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62feeaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c62fee1930 con 0x55c62fb381b0 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c62fdfb370 con 0x55c62fb174d0 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c62fee1ab0 con 0x55c62fb0f2f0 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8c0e14740 10 client.0 _ll_register_callbacks cb 0x55c62fb4fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62feeaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62feeaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50288/0 (socket says 172.21.15.39:50288) 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 -- 172.21.15.39:0/82845952 learned_addr learned my addr 172.21.15.39:0/82845952 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62fee3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 0x55c62fee3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 -- 172.21.15.39:0/82845952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 msgr2=0x55c62fee3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 0x55c62fee3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 -- 172.21.15.39:0/82845952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 msgr2=0x55c62fee3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62fee3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 -- 172.21.15.39:0/82845952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62fc1c620 con 0x55c62fb381b0 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 0x55c62fee3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62feeaae0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005fd0 tx=0x7ff8ac0124a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460ca2c2b71fca server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.233+0000 7ff892ffd640 1 -- 172.21.15.39:0/82845952 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac02c040 con 0x55c62fb381b0 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff892ffd640 1 -- 172.21.15.39:0/82845952 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff8ac01bc20 con 0x55c62fb381b0 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 1 -- 172.21.15.39:0/82845952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ac01cc30 con 0x55c62fb381b0 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff892ffd640 1 -- 172.21.15.39:0/82845952 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8ac01be20 con 0x55c62fb381b0 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 10 client.15528 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 1 -- 172.21.15.39:0/82845952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c62feeda10 con 0x55c62fb381b0 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff892ffd640 1 -- 172.21.15.39:0/82845952 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8ac01cc30 con 0x55c62fb381b0 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff892ffd640 1 -- 172.21.15.39:0/82845952 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff8ac026480 con 0x55c62fb381b0 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 10 client.15528 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 20 client.15528 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c62faf6d80 0x55c62fef1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 4 client.15528 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.234+0000 7ff8c0e14740 1 -- 172.21.15.39:0/82845952 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c62fe4c1e0 con 0x55c62faf6d80 2024-05-04T03:12:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.235+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c62faf6d80 0x55c62fef1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.235+0000 7ff892ffd640 10 client.15528 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.235+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c62faf6d80 0x55c62fef1060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df882aaacb354c35 server_cookie=bec4ef226b4e9858 in_seq=0 out_seq=0 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff892ffd640 1 -- 172.21.15.39:0/82845952 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c62fe4c1e0 con 0x55c62faf6d80 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff892ffd640 10 client.15528 handle_command_reply: tid=0 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 client.15528 shutdown 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 -- 172.21.15.39:0/82845952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 msgr2=0x55c62feeaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62feeaae0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005fd0 tx=0x7ff8ac0124a0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 -- 172.21.15.39:0/82845952 shutdown_connections 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c62faf6d80 0x55c62fef1060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb381b0 0x55c62feeaae0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62fee3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff892ffd640 0 client.15528 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/82845952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 0x55c62fee3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 -- 172.21.15.39:0/82845952 >> 172.21.15.39:0/82845952 conn(0x55c62fb2c300 msgr2=0x55c62fed8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 -- 172.21.15.39:0/82845952 shutdown_connections 2024-05-04T03:12:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 1 -- 172.21.15.39:0/82845952 wait complete. 2024-05-04T03:12:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.236+0000 7ff8c0e14740 20 client.15528 trim_cache size 0 max 16384 2024-05-04T03:12:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8c0e14740 1 Processor -- start 2024-05-04T03:12:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8c0e14740 1 -- start start 2024-05-04T03:12:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62faf6d80 0x55c62fed5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 0x55c62feeaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb174d0 0x55c62feeaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41ab2d0 con 0x55c62fb174d0 2024-05-04T03:12:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41aced0 con 0x55c62faf6d80 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41ab9a0 con 0x55c62fb0f2f0 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb174d0 0x55c62feeaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb174d0 0x55c62feeaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50294/0 (socket says 172.21.15.39:50294) 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 -- 172.21.15.39:0/3072286897 learned_addr learned my addr 172.21.15.39:0/3072286897 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62faf6d80 0x55c62fed5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 -- 172.21.15.39:0/3072286897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 msgr2=0x55c62feeaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 0x55c62feeaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 -- 172.21.15.39:0/3072286897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62faf6d80 msgr2=0x55c62fed5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62faf6d80 0x55c62fed5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 -- 172.21.15.39:0/3072286897 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62fee46c0 con 0x55c62fb174d0 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62faf6d80 0x55c62fed5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.252+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb174d0 0x55c62feeaf40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac014040 tx=0x7ff8ac01bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a6d17bbd6cc5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8a97fa640 1 -- 172.21.15.39:0/3072286897 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac02c030 con 0x55c62fb174d0 2024-05-04T03:12:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8a97fa640 1 -- 172.21.15.39:0/3072286897 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8ac037030 con 0x55c62fb174d0 2024-05-04T03:12:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8a97fa640 1 -- 172.21.15.39:0/3072286897 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac026ac0 con 0x55c62fb174d0 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8c0e14740 1 -- 172.21.15.39:0/3072286897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb174d0 msgr2=0x55c62feeaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb174d0 0x55c62feeaf40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac014040 tx=0x7ff8ac01bdb0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8c0e14740 1 -- 172.21.15.39:0/3072286897 shutdown_connections 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb174d0 0x55c62feeaf40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62fb0f2f0 0x55c62feeaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/3072286897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62faf6d80 0x55c62fed5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.253+0000 7ff8c0e14740 1 -- 172.21.15.39:0/3072286897 >> 172.21.15.39:0/3072286897 conn(0x55c62fb2c300 msgr2=0x55c62fed4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 -- 172.21.15.39:0/3072286897 shutdown_connections 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 -- 172.21.15.39:0/3072286897 wait complete. 2024-05-04T03:12:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 -- start start 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 Processor -- start 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62faf6d80 0x55c62fee81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb0f2f0 0x55c62fee86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62ff573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41ac740 con 0x55c62fb0f2f0 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41ac070 con 0x55c62fb174d0 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41a92b0 con 0x55c62faf6d80 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8c0e14740 10 client.0 _ll_register_callbacks cb 0x7ff8ac029040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8b2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb0f2f0 0x55c62fee86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8b2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb0f2f0 0x55c62fee86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50302/0 (socket says 172.21.15.39:50302) 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.254+0000 7ff8b2ffd640 1 -- 172.21.15.39:0/1981966401 learned_addr learned my addr 172.21.15.39:0/1981966401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62faf6d80 0x55c62fee81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62ff573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b2ffd640 1 -- 172.21.15.39:0/1981966401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62faf6d80 msgr2=0x55c62fee81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62faf6d80 0x55c62fee81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b2ffd640 1 -- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 msgr2=0x55c62ff573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62ff573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b2ffd640 1 -- 172.21.15.39:0/1981966401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62ff57900 con 0x55c62fb0f2f0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b3fff640 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62ff573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8b2ffd640 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb0f2f0 0x55c62fee86f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0dfeb0 tx=0x7ff89c0a91d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9576e4038b0e185e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff892ffd640 1 -- 172.21.15.39:0/1981966401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0b25a0 con 0x55c62fb0f2f0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff892ffd640 1 -- 172.21.15.39:0/1981966401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff89c0cc070 con 0x55c62fb0f2f0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8c0e14740 1 -- 172.21.15.39:0/1981966401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ac037030 con 0x55c62fb0f2f0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff892ffd640 1 -- 172.21.15.39:0/1981966401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff89c0a94e0 con 0x55c62fb0f2f0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8c0e14740 10 client.15534 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff8c0e14740 1 -- 172.21.15.39:0/1981966401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c62ff58a50 con 0x55c62fb0f2f0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.255+0000 7ff892ffd640 1 -- 172.21.15.39:0/1981966401 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8ac037030 con 0x55c62fb0f2f0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff892ffd640 1 -- 172.21.15.39:0/1981966401 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff89c06f2c0 con 0x55c62fb0f2f0 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff8c0e14740 10 client.15534 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff8c0e14740 10 client.15534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:12:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff8c0e14740 20 client.15534 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c62fb381b0 0x55c62ff5b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff8c0e14740 4 client.15534 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:12:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff8c0e14740 1 -- 172.21.15.39:0/1981966401 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c62fe46050 con 0x55c62fb381b0 2024-05-04T03:12:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c62fb381b0 0x55c62ff5b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff8b37fe640 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c62fb381b0 0x55c62ff5b4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ca0259939b4622 server_cookie=ba478824f35b15f6 in_seq=0 out_seq=0 2024-05-04T03:12:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.256+0000 7ff892ffd640 10 client.15534 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff892ffd640 1 -- 172.21.15.39:0/1981966401 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c62fe46050 con 0x55c62fb381b0 2024-05-04T03:12:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff892ffd640 10 client.15534 handle_command_reply: tid=0 2024-05-04T03:12:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 client.15534 shutdown 2024-05-04T03:12:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 -- 172.21.15.39:0/1981966401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb0f2f0 msgr2=0x55c62fee86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb0f2f0 0x55c62fee86f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0dfeb0 tx=0x7ff89c0a91d0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 -- 172.21.15.39:0/1981966401 shutdown_connections 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c62fb381b0 0x55c62ff5b4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c62fb174d0 0x55c62ff573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c62fb0f2f0 0x55c62fee86f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff892ffd640 0 client.15534 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 --2- 172.21.15.39:0/1981966401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c62faf6d80 0x55c62fee81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 -- 172.21.15.39:0/1981966401 >> 172.21.15.39:0/1981966401 conn(0x55c62fb2c300 msgr2=0x55c62fee1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 -- 172.21.15.39:0/1981966401 shutdown_connections 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 1 -- 172.21.15.39:0/1981966401 wait complete. 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.261+0000 7ff8c0e14740 20 client.15534 trim_cache size 0 max 16384 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:12:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:12:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:12:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8a00b27e0 msgr2=0x7ff8a00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8a00b27e0 0x7ff8a00b4cf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac020000 tx=0x7ff8ac01d000 comp rx=0 tx=0).stop 2024-05-04T03:12:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4103d80 msgr2=0x7ff8b419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4103d80 0x7ff8b419a820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c920 tx=0x7ff89c00cde0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/277154755 shutdown_connections 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8a00b27e0 0x7ff8a00b4cf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b4108a80 0x7ff8b41a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8b4103d80 0x7ff8b419a820 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 --2- 172.21.15.39:0/277154755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8b4102b40 0x7ff8b419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/277154755 >> 172.21.15.39:0/277154755 conn(0x7ff8b40fdd30 msgr2=0x7ff8b410a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/277154755 shutdown_connections 2024-05-04T03:12:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:39.262+0000 7ff8b9e8b640 1 -- 172.21.15.39:0/277154755 wait complete. 2024-05-04T03:12:40.541 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:40 smithi039 ceph-mon[29715]: pgmap v442: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-04T03:12:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:40 smithi167 ceph-mon[33614]: pgmap v442: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-04T03:12:40.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:40 smithi066 ceph-mon[33264]: pgmap v442: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-05-04T03:12:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 -- 172.21.15.39:0/2904974962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448109880 msgr2=0x7f44481111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 --2- 172.21.15.39:0/2904974962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448109880 0x7f44481111e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f444000b0d0 tx=0x7f444000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:12:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 -- 172.21.15.39:0/2904974962 shutdown_connections 2024-05-04T03:12:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 --2- 172.21.15.39:0/2904974962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448109880 0x7f44481111e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 --2- 172.21.15.39:0/2904974962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4448104650 0x7f4448104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 --2- 172.21.15.39:0/2904974962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4448108f30 0x7f4448109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 -- 172.21.15.39:0/2904974962 >> 172.21.15.39:0/2904974962 conn(0x7f4448079f00 msgr2=0x7f444807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 -- 172.21.15.39:0/2904974962 shutdown_connections 2024-05-04T03:12:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 -- 172.21.15.39:0/2904974962 wait complete. 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.922+0000 7f444d27e700 1 Processor -- start 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f444d27e700 1 -- start start 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f444d27e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448104650 0x7f44481a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f444d27e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4448108f30 0x7f44481a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f444d27e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4448109880 0x7f444819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f444d27e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4448114360 con 0x7f4448104650 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f444d27e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44481141e0 con 0x7f4448108f30 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f444d27e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f44481144e0 con 0x7f4448109880 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f4446ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448104650 0x7f44481a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:40.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f4446ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448104650 0x7f44481a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50324/0 (socket says 172.21.15.39:50324) 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f4446ffd700 1 -- 172.21.15.39:0/2379611863 learned_addr learned my addr 172.21.15.39:0/2379611863 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f44477fe700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4448109880 0x7f444819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f443ffff700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4448108f30 0x7f44481a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f4446ffd700 1 -- 172.21.15.39:0/2379611863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4448109880 msgr2=0x7f444819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f4446ffd700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4448109880 0x7f444819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f4446ffd700 1 -- 172.21.15.39:0/2379611863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4448108f30 msgr2=0x7f44481a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f4446ffd700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4448108f30 0x7f44481a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.923+0000 7f4446ffd700 1 -- 172.21.15.39:0/2379611863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444819b220 con 0x7f4448104650 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.924+0000 7f4446ffd700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448104650 0x7f44481a09a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f443800d950 tx=0x7f443800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.924+0000 7f4444ff9700 1 -- 172.21.15.39:0/2379611863 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438016070 con 0x7f4448104650 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.924+0000 7f4444ff9700 1 -- 172.21.15.39:0/2379611863 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4438010460 con 0x7f4448104650 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.924+0000 7f4444ff9700 1 -- 172.21.15.39:0/2379611863 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443800f620 con 0x7f4448104650 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.924+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444819b510 con 0x7f4448104650 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.924+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44481a7750 con 0x7f4448104650 2024-05-04T03:12:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.925+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4448105ab0 con 0x7f4448104650 2024-05-04T03:12:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.927+0000 7f4444ff9700 1 -- 172.21.15.39:0/2379611863 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4438019070 con 0x7f4448104650 2024-05-04T03:12:40.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.928+0000 7f4444ff9700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44340b2900 0x7f44340b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:40.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.928+0000 7f443ffff700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44340b2900 0x7f44340b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:40.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.928+0000 7f4444ff9700 1 -- 172.21.15.39:0/2379611863 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f44380e1d90 con 0x7f4448104650 2024-05-04T03:12:40.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.928+0000 7f443ffff700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44340b2900 0x7f44340b4e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4430005fd0 tx=0x7f443000af00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:40.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:40.932+0000 7f4444ff9700 1 -- 172.21.15.39:0/2379611863 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44380a9ac0 con 0x7f4448104650 2024-05-04T03:12:41.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.203+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f444802e5f0 con 0x7f4448104650 2024-05-04T03:12:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.204+0000 7f4444ff9700 1 -- 172.21.15.39:0/2379611863 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11937 (secure 0 0 0) 0x7f44380a93c0 con 0x7f4448104650 2024-05-04T03:12:41.205 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:12:41.205 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:11:36.224039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24472,24506,24466,24490,24455]},"id":1}]} 2024-05-04T03:12:41.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44340b2900 msgr2=0x7f44340b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:41.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44340b2900 0x7f44340b4e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4430005fd0 tx=0x7f443000af00 comp rx=0 tx=0).stop 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448104650 msgr2=0x7f44481a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448104650 0x7f44481a09a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f443800d950 tx=0x7f443800dd10 comp rx=0 tx=0).stop 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 shutdown_connections 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f44340b2900 0x7f44340b4e10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4448104650 0x7f44481a09a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4448109880 0x7f444819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 --2- 172.21.15.39:0/2379611863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4448108f30 0x7f44481a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 >> 172.21.15.39:0/2379611863 conn(0x7f4448079f00 msgr2=0x7f44481020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 shutdown_connections 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:41.206+0000 7f444d27e700 1 -- 172.21.15.39:0/2379611863 wait complete. 2024-05-04T03:12:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-05-04T03:12:41.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:41 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2379611863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:41.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:41 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2379611863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:41.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:41 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2379611863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:42.024 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 80d64dad --release --await' 2024-05-04T03:12:42.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id 80d64dad --release --await 2024-05-04T03:12:42.278 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:12:42.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:42 smithi039 ceph-mon[29715]: pgmap v443: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-04T03:12:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:42 smithi167 ceph-mon[33614]: pgmap v443: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-04T03:12:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:42 smithi066 ceph-mon[33264]: pgmap v443: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-04T03:12:44.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.157+0000 7fcb237c9700 1 -- 172.21.15.39:0/4047735695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105520 msgr2=0x7fcb1c105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.157+0000 7fcb237c9700 1 --2- 172.21.15.39:0/4047735695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105520 0x7fcb1c105930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b0a0 tx=0x7fcb1400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.157+0000 7fcb237c9700 1 -- 172.21.15.39:0/4047735695 shutdown_connections 2024-05-04T03:12:44.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.157+0000 7fcb237c9700 1 --2- 172.21.15.39:0/4047735695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105520 0x7fcb1c105930 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.157+0000 7fcb237c9700 1 --2- 172.21.15.39:0/4047735695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c10e900 0x7fcb1c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.157+0000 7fcb237c9700 1 --2- 172.21.15.39:0/4047735695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb1c105e70 0x7fcb1c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.157+0000 7fcb237c9700 1 -- 172.21.15.39:0/4047735695 >> 172.21.15.39:0/4047735695 conn(0x7fcb1c0fef90 msgr2=0x7fcb1c1013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:44.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.158+0000 7fcb237c9700 1 -- 172.21.15.39:0/4047735695 shutdown_connections 2024-05-04T03:12:44.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.158+0000 7fcb237c9700 1 -- 172.21.15.39:0/4047735695 wait complete. 2024-05-04T03:12:44.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.158+0000 7fcb237c9700 1 Processor -- start 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.158+0000 7fcb237c9700 1 -- start start 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb237c9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c105520 0x7fcb1c103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb237c9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105e70 0x7fcb1c103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb237c9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb1c10e900 0x7fcb1c103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb237c9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c114340 con 0x7fcb1c105e70 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb237c9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1141c0 con 0x7fcb1c105520 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb237c9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1144c0 con 0x7fcb1c10e900 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105e70 0x7fcb1c103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb21565700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c105520 0x7fcb1c103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105e70 0x7fcb1c103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50352/0 (socket says 172.21.15.39:50352) 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 -- 172.21.15.39:0/1436995568 learned_addr learned my addr 172.21.15.39:0/1436995568 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb21d66700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb1c10e900 0x7fcb1c103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 -- 172.21.15.39:0/1436995568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb1c10e900 msgr2=0x7fcb1c103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb1c10e900 0x7fcb1c103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 -- 172.21.15.39:0/1436995568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c105520 msgr2=0x7fcb1c103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c105520 0x7fcb1c103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 -- 172.21.15.39:0/1436995568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c1a90f0 con 0x7fcb1c105e70 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb21d66700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb1c10e900 0x7fcb1c103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb20d64700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105e70 0x7fcb1c103750 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d950 tx=0x7fcb1000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.159+0000 7fcb1a7fc700 1 -- 172.21.15.39:0/1436995568 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10016070 con 0x7fcb1c105e70 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.160+0000 7fcb1a7fc700 1 -- 172.21.15.39:0/1436995568 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb10009dd0 con 0x7fcb1c105e70 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.160+0000 7fcb237c9700 1 -- 172.21.15.39:0/1436995568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c1a9380 con 0x7fcb1c105e70 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.160+0000 7fcb1a7fc700 1 -- 172.21.15.39:0/1436995568 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000f3c0 con 0x7fcb1c105e70 2024-05-04T03:12:44.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.160+0000 7fcb237c9700 1 -- 172.21.15.39:0/1436995568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c1a98c0 con 0x7fcb1c105e70 2024-05-04T03:12:44.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.162+0000 7fcb1a7fc700 1 -- 172.21.15.39:0/1436995568 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcb10019070 con 0x7fcb1c105e70 2024-05-04T03:12:44.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.163+0000 7fcb1a7fc700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb040b2880 0x7fcb040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.163+0000 7fcb21565700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb040b2880 0x7fcb040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.163+0000 7fcb1a7fc700 1 -- 172.21.15.39:0/1436995568 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fcb100e1e70 con 0x7fcb1c105e70 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.163+0000 7fcb321ddb80 1 Processor -- start 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.163+0000 7fcb21565700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb040b2880 0x7fcb040b4d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009fd0 tx=0x7fcb14006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.163+0000 7fcb321ddb80 1 -- start start 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a25ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a30700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a33060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562882948a40 con 0x562882a30c40 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5628828adea0 con 0x562882a26420 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562882944a00 con 0x562882a25ad0 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a33060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a33060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50366/0 (socket says 172.21.15.39:50366) 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21d66700 1 -- 172.21.15.39:0/2090950586 learned_addr learned my addr 172.21.15.39:0/2090950586 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21565700 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a25ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb20d64700 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a30700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21d66700 1 -- 172.21.15.39:0/2090950586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 msgr2=0x562882a25ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21d66700 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a25ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21d66700 1 -- 172.21.15.39:0/2090950586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 msgr2=0x562882a30700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21d66700 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a30700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb21d66700 1 -- 172.21.15.39:0/2090950586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628828959d0 con 0x562882a30c40 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.164+0000 7fcb20d64700 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a30700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.165+0000 7fcb21d66700 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a33060 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00cc20 tx=0x7fcb0c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e86cf59af4dc67 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.165+0000 7fcb037fe700 1 -- 172.21.15.39:0/2090950586 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00ed10 con 0x562882a30c40 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.165+0000 7fcb037fe700 1 -- 172.21.15.39:0/2090950586 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb0c016070 con 0x562882a30c40 2024-05-04T03:12:44.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.165+0000 7fcb037fe700 1 -- 172.21.15.39:0/2090950586 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c005020 con 0x562882a30c40 2024-05-04T03:12:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 -- 172.21.15.39:0/2090950586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 msgr2=0x562882a33060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a33060 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00cc20 tx=0x7fcb0c00cf30 comp rx=0 tx=0).stop 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 -- 172.21.15.39:0/2090950586 shutdown_connections 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a33060 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a25ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/2090950586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a30700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 -- 172.21.15.39:0/2090950586 >> 172.21.15.39:0/2090950586 conn(0x5628828a8f40 msgr2=0x562882a219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 -- 172.21.15.39:0/2090950586 shutdown_connections 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 -- 172.21.15.39:0/2090950586 wait complete. 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 -- start start 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.166+0000 7fcb321ddb80 1 Processor -- start 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 0x562882a2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 0x562882a2e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a2e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562882a33a10 con 0x562882a30c40 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562882901fd0 con 0x562882a25ad0 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562882a33b90 con 0x562882a26420 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb321ddb80 10 client.0 _ll_register_callbacks cb 0x5628828e0a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a2e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a2e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50380/0 (socket says 172.21.15.39:50380) 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21d66700 1 -- 172.21.15.39:0/1443904253 learned_addr learned my addr 172.21.15.39:0/1443904253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb20d64700 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 0x562882a2e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21565700 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 0x562882a2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21d66700 1 -- 172.21.15.39:0/1443904253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 msgr2=0x562882a2e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21d66700 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 0x562882a2e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21d66700 1 -- 172.21.15.39:0/1443904253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 msgr2=0x562882a2dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21d66700 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 0x562882a2dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.167+0000 7fcb21d66700 1 -- 172.21.15.39:0/1443904253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628828ff860 con 0x562882a30c40 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb21d66700 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a2e740 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b663fc1062e6168c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb00ff9700 1 -- 172.21.15.39:0/1443904253 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c029060 con 0x562882a30c40 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 1 -- 172.21.15.39:0/1443904253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb0c016070 con 0x562882a30c40 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb00ff9700 1 -- 172.21.15.39:0/1443904253 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fcb0c01a930 con 0x562882a30c40 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb00ff9700 1 -- 172.21.15.39:0/1443904253 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c01ab30 con 0x562882a30c40 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 10 client.15552 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 1 -- 172.21.15.39:0/1443904253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562882a3e6f0 con 0x562882a30c40 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb00ff9700 1 -- 172.21.15.39:0/1443904253 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb0c016070 con 0x562882a30c40 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb00ff9700 1 -- 172.21.15.39:0/1443904253 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcb0c023030 con 0x562882a30c40 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 10 client.15552 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 10 client.15552 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 20 client.15552 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:44.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562882a40c70 0x562882a43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 4 client.15552 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:12:44.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.168+0000 7fcb321ddb80 1 -- 172.21.15.39:0/1443904253 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562882956420 con 0x562882a40c70 2024-05-04T03:12:44.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.169+0000 7fcb21565700 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562882a40c70 0x562882a43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.169+0000 7fcb00ff9700 10 client.15552 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:44.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.169+0000 7fcb21565700 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562882a40c70 0x562882a43090 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38da80f0bf255648 server_cookie=782deaadbc5ea3d1 in_seq=0 out_seq=0 2024-05-04T03:12:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.170+0000 7fcb00ff9700 1 -- 172.21.15.39:0/1443904253 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562882956420 con 0x562882a40c70 2024-05-04T03:12:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.170+0000 7fcb00ff9700 10 client.15552 handle_command_reply: tid=0 2024-05-04T03:12:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.170+0000 7fcb321ddb80 1 client.15552 shutdown 2024-05-04T03:12:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 -- 172.21.15.39:0/1443904253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 msgr2=0x562882a2e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a2e740 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c00ed40 comp rx=0 tx=0).stop 2024-05-04T03:12:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 -- 172.21.15.39:0/1443904253 shutdown_connections 2024-05-04T03:12:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562882a40c70 0x562882a43090 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a2e740 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 0x562882a2dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb00ff9700 0 client.15552 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/1443904253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 0x562882a2e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 -- 172.21.15.39:0/1443904253 >> 172.21.15.39:0/1443904253 conn(0x5628828a8f40 msgr2=0x562882a27570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 -- 172.21.15.39:0/1443904253 shutdown_connections 2024-05-04T03:12:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 1 -- 172.21.15.39:0/1443904253 wait complete. 2024-05-04T03:12:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.171+0000 7fcb321ddb80 20 client.15552 trim_cache size 0 max 16384 2024-05-04T03:12:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb321ddb80 1 Processor -- start 2024-05-04T03:12:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb321ddb80 1 -- start start 2024-05-04T03:12:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 0x562882a2e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 0x562882a295b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a27c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1b0dd0 con 0x562882a30c40 2024-05-04T03:12:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1b29d0 con 0x562882a25ad0 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1b14a0 con 0x562882a26420 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a27c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb20d64700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 0x562882a295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a27c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50394/0 (socket says 172.21.15.39:50394) 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21d66700 1 -- 172.21.15.39:0/3807943429 learned_addr learned my addr 172.21.15.39:0/3807943429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21565700 1 --2- 172.21.15.39:0/3807943429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 0x562882a2e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21d66700 1 -- 172.21.15.39:0/3807943429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 msgr2=0x562882a295b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21d66700 1 --2- 172.21.15.39:0/3807943429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 0x562882a295b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21d66700 1 -- 172.21.15.39:0/3807943429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 msgr2=0x562882a2e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21d66700 1 --2- 172.21.15.39:0/3807943429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 0x562882a2e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.192+0000 7fcb21d66700 1 -- 172.21.15.39:0/3807943429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562882a22510 con 0x562882a30c40 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.193+0000 7fcb21d66700 1 --2- 172.21.15.39:0/3807943429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a27c30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e630 tx=0x7fcb0c00c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1838df531ab8617d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.193+0000 7fcb02ffd700 1 -- 172.21.15.39:0/3807943429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c034020 con 0x562882a30c40 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.193+0000 7fcb02ffd700 1 -- 172.21.15.39:0/3807943429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb0c016070 con 0x562882a30c40 2024-05-04T03:12:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.193+0000 7fcb02ffd700 1 -- 172.21.15.39:0/3807943429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c029040 con 0x562882a30c40 2024-05-04T03:12:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 -- 172.21.15.39:0/3807943429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 msgr2=0x562882a27c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/3807943429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a27c30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00e630 tx=0x7fcb0c00c3b0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 -- 172.21.15.39:0/3807943429 shutdown_connections 2024-05-04T03:12:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/3807943429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a27c30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/3807943429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a25ad0 0x562882a2e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/3807943429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a26420 0x562882a295b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 -- 172.21.15.39:0/3807943429 >> 172.21.15.39:0/3807943429 conn(0x5628828a8f40 msgr2=0x562882a27520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 -- 172.21.15.39:0/3807943429 shutdown_connections 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 -- 172.21.15.39:0/3807943429 wait complete. 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 -- start start 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.194+0000 7fcb321ddb80 1 Processor -- start 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a39350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb321ddb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a34bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1b2240 con 0x562882a30c40 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1b1b70 con 0x562882a26420 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb321ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c1aed70 con 0x562882a25ad0 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb321ddb80 10 client.0 _ll_register_callbacks cb 0x5628828e0a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a34bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21d66700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a34bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50400/0 (socket says 172.21.15.39:50400) 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21d66700 1 -- 172.21.15.39:0/126259132 learned_addr learned my addr 172.21.15.39:0/126259132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb20d64700 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21565700 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a39350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21d66700 1 -- 172.21.15.39:0/126259132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 msgr2=0x562882a39350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21d66700 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a39350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21d66700 1 -- 172.21.15.39:0/126259132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 msgr2=0x562882a346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21d66700 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a346b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21d66700 1 -- 172.21.15.39:0/126259132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562882a35320 con 0x562882a30c40 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.195+0000 7fcb21565700 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a39350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb21d66700 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a34bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c010450 tx=0x7fcb0c00e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdde511fa5caa9cb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb00ff9700 1 -- 172.21.15.39:0/126259132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c03d070 con 0x562882a30c40 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb00ff9700 1 -- 172.21.15.39:0/126259132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fcb0c0109c0 con 0x562882a30c40 2024-05-04T03:12:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 1 -- 172.21.15.39:0/126259132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb0c016070 con 0x562882a30c40 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb00ff9700 1 -- 172.21.15.39:0/126259132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c049050 con 0x562882a30c40 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 10 client.15558 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 1 -- 172.21.15.39:0/126259132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562882aab220 con 0x562882a30c40 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb00ff9700 1 -- 172.21.15.39:0/126259132 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb0c016070 con 0x562882a30c40 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb00ff9700 1 -- 172.21.15.39:0/126259132 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcb0c023040 con 0x562882a30c40 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 10 client.15558 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 10 client.15558 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 20 client.15558 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562882aab910 0x562882aadd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 4 client.15558 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "80d64dad", "release": true, "await": true}] 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.196+0000 7fcb321ddb80 1 -- 172.21.15.39:0/126259132 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "80d64dad", "release": true, "await": true}) -- 0x5628829504c0 con 0x562882aab910 2024-05-04T03:12:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.197+0000 7fcb21565700 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562882aab910 0x562882aadd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:44.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.197+0000 7fcb21565700 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562882aab910 0x562882aadd30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a266fc14a4c60f33 server_cookie=4794e98de4907d12 in_seq=0 out_seq=0 2024-05-04T03:12:44.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.197+0000 7fcb00ff9700 10 client.15558 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:44.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.201+0000 7fcb00ff9700 1 -- 172.21.15.39:0/126259132 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5628829504c0 con 0x562882aab910 2024-05-04T03:12:44.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.201+0000 7fcb00ff9700 10 client.15558 handle_command_reply: tid=0 2024-05-04T03:12:44.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.201+0000 7fcb321ddb80 1 client.15558 shutdown 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.201+0000 7fcb321ddb80 1 -- 172.21.15.39:0/126259132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 msgr2=0x562882a34bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.201+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a34bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c010450 tx=0x7fcb0c00e6e0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 1 -- 172.21.15.39:0/126259132 shutdown_connections 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562882aab910 0x562882aadd30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562882a30c40 0x562882a34bf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562882a25ad0 0x562882a39350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 1 --2- 172.21.15.39:0/126259132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562882a26420 0x562882a346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 1 -- 172.21.15.39:0/126259132 >> 172.21.15.39:0/126259132 conn(0x5628828a8f40 msgr2=0x562882a29410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb00ff9700 0 client.15558 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 1 -- 172.21.15.39:0/126259132 shutdown_connections 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 1 -- 172.21.15.39:0/126259132 wait complete. 2024-05-04T03:12:44.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb321ddb80 20 client.15558 trim_cache size 0 max 16384 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 4, 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "80d64dad": { 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 4, 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:12:44.205 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 89.2, 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:44.206 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:12:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb237c9700 1 -- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb040b2880 msgr2=0x7fcb040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb237c9700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb040b2880 0x7fcb040b4d90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009fd0 tx=0x7fcb14006210 comp rx=0 tx=0).stop 2024-05-04T03:12:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb237c9700 1 -- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105e70 msgr2=0x7fcb1c103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb237c9700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105e70 0x7fcb1c103750 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d950 tx=0x7fcb1000dc60 comp rx=0 tx=0).stop 2024-05-04T03:12:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb237c9700 1 -- 172.21.15.39:0/1436995568 shutdown_connections 2024-05-04T03:12:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.202+0000 7fcb237c9700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcb040b2880 0x7fcb040b4d90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.203+0000 7fcb237c9700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb1c10e900 0x7fcb1c103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.203+0000 7fcb237c9700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb1c105520 0x7fcb1c103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.203+0000 7fcb237c9700 1 --2- 172.21.15.39:0/1436995568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c105e70 0x7fcb1c103750 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.203+0000 7fcb237c9700 1 -- 172.21.15.39:0/1436995568 >> 172.21.15.39:0/1436995568 conn(0x7fcb1c0fef90 msgr2=0x7fcb1c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.203+0000 7fcb237c9700 1 -- 172.21.15.39:0/1436995568 shutdown_connections 2024-05-04T03:12:44.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:44.203+0000 7fcb237c9700 1 -- 172.21.15.39:0/1436995568 wait complete. 2024-05-04T03:12:44.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:44 smithi039 ceph-mon[29715]: pgmap v444: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-04T03:12:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:44 smithi167 ceph-mon[33614]: pgmap v444: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-04T03:12:44.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:44 smithi066 ceph-mon[33264]: pgmap v444: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-04T03:12:45.081 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '80d64dad', seconds elapsed: 43.3 2024-05-04T03:12:45.082 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.25 2024-05-04T03:12:45.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:12:45.218 INFO:teuthology.orchestra.run.smithi039.stderr:1800000 of 3200000 tuples (56%) done (elapsed 55.19 s, remaining 42.93 s) 2024-05-04T03:12:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:12:46.456 INFO:teuthology.orchestra.run.smithi039.stderr:1900000 of 3200000 tuples (59%) done (elapsed 56.43 s, remaining 38.61 s) 2024-05-04T03:12:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:46 smithi167 ceph-mon[33614]: pgmap v445: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-04T03:12:46.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:46 smithi039 ceph-mon[29715]: pgmap v445: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-04T03:12:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:46 smithi066 ceph-mon[33264]: pgmap v445: 97 pgs: 97 active+clean; 436 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-04T03:12:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 -- 172.21.15.39:0/926159887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa328108740 msgr2=0x7fa3281100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 --2- 172.21.15.39:0/926159887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa328108740 0x7fa3281100a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa32000b0d0 tx=0x7fa32000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 -- 172.21.15.39:0/926159887 shutdown_connections 2024-05-04T03:12:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 --2- 172.21.15.39:0/926159887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa328108740 0x7fa3281100a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 --2- 172.21.15.39:0/926159887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3281034c0 0x7fa3281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 --2- 172.21.15.39:0/926159887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa328107df0 0x7fa328108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 -- 172.21.15.39:0/926159887 >> 172.21.15.39:0/926159887 conn(0x7fa3280fedc0 msgr2=0x7fa3281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:47.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 -- 172.21.15.39:0/926159887 shutdown_connections 2024-05-04T03:12:47.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.318+0000 7fa32d741700 1 -- 172.21.15.39:0/926159887 wait complete. 2024-05-04T03:12:47.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.319+0000 7fa32d741700 1 Processor -- start 2024-05-04T03:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.319+0000 7fa32d741700 1 -- start start 2024-05-04T03:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa32d741700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3281034c0 0x7fa32819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa32d741700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa328107df0 0x7fa32819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa32d741700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa328108740 0x7fa3281999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa32d741700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa328113220 con 0x7fa328108740 2024-05-04T03:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa32d741700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281130a0 con 0x7fa328107df0 2024-05-04T03:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa32d741700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281133a0 con 0x7fa3281034c0 2024-05-04T03:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa3267fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa328107df0 0x7fa32819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa3267fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa328107df0 0x7fa32819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:52268/0 (socket says 172.21.15.39:52268) 2024-05-04T03:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.320+0000 7fa3267fc700 1 -- 172.21.15.39:0/3288899804 learned_addr learned my addr 172.21.15.39:0/3288899804 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa3277fe700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa328108740 0x7fa3281999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa326ffd700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3281034c0 0x7fa32819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa3267fc700 1 -- 172.21.15.39:0/3288899804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3281034c0 msgr2=0x7fa32819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa3267fc700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3281034c0 0x7fa32819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa3267fc700 1 -- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa328108740 msgr2=0x7fa3281999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa3267fc700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa328108740 0x7fa3281999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa3267fc700 1 -- 172.21.15.39:0/3288899804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32819a090 con 0x7fa328107df0 2024-05-04T03:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa3267fc700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa328107df0 0x7fa32819fd00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00e970 tx=0x7fa31c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.321+0000 7fa30ffff700 1 -- 172.21.15.39:0/3288899804 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c013be0 con 0x7fa328107df0 2024-05-04T03:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.322+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32819a380 con 0x7fa328107df0 2024-05-04T03:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.322+0000 7fa30ffff700 1 -- 172.21.15.39:0/3288899804 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa31c013d80 con 0x7fa328107df0 2024-05-04T03:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.322+0000 7fa30ffff700 1 -- 172.21.15.39:0/3288899804 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c00cda0 con 0x7fa328107df0 2024-05-04T03:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.322+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32806b610 con 0x7fa328107df0 2024-05-04T03:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.323+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3280512c0 con 0x7fa328107df0 2024-05-04T03:12:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.325+0000 7fa30ffff700 1 -- 172.21.15.39:0/3288899804 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa31c004750 con 0x7fa328107df0 2024-05-04T03:12:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.326+0000 7fa30ffff700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3100b2900 0x7fa3100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.326+0000 7fa30ffff700 1 -- 172.21.15.39:0/3288899804 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa31c0e6400 con 0x7fa328107df0 2024-05-04T03:12:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.326+0000 7fa326ffd700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3100b2900 0x7fa3100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.327+0000 7fa326ffd700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3100b2900 0x7fa3100b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa32819ab20 tx=0x7fa318009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:47.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.331+0000 7fa30ffff700 1 -- 172.21.15.39:0/3288899804 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa31c0ae0d0 con 0x7fa328107df0 2024-05-04T03:12:47.406 INFO:teuthology.orchestra.run.smithi039.stderr:2000000 of 3200000 tuples (62%) done (elapsed 57.38 s, remaining 34.43 s) 2024-05-04T03:12:47.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:47 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.683+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa32806c040 con 0x7fa328107df0 2024-05-04T03:12:47.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:47 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.684+0000 7fa30ffff700 1 -- 172.21.15.39:0/3288899804 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11937 (secure 0 0 0) 0x7fa31c0c2e20 con 0x7fa328107df0 2024-05-04T03:12:47.687 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:12:47.687 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:11:36.224039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24472,24506,24466,24490,24455]},"id":1}]} 2024-05-04T03:12:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3100b2900 msgr2=0x7fa3100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3100b2900 0x7fa3100b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa32819ab20 tx=0x7fa318009260 comp rx=0 tx=0).stop 2024-05-04T03:12:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa328107df0 msgr2=0x7fa32819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa328107df0 0x7fa32819fd00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00e970 tx=0x7fa31c00ec80 comp rx=0 tx=0).stop 2024-05-04T03:12:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 shutdown_connections 2024-05-04T03:12:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa3100b2900 0x7fa3100b4e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa328108740 0x7fa3281999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3281034c0 0x7fa32819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 --2- 172.21.15.39:0/3288899804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa328107df0 0x7fa32819fd00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.687+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 >> 172.21.15.39:0/3288899804 conn(0x7fa3280fedc0 msgr2=0x7fa328107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:47.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.688+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 shutdown_connections 2024-05-04T03:12:47.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:47.688+0000 7fa32d741700 1 -- 172.21.15.39:0/3288899804 wait complete. 2024-05-04T03:12:47.691 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-05-04T03:12:47.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:47 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:48.452 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 89.25 --await-for 42.0' 2024-05-04T03:12:48.453 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 89.25 --await-for 42.0 2024-05-04T03:12:48.639 INFO:teuthology.orchestra.run.smithi039.stderr:2100000 of 3200000 tuples (65%) done (elapsed 58.61 s, remaining 30.70 s) 2024-05-04T03:12:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:48 smithi167 ceph-mon[33614]: pgmap v446: 97 pgs: 97 active+clean; 448 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 MiB/s wr, 11 op/s 2024-05-04T03:12:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:48 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3288899804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:48.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:48 smithi039 ceph-mon[29715]: pgmap v446: 97 pgs: 97 active+clean; 448 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 MiB/s wr, 11 op/s 2024-05-04T03:12:48.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:48 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3288899804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:48.754 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:12:48.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:48 smithi066 ceph-mon[33264]: pgmap v446: 97 pgs: 97 active+clean; 448 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.2 MiB/s wr, 11 op/s 2024-05-04T03:12:48.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:48 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3288899804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:12:49.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:49.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:49.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:12:49.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:49.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:49.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:12:49.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:49.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:49.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:12:49.854 INFO:teuthology.orchestra.run.smithi039.stderr:2200000 of 3200000 tuples (68%) done (elapsed 59.83 s, remaining 27.19 s) 2024-05-04T03:12:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.465+0000 7f428af8e700 1 -- 172.21.15.39:0/1501932094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42841034c0 msgr2=0x7f42841038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.465+0000 7f428af8e700 1 --2- 172.21.15.39:0/1501932094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42841034c0 0x7f42841038d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42780099c0 tx=0x7f4278009cd0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 -- 172.21.15.39:0/1501932094 shutdown_connections 2024-05-04T03:12:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 --2- 172.21.15.39:0/1501932094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42841034c0 0x7f42841038d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 --2- 172.21.15.39:0/1501932094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4284108740 0x7f42841100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 --2- 172.21.15.39:0/1501932094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4284107df0 0x7f4284108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 -- 172.21.15.39:0/1501932094 >> 172.21.15.39:0/1501932094 conn(0x7f42840fedc0 msgr2=0x7f42841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 -- 172.21.15.39:0/1501932094 shutdown_connections 2024-05-04T03:12:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 -- 172.21.15.39:0/1501932094 wait complete. 2024-05-04T03:12:50.469 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:50 smithi039 ceph-mon[29715]: pgmap v447: 97 pgs: 97 active+clean; 486 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 15 op/s 2024-05-04T03:12:50.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 Processor -- start 2024-05-04T03:12:50.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.466+0000 7f428af8e700 1 -- start start 2024-05-04T03:12:50.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428af8e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42841034c0 0x7f428410a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428af8e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4284107df0 0x7f428410a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428af8e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4284108740 0x7f428410ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428af8e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4284113270 con 0x7f4284108740 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428af8e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f42841130f0 con 0x7f4284107df0 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428af8e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42841133f0 con 0x7f42841034c0 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428952b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4284108740 0x7f428410ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428952b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4284108740 0x7f428410ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54910/0 (socket says 172.21.15.39:54910) 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428952b700 1 -- 172.21.15.39:0/2771800712 learned_addr learned my addr 172.21.15.39:0/2771800712 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42841034c0 0x7f428410a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f4283fff700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4284107df0 0x7f428410a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.467+0000 7f428952b700 1 -- 172.21.15.39:0/2771800712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42841034c0 msgr2=0x7f428410a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f428952b700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42841034c0 0x7f428410a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f428952b700 1 -- 172.21.15.39:0/2771800712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4284107df0 msgr2=0x7f428410a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f428952b700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4284107df0 0x7f428410a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f428952b700 1 -- 172.21.15.39:0/2771800712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42841ad590 con 0x7f4284108740 2024-05-04T03:12:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f4283fff700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4284107df0 0x7f428410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f428952b700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4284108740 0x7f428410ab40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f427c007ee0 tx=0x7f427c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f4281ffb700 1 -- 172.21.15.39:0/2771800712 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c00dbd0 con 0x7f4284108740 2024-05-04T03:12:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f4281ffb700 1 -- 172.21.15.39:0/2771800712 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f427c00dd70 con 0x7f4284108740 2024-05-04T03:12:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f428af8e700 1 -- 172.21.15.39:0/2771800712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42841ad820 con 0x7f4284108740 2024-05-04T03:12:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f4281ffb700 1 -- 172.21.15.39:0/2771800712 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c0058d0 con 0x7f4284108740 2024-05-04T03:12:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.468+0000 7f428af8e700 1 -- 172.21.15.39:0/2771800712 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42841add10 con 0x7f4284108740 2024-05-04T03:12:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.471+0000 7f4281ffb700 1 -- 172.21.15.39:0/2771800712 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f427c01a020 con 0x7f4284108740 2024-05-04T03:12:50.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.472+0000 7f4281ffb700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f42740b2880 0x7f42740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.472+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f42740b2880 0x7f42740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.473+0000 7f4281ffb700 1 -- 172.21.15.39:0/2771800712 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f427c0e19b0 con 0x7f4284108740 2024-05-04T03:12:50.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.473+0000 7f42999a2b80 1 Processor -- start 2024-05-04T03:12:50.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.473+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f42740b2880 0x7f42740b4d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4278000c00 tx=0x7f4278005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.473+0000 7f42999a2b80 1 -- start start 2024-05-04T03:12:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.473+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.473+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b74b0 0x5622805c1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.473+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f42999a2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5622804d9ad0 con 0x5622805b6b60 2024-05-04T03:12:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f42999a2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56228043ef30 con 0x5622805c1cd0 2024-05-04T03:12:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f42999a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5622804d5a90 con 0x5622805b74b0 2024-05-04T03:12:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54918/0 (socket says 172.21.15.39:54918) 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 -- 172.21.15.39:0/3485356150 learned_addr learned my addr 172.21.15.39:0/3485356150 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4283fff700 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b74b0 0x5622805c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f428952b700 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 -- 172.21.15.39:0/3485356150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b74b0 msgr2=0x5622805c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b74b0 0x5622805c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 -- 172.21.15.39:0/3485356150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 msgr2=0x5622805c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 -- 172.21.15.39:0/3485356150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562280426a60 con 0x5622805b6b60 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f428952b700 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f4288d2a700 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805b6f70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f42780099c0 tx=0x7f427801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabd0a7b7705ab0f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f426effd700 1 -- 172.21.15.39:0/3485356150 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427800bb70 con 0x5622805b6b60 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f426effd700 1 -- 172.21.15.39:0/3485356150 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f427800bd10 con 0x5622805b6b60 2024-05-04T03:12:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.474+0000 7f426effd700 1 -- 172.21.15.39:0/3485356150 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427800f880 con 0x5622805b6b60 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.475+0000 7f42999a2b80 1 -- 172.21.15.39:0/3485356150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 msgr2=0x5622805b6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.475+0000 7f42999a2b80 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805b6f70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f42780099c0 tx=0x7f427801e000 comp rx=0 tx=0).stop 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 -- 172.21.15.39:0/3485356150 shutdown_connections 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805b6f70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 --2- 172.21.15.39:0/3485356150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b74b0 0x5622805c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 -- 172.21.15.39:0/3485356150 >> 172.21.15.39:0/3485356150 conn(0x562280439fd0 msgr2=0x5622805b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 -- 172.21.15.39:0/3485356150 shutdown_connections 2024-05-04T03:12:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 -- 172.21.15.39:0/3485356150 wait complete. 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 -- start start 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.476+0000 7f42999a2b80 1 Processor -- start 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805bed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 0x5622805bf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805c1cd0 0x5622805bf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f42999a2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5622805c49e0 con 0x5622805b6b60 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f42999a2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562280493060 con 0x5622805b74b0 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f42999a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5622805c4b60 con 0x5622805c1cd0 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f42999a2b80 10 client.0 _ll_register_callbacks cb 0x562280471ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4288d2a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4288d2a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805bed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54926/0 (socket says 172.21.15.39:54926) 2024-05-04T03:12:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4288d2a700 1 -- 172.21.15.39:0/2521102670 learned_addr learned my addr 172.21.15.39:0/2521102670 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4283fff700 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 0x5622805bf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f428952b700 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805c1cd0 0x5622805bf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4288d2a700 1 -- 172.21.15.39:0/2521102670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805c1cd0 msgr2=0x5622805bf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805c1cd0 0x5622805bf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4288d2a700 1 -- 172.21.15.39:0/2521102670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 msgr2=0x5622805bf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 0x5622805bf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.477+0000 7f4288d2a700 1 -- 172.21.15.39:0/2521102670 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622804908f0 con 0x5622805b6b60 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805bed10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4278005850 tx=0x7f427800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b89b269ec1b3522 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f4263fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f4263fff700 1 -- 172.21.15.39:0/2521102670 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427803a020 con 0x5622805b6b60 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f42999a2b80 1 -- 172.21.15.39:0/2521102670 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427800bd10 con 0x5622805b6b60 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f4263fff700 1 -- 172.21.15.39:0/2521102670 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4278003890 con 0x5622805b6b60 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f4263fff700 1 -- 172.21.15.39:0/2521102670 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f427800bb70 con 0x5622805b6b60 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f42999a2b80 10 client.15576 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:50.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f42999a2b80 1 -- 172.21.15.39:0/2521102670 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5622805cf5f0 con 0x5622805b6b60 2024-05-04T03:12:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.478+0000 7f4263fff700 1 -- 172.21.15.39:0/2521102670 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f427800bd10 con 0x5622805b6b60 2024-05-04T03:12:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f4263fff700 1 -- 172.21.15.39:0/2521102670 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4278022070 con 0x5622805b6b60 2024-05-04T03:12:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f42999a2b80 10 client.15576 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f42999a2b80 10 client.15576 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:12:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f42999a2b80 20 client.15576 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5622805d1ac0 0x5622805d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f42999a2b80 4 client.15576 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:12:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f42999a2b80 1 -- 172.21.15.39:0/2521102670 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622804e74b0 con 0x5622805d1ac0 2024-05-04T03:12:50.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f4283fff700 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5622805d1ac0 0x5622805d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f4263fff700 10 client.15576 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:50.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.479+0000 7f4283fff700 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5622805d1ac0 0x5622805d3ee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0761be4127cf81 server_cookie=e62d21c4a1570e8a in_seq=0 out_seq=0 2024-05-04T03:12:50.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.480+0000 7f4263fff700 1 -- 172.21.15.39:0/2521102670 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622804e74b0 con 0x5622805d1ac0 2024-05-04T03:12:50.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.480+0000 7f4263fff700 10 client.15576 handle_command_reply: tid=0 2024-05-04T03:12:50.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.480+0000 7f42999a2b80 1 client.15576 shutdown 2024-05-04T03:12:50.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.480+0000 7f42999a2b80 1 -- 172.21.15.39:0/2521102670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 msgr2=0x5622805bed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.480+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805bed10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4278005850 tx=0x7f427800f7b0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.480+0000 7f42999a2b80 1 -- 172.21.15.39:0/2521102670 shutdown_connections 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.480+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5622805d1ac0 0x5622805d3ee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.481+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b6b60 0x5622805bed10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.481+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805c1cd0 0x5622805bf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.481+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2521102670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 0x5622805bf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.481+0000 7f4263fff700 0 client.15576 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.481+0000 7f42999a2b80 1 -- 172.21.15.39:0/2521102670 >> 172.21.15.39:0/2521102670 conn(0x562280439fd0 msgr2=0x5622805b8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.481+0000 7f42999a2b80 1 -- 172.21.15.39:0/2521102670 shutdown_connections 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.481+0000 7f42999a2b80 1 -- 172.21.15.39:0/2521102670 wait complete. 2024-05-04T03:12:50.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.481+0000 7f42999a2b80 20 client.15576 trim_cache size 0 max 16384 2024-05-04T03:12:50.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.506+0000 7f42999a2b80 1 Processor -- start 2024-05-04T03:12:50.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.506+0000 7f42999a2b80 1 -- start start 2024-05-04T03:12:50.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.506+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b74b0 0x5622805cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f42999a2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f42841b50b0 con 0x5622805b74b0 2024-05-04T03:12:50.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f42999a2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f42841b6cb0 con 0x5622805c1cd0 2024-05-04T03:12:50.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f42999a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42841b5780 con 0x5622805b6b60 2024-05-04T03:12:50.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f4283fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b74b0 0x5622805cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f4283fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b74b0 0x5622805cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54930/0 (socket says 172.21.15.39:54930) 2024-05-04T03:12:50.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f4283fff700 1 -- 172.21.15.39:0/493249596 learned_addr learned my addr 172.21.15.39:0/493249596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:50.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f428952b700 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f4288d2a700 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f428952b700 1 -- 172.21.15.39:0/493249596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 msgr2=0x5622805b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f428952b700 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f428952b700 1 -- 172.21.15.39:0/493249596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b74b0 msgr2=0x5622805cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f428952b700 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b74b0 0x5622805cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f428952b700 1 -- 172.21.15.39:0/493249596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622805b35a0 con 0x5622805c1cd0 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.507+0000 7f4288d2a700 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.508+0000 7f428952b700 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c0ad0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f427c0df460 tx=0x7f427c0a8a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622f0fd1f6b47403 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.508+0000 7f426e7fc700 1 -- 172.21.15.39:0/493249596 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c0ad330 con 0x5622805c1cd0 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.508+0000 7f426e7fc700 1 -- 172.21.15.39:0/493249596 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f427c0b1de0 con 0x5622805c1cd0 2024-05-04T03:12:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.508+0000 7f426e7fc700 1 -- 172.21.15.39:0/493249596 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c0a9440 con 0x5622805c1cd0 2024-05-04T03:12:50.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 -- 172.21.15.39:0/493249596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 msgr2=0x5622805c0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c0ad0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f427c0df460 tx=0x7f427c0a8a90 comp rx=0 tx=0).stop 2024-05-04T03:12:50.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 -- 172.21.15.39:0/493249596 shutdown_connections 2024-05-04T03:12:50.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805c1cd0 0x5622805c0ad0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 --2- 172.21.15.39:0/493249596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805b74b0 0x5622805cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 -- 172.21.15.39:0/493249596 >> 172.21.15.39:0/493249596 conn(0x562280439fd0 msgr2=0x5622805c03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:50.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 -- 172.21.15.39:0/493249596 shutdown_connections 2024-05-04T03:12:50.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.510+0000 7f42999a2b80 1 -- 172.21.15.39:0/493249596 wait complete. 2024-05-04T03:12:50.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.511+0000 7f42999a2b80 1 -- start start 2024-05-04T03:12:50.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.511+0000 7f42999a2b80 1 Processor -- start 2024-05-04T03:12:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 0x5622805d5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f42999a2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805c1cd0 0x5622805d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4288d2a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f428952b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805c1cd0 0x5622805d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4288d2a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805d6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:35072/0 (socket says 172.21.15.39:35072) 2024-05-04T03:12:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4288d2a700 1 -- 172.21.15.39:0/2999904374 learned_addr learned my addr 172.21.15.39:0/2999904374 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f42841b6520 con 0x5622805c1cd0 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f42841b5e50 con 0x5622805b74b0 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42841b3050 con 0x5622805b6b60 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4283fff700 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 0x5622805d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f42999a2b80 10 client.0 _ll_register_callbacks cb 0x562280471ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4288d2a700 1 -- 172.21.15.39:0/2999904374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 msgr2=0x5622805d5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 0x5622805d5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4288d2a700 1 -- 172.21.15.39:0/2999904374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805c1cd0 msgr2=0x5622805d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805c1cd0 0x5622805d5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.512+0000 7f4288d2a700 1 -- 172.21.15.39:0/2999904374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622805d5bd0 con 0x5622805b6b60 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.513+0000 7f4288d2a700 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805d6d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4278041e00 tx=0x7f4278009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d816624aeac838d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.513+0000 7f4263fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.513+0000 7f4263fff700 1 -- 172.21.15.39:0/2999904374 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427803a070 con 0x5622805b6b60 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.513+0000 7f4263fff700 1 -- 172.21.15.39:0/2999904374 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f42780416f0 con 0x5622805b6b60 2024-05-04T03:12:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.513+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427c0b1de0 con 0x5622805b6b60 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.513+0000 7f4263fff700 1 -- 172.21.15.39:0/2999904374 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42780418f0 con 0x5622805b6b60 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.513+0000 7f42999a2b80 10 client.25252 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56228063cb50 con 0x5622805b6b60 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f4263fff700 1 -- 172.21.15.39:0/2999904374 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f427c0b1de0 con 0x5622805b6b60 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f4263fff700 1 -- 172.21.15.39:0/2999904374 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4278022040 con 0x5622805b6b60 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f42999a2b80 10 client.25252 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f42999a2b80 10 client.25252 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f42999a2b80 20 client.25252 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56228063d380 0x56228063f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f42999a2b80 4 client.25252 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.25, "await_for": 42.0}] 2024-05-04T03:12:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.25, "await_for": 42.0}) -- 0x5622804e1550 con 0x56228063d380 2024-05-04T03:12:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.514+0000 7f4283fff700 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56228063d380 0x56228063f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.515+0000 7f4263fff700 10 client.25252 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.515+0000 7f4283fff700 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56228063d380 0x56228063f7a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a14494f4b62f703 server_cookie=c44780e5d3a40ca9 in_seq=0 out_seq=0 2024-05-04T03:12:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:50 smithi167 ceph-mon[33614]: pgmap v447: 97 pgs: 97 active+clean; 486 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 15 op/s 2024-05-04T03:12:50.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:50 smithi066 ceph-mon[33264]: pgmap v447: 97 pgs: 97 active+clean; 486 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 15 op/s 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.831+0000 7f4263fff700 1 -- 172.21.15.39:0/2999904374 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5622804e1550 con 0x56228063d380 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.831+0000 7f4263fff700 10 client.25252 handle_command_reply: tid=0 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.831+0000 7f42999a2b80 1 client.25252 shutdown 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 msgr2=0x5622805d6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805d6d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4278041e00 tx=0x7f4278009650 comp rx=0 tx=0).stop 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 shutdown_connections 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56228063d380 0x56228063f7a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622805c1cd0 0x5622805d5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5622805b6b60 0x5622805d6d70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 --2- 172.21.15.39:0/2999904374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5622805b74b0 0x5622805d5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f4263fff700 0 client.25252 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 >> 172.21.15.39:0/2999904374 conn(0x562280439fd0 msgr2=0x5622805b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 shutdown_connections 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 1 -- 172.21.15.39:0/2999904374 wait complete. 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.832+0000 7f42999a2b80 20 client.25252 trim_cache size 0 max 16384 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:12:50.835 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 6, 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "750d580f": { 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 6, 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 89.2, 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:12:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:12:50.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:12:50.837 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:12:50.837 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:50.837 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:50.837 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:50.837 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:50.837 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:12:50.837 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 -- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f42740b2880 msgr2=0x7f42740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f42740b2880 0x7f42740b4d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4278000c00 tx=0x7f4278005fb0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 -- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4284108740 msgr2=0x7f428410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4284108740 0x7f428410ab40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f427c007ee0 tx=0x7f427c00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 -- 172.21.15.39:0/2771800712 shutdown_connections 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f42740b2880 0x7f42740b4d90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4284108740 0x7f428410ab40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42841034c0 0x7f428410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 --2- 172.21.15.39:0/2771800712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4284107df0 0x7f428410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 -- 172.21.15.39:0/2771800712 >> 172.21.15.39:0/2771800712 conn(0x7f42840fedc0 msgr2=0x7f4284109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 -- 172.21.15.39:0/2771800712 shutdown_connections 2024-05-04T03:12:50.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:12:50.833+0000 7f428af8e700 1 -- 172.21.15.39:0/2771800712 wait complete. 2024-05-04T03:12:51.638 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '750d580f', quiesce took 6.6 seconds. Will release after: 29.25 2024-05-04T03:12:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 -- 172.21.15.39:0/1885659572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 msgr2=0x7f751010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 --2- 172.21.15.39:0/1885659572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 0x7f751010aeb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7508009ca0 tx=0x7f750802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 -- 172.21.15.39:0/1885659572 shutdown_connections 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 --2- 172.21.15.39:0/1885659572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 0x7f751010aeb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 --2- 172.21.15.39:0/1885659572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7510103d90 0x7f75101041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 --2- 172.21.15.39:0/1885659572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7510102b50 0x7f7510103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 -- 172.21.15.39:0/1885659572 >> 172.21.15.39:0/1885659572 conn(0x7f75100fdd40 msgr2=0x7f75101001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 -- 172.21.15.39:0/1885659572 shutdown_connections 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.374+0000 7f75152f2640 1 -- 172.21.15.39:0/1885659572 wait complete. 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f75152f2640 1 Processor -- start 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f75152f2640 1 -- start start 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f75152f2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7510102b50 0x7f751019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f75152f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7510103d90 0x7f751019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f75152f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 0x7f75101a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f75152f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f751010e580 con 0x7f7510108a90 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f75152f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f751010e400 con 0x7f7510103d90 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f75152f2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f751010e700 con 0x7f7510102b50 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f750f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 0x7f75101a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f750f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 0x7f75101a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54954/0 (socket says 172.21.15.39:54954) 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f750f7fe640 1 -- 172.21.15.39:0/1970480591 learned_addr learned my addr 172.21.15.39:0/1970480591 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.375+0000 7f750e7fc640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7510103d90 0x7f751019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f750effd640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7510102b50 0x7f751019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f750f7fe640 1 -- 172.21.15.39:0/1970480591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7510102b50 msgr2=0x7f751019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f750f7fe640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7510102b50 0x7f751019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f750f7fe640 1 -- 172.21.15.39:0/1970480591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7510103d90 msgr2=0x7f751019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f750f7fe640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7510103d90 0x7f751019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f750f7fe640 1 -- 172.21.15.39:0/1970480591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75101a1d60 con 0x7f7510108a90 2024-05-04T03:12:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f750e7fc640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7510103d90 0x7f751019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f750f7fe640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 0x7f75101a16f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7508031040 tx=0x7f7508003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f7507fff640 1 -- 172.21.15.39:0/1970480591 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508004040 con 0x7f7510108a90 2024-05-04T03:12:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f7507fff640 1 -- 172.21.15.39:0/1970480591 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7508004670 con 0x7f7510108a90 2024-05-04T03:12:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f75152f2640 1 -- 172.21.15.39:0/1970480591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75101a1ff0 con 0x7f7510108a90 2024-05-04T03:12:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f7507fff640 1 -- 172.21.15.39:0/1970480591 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508030d50 con 0x7f7510108a90 2024-05-04T03:12:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.376+0000 7f75152f2640 1 -- 172.21.15.39:0/1970480591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75101a2450 con 0x7f7510108a90 2024-05-04T03:12:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.379+0000 7f7507fff640 1 -- 172.21.15.39:0/1970480591 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f75080305c0 con 0x7f7510108a90 2024-05-04T03:12:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.379+0000 7f751c1a2740 1 Processor -- start 2024-05-04T03:12:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.379+0000 7f751c1a2740 1 -- start start 2024-05-04T03:12:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 0x561dc973be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc936d2f0 0x561dc973c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 0x561dc97432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f751c1a2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561dc969c5d0 con 0x561dc936d2f0 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f751c1a2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561dc9655890 con 0x561dc93754d0 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f751c1a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561dc9698590 con 0x561dc93961b0 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc936d2f0 0x561dc973c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc936d2f0 0x561dc973c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54970/0 (socket says 172.21.15.39:54970) 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 -- 172.21.15.39:0/4272483008 learned_addr learned my addr 172.21.15.39:0/4272483008 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750e7fc640 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 0x561dc973be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f7507fff640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f74fc0b27e0 0x7f74fc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750e7fc640 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 0x561dc97432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 -- 172.21.15.39:0/4272483008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 msgr2=0x561dc973be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 0x561dc973be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 -- 172.21.15.39:0/4272483008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 msgr2=0x561dc97432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 0x561dc97432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 -- 172.21.15.39:0/4272483008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc93ae210 con 0x561dc936d2f0 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750f7fe640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f74fc0b27e0 0x7f74fc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f7507fff640 1 -- 172.21.15.39:0/1970480591 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7508103a80 con 0x7f7510108a90 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.380+0000 7f750effd640 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc936d2f0 0x561dc973c3f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f74f800bc50 tx=0x7f74f800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3906010a90ff30 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.381+0000 7f75057fa640 1 -- 172.21.15.39:0/4272483008 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8015070 con 0x561dc936d2f0 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.381+0000 7f75057fa640 1 -- 172.21.15.39:0/4272483008 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74f80035f0 con 0x561dc936d2f0 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.381+0000 7f75057fa640 1 -- 172.21.15.39:0/4272483008 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8011680 con 0x561dc936d2f0 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 -- 172.21.15.39:0/4272483008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc936d2f0 msgr2=0x561dc973c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f750f7fe640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f74fc0b27e0 0x7f74fc0b4cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7508101900 tx=0x7f75080949a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc936d2f0 0x561dc973c3f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f74f800bc50 tx=0x7f74f800f730 comp rx=0 tx=0).stop 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 -- 172.21.15.39:0/4272483008 shutdown_connections 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 0x561dc97432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc936d2f0 0x561dc973c3f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 --2- 172.21.15.39:0/4272483008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 0x561dc973be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 -- 172.21.15.39:0/4272483008 >> 172.21.15.39:0/4272483008 conn(0x561dc938a300 msgr2=0x561dc97333a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 -- 172.21.15.39:0/4272483008 shutdown_connections 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 -- 172.21.15.39:0/4272483008 wait complete. 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 -- start start 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.382+0000 7f751c1a2740 1 Processor -- start 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc9745ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc9745ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93961b0 0x561dc974cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f751c1a2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561dc9743d40 con 0x561dc93754d0 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f751c1a2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561dc9659370 con 0x561dc93961b0 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f751c1a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561dc9743ec0 con 0x561dc936d2f0 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f751c1a2740 10 client.0 _ll_register_callbacks cb 0x561dc93ada00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc9745ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc9745ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54976/0 (socket says 172.21.15.39:54976) 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 -- 172.21.15.39:0/1739941582 learned_addr learned my addr 172.21.15.39:0/1739941582 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750e7fc640 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc9745ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750e7fc640 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93961b0 0x561dc974cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 -- 172.21.15.39:0/1739941582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 msgr2=0x561dc9745ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc9745ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 -- 172.21.15.39:0/1739941582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93961b0 msgr2=0x561dc974cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93961b0 0x561dc974cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 -- 172.21.15.39:0/1739941582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc947a620 con 0x561dc93754d0 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750e7fc640 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc9745ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f750effd640 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc9745ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f74f800bf70 tx=0x7f74f800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f0c3acc3bb1f99 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f74f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f74f67fc640 1 -- 172.21.15.39:0/1739941582 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8015030 con 0x561dc93754d0 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f74f67fc640 1 -- 172.21.15.39:0/1739941582 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f74f8003af0 con 0x561dc93754d0 2024-05-04T03:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.383+0000 7f751c1a2740 1 -- 172.21.15.39:0/1739941582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74f80035f0 con 0x561dc93754d0 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f74f67fc640 1 -- 172.21.15.39:0/1739941582 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74f8004000 con 0x561dc93754d0 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f751c1a2740 10 client.15594 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f751c1a2740 1 -- 172.21.15.39:0/1739941582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dc974fe80 con 0x561dc93754d0 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f74f67fc640 1 -- 172.21.15.39:0/1739941582 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74f80035f0 con 0x561dc93754d0 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f74f67fc640 1 -- 172.21.15.39:0/1739941582 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f74f80047e0 con 0x561dc93754d0 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f751c1a2740 10 client.15594 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f751c1a2740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f751c1a2740 20 client.15594 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f751c1a2740 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561dc9354d80 0x561dc97533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f751c1a2740 4 client.15594 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f751c1a2740 1 -- 172.21.15.39:0/1739941582 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dc96aa1e0 con 0x561dc9354d80 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.384+0000 7f750e7fc640 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561dc9354d80 0x561dc97533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.385+0000 7f74f67fc640 10 client.15594 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.385+0000 7f750e7fc640 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561dc9354d80 0x561dc97533f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9776f298a688d223 server_cookie=172a9cbe0086e256 in_seq=0 out_seq=0 2024-05-04T03:12:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f74f67fc640 1 -- 172.21.15.39:0/1739941582 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561dc96aa1e0 con 0x561dc9354d80 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f74f67fc640 10 client.15594 handle_command_reply: tid=0 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 client.15594 shutdown 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 -- 172.21.15.39:0/1739941582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 msgr2=0x561dc9745ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc9745ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f74f800bf70 tx=0x7f74f800bfa0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 -- 172.21.15.39:0/1739941582 shutdown_connections 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561dc9354d80 0x561dc97533f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93961b0 0x561dc974cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc9745ff0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 --2- 172.21.15.39:0/1739941582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc9745ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f74f67fc640 0 client.15594 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 -- 172.21.15.39:0/1739941582 >> 172.21.15.39:0/1739941582 conn(0x561dc938a300 msgr2=0x561dc9736300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 -- 172.21.15.39:0/1739941582 shutdown_connections 2024-05-04T03:12:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 1 -- 172.21.15.39:0/1739941582 wait complete. 2024-05-04T03:12:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.386+0000 7f751c1a2740 20 client.15594 trim_cache size 0 max 16384 2024-05-04T03:12:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f751c1a2740 1 Processor -- start 2024-05-04T03:12:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f751c1a2740 1 -- start start 2024-05-04T03:12:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc97442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 0x561dc9744800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93961b0 0x561dc9732bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f751c1a2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f75101a90a0 con 0x561dc93961b0 2024-05-04T03:12:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f751c1a2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75101aaca0 con 0x561dc93754d0 2024-05-04T03:12:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f751c1a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75101a9770 con 0x561dc936d2f0 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f750e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93961b0 0x561dc9732bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f750e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93961b0 0x561dc9732bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54982/0 (socket says 172.21.15.39:54982) 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f750e7fc640 1 -- 172.21.15.39:0/2483425655 learned_addr learned my addr 172.21.15.39:0/2483425655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f750e7fc640 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc97442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.402+0000 7f750effd640 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 0x561dc9744800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.403+0000 7f750e7fc640 1 -- 172.21.15.39:0/2483425655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 msgr2=0x561dc97442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.403+0000 7f750e7fc640 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc97442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.403+0000 7f750e7fc640 1 -- 172.21.15.39:0/2483425655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 msgr2=0x561dc9744800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.403+0000 7f750e7fc640 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 0x561dc9744800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.403+0000 7f750e7fc640 1 -- 172.21.15.39:0/2483425655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc9746690 con 0x561dc93961b0 2024-05-04T03:12:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.403+0000 7f750e7fc640 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93961b0 0x561dc9732bc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x561dc9753b90 tx=0x7f750000c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9db64843de2a7d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.405+0000 7f7504ff9640 1 -- 172.21.15.39:0/2483425655 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500005220 con 0x561dc93961b0 2024-05-04T03:12:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.405+0000 7f7504ff9640 1 -- 172.21.15.39:0/2483425655 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7500004c30 con 0x561dc93961b0 2024-05-04T03:12:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.405+0000 7f7504ff9640 1 -- 172.21.15.39:0/2483425655 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75000116a0 con 0x561dc93961b0 2024-05-04T03:12:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- 172.21.15.39:0/2483425655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93961b0 msgr2=0x561dc9732bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93961b0 0x561dc9732bc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x561dc9753b90 tx=0x7f750000c6f0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- 172.21.15.39:0/2483425655 shutdown_connections 2024-05-04T03:12:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93961b0 0x561dc9732bc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc93754d0 0x561dc9744800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 --2- 172.21.15.39:0/2483425655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc936d2f0 0x561dc97442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- 172.21.15.39:0/2483425655 >> 172.21.15.39:0/2483425655 conn(0x561dc938a300 msgr2=0x561dc973bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- 172.21.15.39:0/2483425655 shutdown_connections 2024-05-04T03:12:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- 172.21.15.39:0/2483425655 wait complete. 2024-05-04T03:12:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- start start 2024-05-04T03:12:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 Processor -- start 2024-05-04T03:12:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc936d2f0 0x561dc974c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc97490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 0x561dc97496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f75101aa510 con 0x561dc93754d0 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75101a9e40 con 0x561dc936d2f0 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75101a7080 con 0x561dc93961b0 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.406+0000 7f751c1a2740 10 client.0 _ll_register_callbacks cb 0x7f750000c590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc97490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc97490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54986/0 (socket says 172.21.15.39:54986) 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 -- 172.21.15.39:0/159161894 learned_addr learned my addr 172.21.15.39:0/159161894 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750e7fc640 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 0x561dc97496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750e7fc640 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc936d2f0 0x561dc974c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 -- 172.21.15.39:0/159161894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 msgr2=0x561dc97496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 0x561dc97496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 -- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc936d2f0 msgr2=0x561dc974c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc936d2f0 0x561dc974c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 -- 172.21.15.39:0/159161894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc97b9790 con 0x561dc93754d0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750e7fc640 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc936d2f0 0x561dc974c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f750effd640 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc97490c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f74f8001eb0 tx=0x7f74f800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bec9a7fc5759e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f74f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f74f67fc640 1 -- 172.21.15.39:0/159161894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8015040 con 0x561dc93754d0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f74f67fc640 1 -- 172.21.15.39:0/159161894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f74f800b3a0 con 0x561dc93754d0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f751c1a2740 1 -- 172.21.15.39:0/159161894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7500004c30 con 0x561dc93754d0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f74f67fc640 1 -- 172.21.15.39:0/159161894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74f803f050 con 0x561dc93754d0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.407+0000 7f751c1a2740 10 client.15600 fetch_fsmap learned FSMap version 30 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f751c1a2740 1 -- 172.21.15.39:0/159161894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dc97ba9d0 con 0x561dc93754d0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f74f67fc640 1 -- 172.21.15.39:0/159161894 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7500004c30 con 0x561dc93754d0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f74f67fc640 1 -- 172.21.15.39:0/159161894 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f74f8011830 con 0x561dc93754d0 2024-05-04T03:12:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f751c1a2740 10 client.15600 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:12:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f751c1a2740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:12:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f751c1a2740 20 client.15600 populate_metadata read hostname 'smithi039' 2024-05-04T03:12:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f751c1a2740 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561dc932a1c0 0x561dc97bd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f751c1a2740 4 client.15600 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:12:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f751c1a2740 1 -- 172.21.15.39:0/159161894 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dc96a4050 con 0x561dc932a1c0 2024-05-04T03:12:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.408+0000 7f750e7fc640 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561dc932a1c0 0x561dc97bd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.409+0000 7f750e7fc640 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561dc932a1c0 0x561dc97bd450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55b8580e1c59fc92 server_cookie=d21d8f375cd8348f in_seq=0 out_seq=0 2024-05-04T03:12:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.409+0000 7f74f67fc640 10 client.15600 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f74f67fc640 1 -- 172.21.15.39:0/159161894 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561dc96a4050 con 0x561dc932a1c0 2024-05-04T03:12:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f74f67fc640 10 client.15600 handle_command_reply: tid=0 2024-05-04T03:12:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 client.15600 shutdown 2024-05-04T03:12:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 -- 172.21.15.39:0/159161894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 msgr2=0x561dc97490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc97490c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f74f8001eb0 tx=0x7f74f800b7b0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 -- 172.21.15.39:0/159161894 shutdown_connections 2024-05-04T03:12:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561dc932a1c0 0x561dc97bd450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561dc93961b0 0x561dc97496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561dc93754d0 0x561dc97490c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f74f67fc640 0 client.15600 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 --2- 172.21.15.39:0/159161894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561dc936d2f0 0x561dc974c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 -- 172.21.15.39:0/159161894 >> 172.21.15.39:0/159161894 conn(0x561dc938a300 msgr2=0x561dc97442c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.413+0000 7f751c1a2740 1 -- 172.21.15.39:0/159161894 shutdown_connections 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f751c1a2740 1 -- 172.21.15.39:0/159161894 wait complete. 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f751c1a2740 20 client.15600 trim_cache size 0 max 16384 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:12:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:12:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 -- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f74fc0b27e0 msgr2=0x7f74fc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f74fc0b27e0 0x7f74fc0b4cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7508101900 tx=0x7f75080949a0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 -- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 msgr2=0x7f75101a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 0x7f75101a16f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7508031040 tx=0x7f7508003ba0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 -- 172.21.15.39:0/1970480591 shutdown_connections 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f74fc0b27e0 0x7f74fc0b4cf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510108a90 0x7f75101a16f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7510103d90 0x7f751019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 --2- 172.21.15.39:0/1970480591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7510102b50 0x7f751019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 -- 172.21.15.39:0/1970480591 >> 172.21.15.39:0/1970480591 conn(0x7f75100fdd40 msgr2=0x7f7510109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 -- 172.21.15.39:0/1970480591 shutdown_connections 2024-05-04T03:12:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:12:52.414+0000 7f75152f2640 1 -- 172.21.15.39:0/1970480591 wait complete. 2024-05-04T03:12:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:52 smithi167 ceph-mon[33614]: pgmap v448: 97 pgs: 97 active+clean; 506 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 16 op/s 2024-05-04T03:12:52.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:52 smithi039 ceph-mon[29715]: pgmap v448: 97 pgs: 97 active+clean; 506 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 16 op/s 2024-05-04T03:12:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:52 smithi066 ceph-mon[33264]: pgmap v448: 97 pgs: 97 active+clean; 506 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 16 op/s 2024-05-04T03:12:54.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:54 smithi167 ceph-mon[33614]: pgmap v449: 97 pgs: 97 active+clean; 554 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-05-04T03:12:54.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:54 smithi039 ceph-mon[29715]: pgmap v449: 97 pgs: 97 active+clean; 554 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-05-04T03:12:54.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:54 smithi066 ceph-mon[33264]: pgmap v449: 97 pgs: 97 active+clean; 554 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-05-04T03:12:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:56 smithi167 ceph-mon[33614]: pgmap v450: 97 pgs: 97 active+clean; 562 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 22 op/s 2024-05-04T03:12:56.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:56 smithi039 ceph-mon[29715]: pgmap v450: 97 pgs: 97 active+clean; 562 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 22 op/s 2024-05-04T03:12:56.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:56 smithi066 ceph-mon[33264]: pgmap v450: 97 pgs: 97 active+clean; 562 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 22 op/s 2024-05-04T03:12:58.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:12:58 smithi167 ceph-mon[33614]: pgmap v451: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-04T03:12:58.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:12:58 smithi039 ceph-mon[29715]: pgmap v451: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-04T03:12:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:12:58 smithi066 ceph-mon[33264]: pgmap v451: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-04T03:13:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:00 smithi167 ceph-mon[33614]: pgmap v452: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 24 op/s 2024-05-04T03:13:00.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:00 smithi039 ceph-mon[29715]: pgmap v452: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 24 op/s 2024-05-04T03:13:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:00 smithi066 ceph-mon[33264]: pgmap v452: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 24 op/s 2024-05-04T03:13:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:02 smithi167 ceph-mon[33614]: pgmap v453: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-05-04T03:13:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:02 smithi039 ceph-mon[29715]: pgmap v453: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-05-04T03:13:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:02 smithi066 ceph-mon[33264]: pgmap v453: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-05-04T03:13:04.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:04 smithi039 ceph-mon[29715]: pgmap v454: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.0 MiB/s wr, 19 op/s 2024-05-04T03:13:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:04 smithi066 ceph-mon[33264]: pgmap v454: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.0 MiB/s wr, 19 op/s 2024-05-04T03:13:04.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:04 smithi167 ceph-mon[33614]: pgmap v454: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.0 MiB/s wr, 19 op/s 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 -- 172.21.15.39:0/3127677668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 msgr2=0x7f6d74104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 --2- 172.21.15.39:0/3127677668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 0x7f6d74104980 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009900 tx=0x7f6d7002a410 comp rx=0 tx=0).stop 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 -- 172.21.15.39:0/3127677668 shutdown_connections 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 --2- 172.21.15.39:0/3127677668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d74109bc0 0x7f6d7410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 --2- 172.21.15.39:0/3127677668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74104ec0 0x7f6d741052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 --2- 172.21.15.39:0/3127677668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 0x7f6d74104980 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 -- 172.21.15.39:0/3127677668 >> 172.21.15.39:0/3127677668 conn(0x7f6d740fee50 msgr2=0x7f6d74101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 -- 172.21.15.39:0/3127677668 shutdown_connections 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 -- 172.21.15.39:0/3127677668 wait complete. 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 Processor -- start 2024-05-04T03:13:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.529+0000 7f6d7c186640 1 -- start start 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d7c186640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 0x7f6d7419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d7c186640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d74104ec0 0x7f6d7419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d7c186640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74109bc0 0x7f6d741a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d7c186640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7410f630 con 0x7f6d74103c80 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d7c186640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7410f4b0 con 0x7f6d74109bc0 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d7c186640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7410f7b0 con 0x7f6d74104ec0 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 0x7f6d7419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 0x7f6d7419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45614/0 (socket says 172.21.15.39:45614) 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 -- 172.21.15.39:0/1111791443 learned_addr learned my addr 172.21.15.39:0/1111791443 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79721640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d74104ec0 0x7f6d7419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74109bc0 0x7f6d741a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 -- 172.21.15.39:0/1111791443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d74104ec0 msgr2=0x7f6d7419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d74104ec0 0x7f6d7419a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 -- 172.21.15.39:0/1111791443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74109bc0 msgr2=0x7f6d741a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74109bc0 0x7f6d741a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 -- 172.21.15.39:0/1111791443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d741a1e10 con 0x7f6d74103c80 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74109bc0 0x7f6d741a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.530+0000 7f6d79f22640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 0x7f6d7419a2d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6d70005760 tx=0x7f6d70005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.531+0000 7f6d66ffd640 1 -- 172.21.15.39:0/1111791443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7003f030 con 0x7f6d74103c80 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.531+0000 7f6d66ffd640 1 -- 172.21.15.39:0/1111791443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d700043d0 con 0x7f6d74103c80 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.531+0000 7f6d7c186640 1 -- 172.21.15.39:0/1111791443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d741a20a0 con 0x7f6d74103c80 2024-05-04T03:13:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.531+0000 7f6d66ffd640 1 -- 172.21.15.39:0/1111791443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70035070 con 0x7f6d74103c80 2024-05-04T03:13:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.531+0000 7f6d7c186640 1 -- 172.21.15.39:0/1111791443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d741a2590 con 0x7f6d74103c80 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.533+0000 7f6d83040740 1 Processor -- start 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d83040740 1 -- start start 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d66ffd640 1 -- 172.21.15.39:0/1111791443 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6d70042030 con 0x7f6d74103c80 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d83040740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c3a1b0 0x558846fdfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d83040740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 0x558846fe03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d83040740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c194d0 0x558846fe72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d83040740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558846f405d0 con 0x558846c3a1b0 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d83040740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558846ef9890 con 0x558846c194d0 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d83040740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558846f3c590 con 0x558846c112f0 2024-05-04T03:13:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d7a723640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c3a1b0 0x558846fdfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d7a723640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c3a1b0 0x558846fdfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45620/0 (socket says 172.21.15.39:45620) 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d7a723640 1 -- 172.21.15.39:0/1255445792 learned_addr learned my addr 172.21.15.39:0/1255445792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d79721640 1 --2- 172.21.15.39:0/1255445792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c194d0 0x558846fe72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d7a723640 1 -- 172.21.15.39:0/1255445792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 msgr2=0x558846fe03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1255445792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 0x558846fe03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d7a723640 1 -- 172.21.15.39:0/1255445792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c194d0 msgr2=0x558846fe72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1255445792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c194d0 0x558846fe72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.534+0000 7f6d7a723640 1 -- 172.21.15.39:0/1255445792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558846c52210 con 0x558846c3a1b0 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.535+0000 7f6d66ffd640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b27e0 0x7f6d600b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.535+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1255445792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c3a1b0 0x558846fdfe70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009fd0 tx=0x7f6d6c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fea9f3be94e6fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.535+0000 7f6d79f22640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b27e0 0x7f6d600b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.535+0000 7f6d5ffff640 1 -- 172.21.15.39:0/1255445792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c011760 con 0x558846c3a1b0 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.535+0000 7f6d66ffd640 1 -- 172.21.15.39:0/1111791443 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d701025e0 con 0x7f6d74103c80 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.535+0000 7f6d5ffff640 1 -- 172.21.15.39:0/1255445792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c011d90 con 0x558846c3a1b0 2024-05-04T03:13:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.535+0000 7f6d5ffff640 1 -- 172.21.15.39:0/1255445792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c01be50 con 0x558846c3a1b0 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 -- 172.21.15.39:0/1255445792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c3a1b0 msgr2=0x558846fdfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d79f22640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b27e0 0x7f6d600b4cf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6d700c9ba0 tx=0x7f6d70097830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 --2- 172.21.15.39:0/1255445792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c3a1b0 0x558846fdfe70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009fd0 tx=0x7f6d6c0108e0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 -- 172.21.15.39:0/1255445792 shutdown_connections 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 --2- 172.21.15.39:0/1255445792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c194d0 0x558846fe72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 --2- 172.21.15.39:0/1255445792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 0x558846fe03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 --2- 172.21.15.39:0/1255445792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c3a1b0 0x558846fdfe70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 -- 172.21.15.39:0/1255445792 >> 172.21.15.39:0/1255445792 conn(0x558846c2e300 msgr2=0x558846fd73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 -- 172.21.15.39:0/1255445792 shutdown_connections 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 -- 172.21.15.39:0/1255445792 wait complete. 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.536+0000 7f6d83040740 1 -- start start 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d83040740 1 Processor -- start 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d83040740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c112f0 0x558846fe9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d83040740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d83040740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c3a1b0 0x558846ff0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d83040740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558846fe7d40 con 0x558846c194d0 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d83040740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558846efd370 con 0x558846c112f0 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d83040740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558846fe7ec0 con 0x558846c3a1b0 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d83040740 10 client.0 _ll_register_callbacks cb 0x558846c51a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d79721640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d79721640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fea000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45630/0 (socket says 172.21.15.39:45630) 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d79721640 1 -- 172.21.15.39:0/1350609885 learned_addr learned my addr 172.21.15.39:0/1350609885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c112f0 0x558846fe9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.537+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c3a1b0 0x558846ff0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d79721640 1 -- 172.21.15.39:0/1350609885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c3a1b0 msgr2=0x558846ff0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d79721640 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c3a1b0 0x558846ff0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d79721640 1 -- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c112f0 msgr2=0x558846fe9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d79721640 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c112f0 0x558846fe9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d79721640 1 -- 172.21.15.39:0/1350609885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558846d1e620 con 0x558846c194d0 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c112f0 0x558846fe9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d79721640 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fea000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d68012700 tx=0x7f6d68012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86fa3c93118e33f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/1350609885 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68004030 con 0x558846c194d0 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/1350609885 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d680041d0 con 0x558846c194d0 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d83040740 1 -- 172.21.15.39:0/1350609885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c011d90 con 0x558846c194d0 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/1350609885 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d680043d0 con 0x558846c194d0 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d83040740 10 client.15612 fetch_fsmap learned FSMap version 30 2024-05-04T03:13:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d83040740 1 -- 172.21.15.39:0/1350609885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558846ff3f60 con 0x558846c194d0 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/1350609885 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c011d90 con 0x558846c194d0 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.538+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/1350609885 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6d680038d0 con 0x558846c194d0 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d83040740 10 client.15612 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d83040740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d83040740 20 client.15612 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d83040740 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558846bf8d80 0x558846ff7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d83040740 4 client.15612 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d83040740 1 -- 172.21.15.39:0/1350609885 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558846f4e1e0 con 0x558846bf8d80 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558846bf8d80 0x558846ff7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d5d7fa640 10 client.15612 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.539+0000 7f6d7a723640 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558846bf8d80 0x558846ff7770 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952db91104ed967a server_cookie=aa4e3db0c431e266 in_seq=0 out_seq=0 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/1350609885 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558846f4e1e0 con 0x558846bf8d80 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d5d7fa640 10 client.15612 handle_command_reply: tid=0 2024-05-04T03:13:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 client.15612 shutdown 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 -- 172.21.15.39:0/1350609885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 msgr2=0x558846fea000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fea000 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d68012700 tx=0x7f6d68012bc0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 -- 172.21.15.39:0/1350609885 shutdown_connections 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558846bf8d80 0x558846ff7770 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c3a1b0 0x558846ff0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fea000 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 --2- 172.21.15.39:0/1350609885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c112f0 0x558846fe9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d5d7fa640 0 client.15612 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 -- 172.21.15.39:0/1350609885 >> 172.21.15.39:0/1350609885 conn(0x558846c2e300 msgr2=0x558846fd63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 -- 172.21.15.39:0/1350609885 shutdown_connections 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 1 -- 172.21.15.39:0/1350609885 wait complete. 2024-05-04T03:13:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.540+0000 7f6d83040740 20 client.15612 trim_cache size 0 max 16384 2024-05-04T03:13:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d83040740 1 Processor -- start 2024-05-04T03:13:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d83040740 1 -- start start 2024-05-04T03:13:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d83040740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c112f0 0x558846fd6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d83040740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c194d0 0x558846fd7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d83040740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c3a1b0 0x558846ff0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d83040740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741a90d0 con 0x558846c112f0 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d83040740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741aacd0 con 0x558846c3a1b0 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d83040740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741a97a0 con 0x558846c194d0 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d7a723640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c112f0 0x558846fd6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d7a723640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c112f0 0x558846fd6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45644/0 (socket says 172.21.15.39:45644) 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.556+0000 7f6d7a723640 1 -- 172.21.15.39:0/2268632109 learned_addr learned my addr 172.21.15.39:0/2268632109 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d79721640 1 --2- 172.21.15.39:0/2268632109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c194d0 0x558846fd7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d7a723640 1 -- 172.21.15.39:0/2268632109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c194d0 msgr2=0x558846fd7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d7a723640 1 --2- 172.21.15.39:0/2268632109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c194d0 0x558846fd7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d7a723640 1 -- 172.21.15.39:0/2268632109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c3a1b0 msgr2=0x558846ff0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d7a723640 1 --2- 172.21.15.39:0/2268632109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c3a1b0 0x558846ff0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d7a723640 1 -- 172.21.15.39:0/2268632109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558846fe0c60 con 0x558846c112f0 2024-05-04T03:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d7a723640 1 --2- 172.21.15.39:0/2268632109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c112f0 0x558846fd6cf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009fd0 tx=0x7f6d6c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0790a9af36d2403 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d5f7fe640 1 -- 172.21.15.39:0/2268632109 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c031070 con 0x558846c112f0 2024-05-04T03:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d5f7fe640 1 -- 172.21.15.39:0/2268632109 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c011bf0 con 0x558846c112f0 2024-05-04T03:13:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.557+0000 7f6d5f7fe640 1 -- 172.21.15.39:0/2268632109 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c024040 con 0x558846c112f0 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 -- 172.21.15.39:0/2268632109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c112f0 msgr2=0x558846fd6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 --2- 172.21.15.39:0/2268632109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c112f0 0x558846fd6cf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009fd0 tx=0x7f6d6c009500 comp rx=0 tx=0).stop 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 -- 172.21.15.39:0/2268632109 shutdown_connections 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 --2- 172.21.15.39:0/2268632109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c3a1b0 0x558846ff0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 --2- 172.21.15.39:0/2268632109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c194d0 0x558846fd7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 --2- 172.21.15.39:0/2268632109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c112f0 0x558846fd6cf0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 -- 172.21.15.39:0/2268632109 >> 172.21.15.39:0/2268632109 conn(0x558846c2e300 msgr2=0x558846fe3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 -- 172.21.15.39:0/2268632109 shutdown_connections 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 -- 172.21.15.39:0/2268632109 wait complete. 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 -- start start 2024-05-04T03:13:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.558+0000 7f6d83040740 1 Processor -- start 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d83040740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 0x558846feca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d83040740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d83040740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c3a1b0 0x558846fefea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d83040740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741aa540 con 0x558846c194d0 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d83040740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741a9e70 con 0x558846c3a1b0 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d83040740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d741a70b0 con 0x558846c112f0 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d83040740 10 client.0 _ll_register_callbacks cb 0x7f6d68009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fecf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45646/0 (socket says 172.21.15.39:45646) 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 -- 172.21.15.39:0/229543501 learned_addr learned my addr 172.21.15.39:0/229543501 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d7a723640 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 0x558846feca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 -- 172.21.15.39:0/229543501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 msgr2=0x558846feca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 0x558846feca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 -- 172.21.15.39:0/229543501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c3a1b0 msgr2=0x558846fefea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c3a1b0 0x558846fefea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 -- 172.21.15.39:0/229543501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558846ff0460 con 0x558846c194d0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d7a723640 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 0x558846feca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d79721640 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fecf60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x558846fd2a80 tx=0x7f6d68004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503a9fb530ad2236 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.559+0000 7f6d5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/229543501 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68034060 con 0x558846c194d0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/229543501 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d68003800 con 0x558846c194d0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 1 -- 172.21.15.39:0/229543501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c011bf0 con 0x558846c194d0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/229543501 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d68003a00 con 0x558846c194d0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 10 client.15618 fetch_fsmap learned FSMap version 30 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 1 -- 172.21.15.39:0/229543501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558846ff1720 con 0x558846c194d0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/229543501 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c011bf0 con 0x558846c194d0 2024-05-04T03:13:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/229543501 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6d6801c7d0 con 0x558846c194d0 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 10 client.15618 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 20 client.15618 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558846bce1c0 0x5588470620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 4 client.15618 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d83040740 1 -- 172.21.15.39:0/229543501 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558846f48050 con 0x558846bce1c0 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.560+0000 7f6d7a723640 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558846bce1c0 0x5588470620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.561+0000 7f6d5d7fa640 10 client.15618 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.561+0000 7f6d7a723640 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558846bce1c0 0x5588470620d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428433a660a62acf server_cookie=3a7e241d74f24853 in_seq=0 out_seq=0 2024-05-04T03:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/229543501 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558846f48050 con 0x558846bce1c0 2024-05-04T03:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d5d7fa640 10 client.15618 handle_command_reply: tid=0 2024-05-04T03:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 client.15618 shutdown 2024-05-04T03:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 -- 172.21.15.39:0/229543501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 msgr2=0x558846fecf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fecf60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x558846fd2a80 tx=0x7f6d68004190 comp rx=0 tx=0).stop 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 -- 172.21.15.39:0/229543501 shutdown_connections 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558846bce1c0 0x5588470620d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558846c3a1b0 0x558846fefea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846c194d0 0x558846fecf60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d5d7fa640 0 client.15618 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 --2- 172.21.15.39:0/229543501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558846c112f0 0x558846feca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.570+0000 7f6d83040740 1 -- 172.21.15.39:0/229543501 >> 172.21.15.39:0/229543501 conn(0x558846c2e300 msgr2=0x558846fd6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d83040740 1 -- 172.21.15.39:0/229543501 shutdown_connections 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d83040740 1 -- 172.21.15.39:0/229543501 wait complete. 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d83040740 20 client.15618 trim_cache size 0 max 16384 2024-05-04T03:13:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:13:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 -- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b27e0 msgr2=0x7f6d600b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b27e0 0x7f6d600b4cf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6d700c9ba0 tx=0x7f6d70097830 comp rx=0 tx=0).stop 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 -- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 msgr2=0x7f6d7419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 0x7f6d7419a2d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6d70005760 tx=0x7f6d70005e20 comp rx=0 tx=0).stop 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 -- 172.21.15.39:0/1111791443 shutdown_connections 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b27e0 0x7f6d600b4cf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d74109bc0 0x7f6d741a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d74104ec0 0x7f6d7419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 --2- 172.21.15.39:0/1111791443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d74103c80 0x7f6d7419a2d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 -- 172.21.15.39:0/1111791443 >> 172.21.15.39:0/1111791443 conn(0x7f6d740fee50 msgr2=0x7f6d74105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 -- 172.21.15.39:0/1111791443 shutdown_connections 2024-05-04T03:13:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:05.571+0000 7f6d7c186640 1 -- 172.21.15.39:0/1111791443 wait complete. 2024-05-04T03:13:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:06 smithi039 ceph-mon[29715]: pgmap v455: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-05-04T03:13:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:06 smithi066 ceph-mon[33264]: pgmap v455: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-05-04T03:13:06.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:06 smithi167 ceph-mon[33614]: pgmap v455: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 MiB/s wr, 11 op/s 2024-05-04T03:13:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:08 smithi039 ceph-mon[29715]: pgmap v456: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 367 KiB/s wr, 13 op/s 2024-05-04T03:13:08.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:08 smithi066 ceph-mon[33264]: pgmap v456: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 367 KiB/s wr, 13 op/s 2024-05-04T03:13:08.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:08 smithi167 ceph-mon[33614]: pgmap v456: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 367 KiB/s wr, 13 op/s 2024-05-04T03:13:10.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:10 smithi066 ceph-mon[33264]: pgmap v457: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-05-04T03:13:10.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:10 smithi167 ceph-mon[33614]: pgmap v457: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-05-04T03:13:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:10 smithi039 ceph-mon[29715]: pgmap v457: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-05-04T03:13:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:12 smithi066 ceph-mon[33264]: pgmap v458: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-04T03:13:12.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:12 smithi167 ceph-mon[33614]: pgmap v458: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-04T03:13:12.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:12 smithi039 ceph-mon[29715]: pgmap v458: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-04T03:13:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:14 smithi066 ceph-mon[33264]: pgmap v459: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-04T03:13:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:14 smithi167 ceph-mon[33614]: pgmap v459: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-04T03:13:14.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:14 smithi039 ceph-mon[29715]: pgmap v459: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-05-04T03:13:16.742 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:16 smithi039 ceph-mon[29715]: pgmap v460: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-04T03:13:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:16 smithi066 ceph-mon[33264]: pgmap v460: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-04T03:13:16.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:16 smithi167 ceph-mon[33614]: pgmap v460: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-04T03:13:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 -- 172.21.15.39:0/951442009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40103da0 msgr2=0x7f3c401041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 --2- 172.21.15.39:0/951442009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40103da0 0x7f3c401041b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009900 tx=0x7f3c3002a410 comp rx=0 tx=0).stop 2024-05-04T03:13:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 -- 172.21.15.39:0/951442009 shutdown_connections 2024-05-04T03:13:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 --2- 172.21.15.39:0/951442009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c40108aa0 0x7f3c4010aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 --2- 172.21.15.39:0/951442009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40103da0 0x7f3c401041b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 --2- 172.21.15.39:0/951442009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40102b60 0x7f3c40103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 -- 172.21.15.39:0/951442009 >> 172.21.15.39:0/951442009 conn(0x7f3c400fdd50 msgr2=0x7f3c40100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 -- 172.21.15.39:0/951442009 shutdown_connections 2024-05-04T03:13:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 -- 172.21.15.39:0/951442009 wait complete. 2024-05-04T03:13:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.671+0000 7f3c45908640 1 Processor -- start 2024-05-04T03:13:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c45908640 1 -- start start 2024-05-04T03:13:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c45908640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40102b60 0x7f3c40075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c45908640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c40103da0 0x7f3c401a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c45908640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40108aa0 0x7f3c401a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c45908640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010e530 con 0x7f3c40108aa0 2024-05-04T03:13:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c45908640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010e3b0 con 0x7f3c40103da0 2024-05-04T03:13:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c45908640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010e6b0 con 0x7f3c40102b60 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40108aa0 0x7f3c401a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40108aa0 0x7f3c401a39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50560/0 (socket says 172.21.15.39:50560) 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3f7fe640 1 -- 172.21.15.39:0/4256974216 learned_addr learned my addr 172.21.15.39:0/4256974216 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c40103da0 0x7f3c401a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3effd640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40102b60 0x7f3c40075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3f7fe640 1 -- 172.21.15.39:0/4256974216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40102b60 msgr2=0x7f3c40075970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3f7fe640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40102b60 0x7f3c40075970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3f7fe640 1 -- 172.21.15.39:0/4256974216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c40103da0 msgr2=0x7f3c401a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3f7fe640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c40103da0 0x7f3c401a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3f7fe640 1 -- 172.21.15.39:0/4256974216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c401a3fe0 con 0x7f3c40108aa0 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.672+0000 7f3c3effd640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40102b60 0x7f3c40075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.673+0000 7f3c3f7fe640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40108aa0 0x7f3c401a39a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800cad0 tx=0x7f3c3800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.673+0000 7f3c44906640 1 -- 172.21.15.39:0/4256974216 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800ae20 con 0x7f3c40108aa0 2024-05-04T03:13:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.673+0000 7f3c45908640 1 -- 172.21.15.39:0/4256974216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c401a42d0 con 0x7f3c40108aa0 2024-05-04T03:13:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.673+0000 7f3c44906640 1 -- 172.21.15.39:0/4256974216 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c38005020 con 0x7f3c40108aa0 2024-05-04T03:13:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.673+0000 7f3c45908640 1 -- 172.21.15.39:0/4256974216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c401a4820 con 0x7f3c40108aa0 2024-05-04T03:13:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.673+0000 7f3c44906640 1 -- 172.21.15.39:0/4256974216 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3801d060 con 0x7f3c40108aa0 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.675+0000 7f3c4c7b6740 1 Processor -- start 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c44906640 1 -- 172.21.15.39:0/4256974216 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3c380051d0 con 0x7f3c40108aa0 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c4c7b6740 1 -- start start 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ef1b0 0x556c5cb90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 0x556c5cb90fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb97ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556c5caf55d0 con 0x556c5c7ce4d0 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c5caae890 con 0x556c5c7ef1b0 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c5caf1590 con 0x556c5c7c62f0 2024-05-04T03:13:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb97ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb97ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50568/0 (socket says 172.21.15.39:50568) 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/467872293 learned_addr learned my addr 172.21.15.39:0/467872293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.676+0000 7f3c3effd640 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 0x556c5cb90fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ef1b0 0x556c5cb90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c44906640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c280b27e0 0x7f3c280b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/467872293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 msgr2=0x556c5cb90fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 0x556c5cb90fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/467872293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ef1b0 msgr2=0x556c5cb90a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ef1b0 0x556c5cb90a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/467872293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5c807210 con 0x556c5c7ce4d0 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c3f7fe640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c280b27e0 0x7f3c280b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb97ee0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3c30031460 tx=0x7f3c30005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c206905b71ccc3d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c44906640 1 -- 172.21.15.39:0/4256974216 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3c380e2560 con 0x7f3c40108aa0 2024-05-04T03:13:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c25ffb640 1 -- 172.21.15.39:0/467872293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30035040 con 0x556c5c7ce4d0 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c25ffb640 1 -- 172.21.15.39:0/467872293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c300044c0 con 0x556c5c7ce4d0 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.677+0000 7f3c25ffb640 1 -- 172.21.15.39:0/467872293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30003b60 con 0x556c5c7ce4d0 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/467872293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 msgr2=0x556c5cb97ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c3f7fe640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c280b27e0 0x7f3c280b4cf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c380a9b20 tx=0x7f3c380057c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb97ee0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3c30031460 tx=0x7f3c30005fd0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/467872293 shutdown_connections 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb97ee0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 0x556c5cb90fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/467872293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ef1b0 0x556c5cb90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/467872293 >> 172.21.15.39:0/467872293 conn(0x556c5c7e3300 msgr2=0x556c5cb8c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/467872293 shutdown_connections 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.678+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/467872293 wait complete. 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 1 -- start start 2024-05-04T03:13:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 1 Processor -- start 2024-05-04T03:13:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 0x556c5cb9a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ce4d0 0x556c5cb9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ef1b0 0x556c5cba1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556c5cb988d0 con 0x556c5c7ef1b0 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c5cab2370 con 0x556c5c7ce4d0 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c5cb98a50 con 0x556c5c7c62f0 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c4c7b6740 10 client.0 _ll_register_callbacks cb 0x556c5c806a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ef1b0 0x556c5cba1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ef1b0 0x556c5cba1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50574/0 (socket says 172.21.15.39:50574) 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/257525129 learned_addr learned my addr 172.21.15.39:0/257525129 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c3effd640 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ce4d0 0x556c5cb9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.679+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 0x556c5cb9a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/257525129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 msgr2=0x556c5cb9a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 0x556c5cb9a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/257525129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ce4d0 msgr2=0x556c5cb9ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ce4d0 0x556c5cb9ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/257525129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5c8d3620 con 0x556c5c7ef1b0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ef1b0 0x556c5cba1ae0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009a20 tx=0x7f3c30031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45112ea341d6318f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c1effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c1effd640 1 -- 172.21.15.39:0/257525129 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30035070 con 0x556c5c7ef1b0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c1effd640 1 -- 172.21.15.39:0/257525129 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3c30003c90 con 0x556c5c7ef1b0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/257525129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c5cb031e0 con 0x556c5c7ef1b0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c1effd640 1 -- 172.21.15.39:0/257525129 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c30003e90 con 0x556c5c7ef1b0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c4c7b6740 10 client.15630 fetch_fsmap learned FSMap version 30 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/257525129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c5cba4a10 con 0x556c5c7ef1b0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.680+0000 7f3c1effd640 1 -- 172.21.15.39:0/257525129 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556c5cb031e0 con 0x556c5c7ef1b0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c1effd640 1 -- 172.21.15.39:0/257525129 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3c3003dbe0 con 0x556c5c7ef1b0 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c4c7b6740 10 client.15630 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c4c7b6740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c4c7b6740 20 client.15630 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c5c7add80 0x556c5cba7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c4c7b6740 4 client.15630 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/257525129 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c5cafd050 con 0x556c5c7add80 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c3effd640 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c5c7add80 0x556c5cba7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c3effd640 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c5c7add80 0x556c5cba7d70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0d5a04baf007ae server_cookie=331264592cae584 in_seq=0 out_seq=0 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.681+0000 7f3c1effd640 10 client.15630 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.682+0000 7f3c1effd640 1 -- 172.21.15.39:0/257525129 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c5cafd050 con 0x556c5c7add80 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.682+0000 7f3c1effd640 10 client.15630 handle_command_reply: tid=0 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.682+0000 7f3c4c7b6740 1 client.15630 shutdown 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/257525129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ef1b0 msgr2=0x556c5cba1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ef1b0 0x556c5cba1ae0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009a20 tx=0x7f3c30031980 comp rx=0 tx=0).stop 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/257525129 shutdown_connections 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c5c7add80 0x556c5cba7d70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ef1b0 0x556c5cba1ae0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7ce4d0 0x556c5cb9ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c1effd640 0 client.15630 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/257525129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7c62f0 0x556c5cb9a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/257525129 >> 172.21.15.39:0/257525129 conn(0x556c5c7e3300 msgr2=0x556c5cb8f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/257525129 shutdown_connections 2024-05-04T03:13:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/257525129 wait complete. 2024-05-04T03:13:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.683+0000 7f3c4c7b6740 20 client.15630 trim_cache size 0 max 16384 2024-05-04T03:13:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c4c7b6740 1 Processor -- start 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c4c7b6740 1 -- start start 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 0x556c5cb99530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cba1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401ad490 con 0x556c5c7ce4d0 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401af090 con 0x556c5c7c62f0 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401adb60 con 0x556c5c7add80 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb8bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50584/0 (socket says 172.21.15.39:50584) 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.698+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/2606766261 learned_addr learned my addr 172.21.15.39:0/2606766261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3effd640 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cba1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 0x556c5cb99530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/2606766261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 msgr2=0x556c5cb99530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 0x556c5cb99530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/2606766261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 msgr2=0x556c5cba1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cba1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/2606766261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5cb9b240 con 0x556c5c7ce4d0 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3effd640 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cba1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb8bbc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3c300057c0 tx=0x7f3c30005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2267f70925be72 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c257fa640 1 -- 172.21.15.39:0/2606766261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30035070 con 0x556c5c7ce4d0 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c257fa640 1 -- 172.21.15.39:0/2606766261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c30054030 con 0x556c5c7ce4d0 2024-05-04T03:13:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.699+0000 7f3c257fa640 1 -- 172.21.15.39:0/2606766261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c300589e0 con 0x556c5c7ce4d0 2024-05-04T03:13:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/2606766261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 msgr2=0x556c5cb8bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb8bbc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3c300057c0 tx=0x7f3c30005fd0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/2606766261 shutdown_connections 2024-05-04T03:13:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cb8bbc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cba1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/2606766261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 0x556c5cb99530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/2606766261 >> 172.21.15.39:0/2606766261 conn(0x556c5c7e3300 msgr2=0x556c5cb98e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/2606766261 shutdown_connections 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/2606766261 wait complete. 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.700+0000 7f3c4c7b6740 1 -- start start 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c4c7b6740 1 Processor -- start 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 0x556c5cb9eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cb9f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c4c7b6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cba0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401ae900 con 0x556c5c7ce4d0 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401ae230 con 0x556c5c7c62f0 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c4c7b6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401ab470 con 0x556c5c7add80 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c4c7b6740 10 client.0 _ll_register_callbacks cb 0x7f3c3400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cba0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cba0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50594/0 (socket says 172.21.15.39:50594) 2024-05-04T03:13:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/4287268888 learned_addr learned my addr 172.21.15.39:0/4287268888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3effd640 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cb9f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 0x556c5cb9eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/4287268888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 msgr2=0x556c5cb9eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 0x556c5cb9eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/4287268888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 msgr2=0x556c5cb9f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cb9f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 -- 172.21.15.39:0/4287268888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5cba0a20 con 0x556c5c7ce4d0 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.701+0000 7f3c3e7fc640 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cba0460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3c30058960 tx=0x7f3c30049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46801c836fa0d76d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c1effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c1effd640 1 -- 172.21.15.39:0/4287268888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30035070 con 0x556c5c7ce4d0 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/4287268888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c30054030 con 0x556c5c7ce4d0 2024-05-04T03:13:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c1effd640 1 -- 172.21.15.39:0/4287268888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3c30004240 con 0x556c5c7ce4d0 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c1effd640 1 -- 172.21.15.39:0/4287268888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c300589e0 con 0x556c5c7ce4d0 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 10 client.15636 fetch_fsmap learned FSMap version 30 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/4287268888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c5cba1c20 con 0x556c5c7ce4d0 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c1effd640 1 -- 172.21.15.39:0/4287268888 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c30054030 con 0x556c5c7ce4d0 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c1effd640 1 -- 172.21.15.39:0/4287268888 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3c3003dc80 con 0x556c5c7ce4d0 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 10 client.15636 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 20 client.15636 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c5c7ef1b0 0x556c5cc126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 4 client.15636 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.702+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/4287268888 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c5cb913e0 con 0x556c5c7ef1b0 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.703+0000 7f3c3effd640 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c5c7ef1b0 0x556c5cc126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.703+0000 7f3c3effd640 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c5c7ef1b0 0x556c5cc126f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5bd9df190a503eb server_cookie=18d3ea9d9e5106cb in_seq=0 out_seq=0 2024-05-04T03:13:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.703+0000 7f3c1effd640 10 client.15636 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.707+0000 7f3c1effd640 1 -- 172.21.15.39:0/4287268888 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c5cb913e0 con 0x556c5c7ef1b0 2024-05-04T03:13:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.707+0000 7f3c1effd640 10 client.15636 handle_command_reply: tid=0 2024-05-04T03:13:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.707+0000 7f3c4c7b6740 1 client.15636 shutdown 2024-05-04T03:13:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/4287268888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 msgr2=0x556c5cba0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cba0460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3c30058960 tx=0x7f3c30049040 comp rx=0 tx=0).stop 2024-05-04T03:13:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/4287268888 shutdown_connections 2024-05-04T03:13:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c5c7ef1b0 0x556c5cc126f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c5c7ce4d0 0x556c5cba0460 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c5c7c62f0 0x556c5cb9f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c1effd640 0 client.15636 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 --2- 172.21.15.39:0/4287268888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c5c7add80 0x556c5cb9eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/4287268888 >> 172.21.15.39:0/4287268888 conn(0x556c5c7e3300 msgr2=0x556c5cb8f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/4287268888 shutdown_connections 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 1 -- 172.21.15.39:0/4287268888 wait complete. 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c4c7b6740 20 client.15636 trim_cache size 0 max 16384 2024-05-04T03:13:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:13:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:13:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:13:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:13:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 -- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c280b27e0 msgr2=0x7f3c280b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c280b27e0 0x7f3c280b4cf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c380a9b20 tx=0x7f3c380057c0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 -- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40108aa0 msgr2=0x7f3c401a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40108aa0 0x7f3c401a39a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800cad0 tx=0x7f3c3800cf90 comp rx=0 tx=0).stop 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 -- 172.21.15.39:0/4256974216 shutdown_connections 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3c280b27e0 0x7f3c280b4cf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c40108aa0 0x7f3c401a39a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c40103da0 0x7f3c401a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 --2- 172.21.15.39:0/4256974216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c40102b60 0x7f3c40075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.708+0000 7f3c45908640 1 -- 172.21.15.39:0/4256974216 >> 172.21.15.39:0/4256974216 conn(0x7f3c400fdd50 msgr2=0x7f3c40104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.709+0000 7f3c45908640 1 -- 172.21.15.39:0/4256974216 shutdown_connections 2024-05-04T03:13:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:18.709+0000 7f3c45908640 1 -- 172.21.15.39:0/4256974216 wait complete. 2024-05-04T03:13:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:18 smithi066 ceph-mon[33264]: pgmap v461: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-04T03:13:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:18 smithi167 ceph-mon[33614]: pgmap v461: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-04T03:13:18.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:18 smithi039 ceph-mon[29715]: pgmap v461: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-04T03:13:20.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:20 smithi066 ceph-mon[33264]: pgmap v462: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-04T03:13:20.889 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:13:20.917 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:20 smithi039 ceph-mon[29715]: pgmap v462: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-04T03:13:20.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:20 smithi167 ceph-mon[33614]: pgmap v462: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-04T03:13:21.155 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:13:22.631 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:22 smithi039 ceph-mon[29715]: pgmap v463: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:22.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:22 smithi066 ceph-mon[33264]: pgmap v463: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:22.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:22 smithi167 ceph-mon[33614]: pgmap v463: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 -- 172.21.15.39:0/3718578289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6701045f0 msgr2=0x7ff670104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 --2- 172.21.15.39:0/3718578289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6701045f0 0x7ff670104a00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff6600099c0 tx=0x7ff660009cd0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 -- 172.21.15.39:0/3718578289 shutdown_connections 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 --2- 172.21.15.39:0/3718578289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6701045f0 0x7ff670104a00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 --2- 172.21.15.39:0/3718578289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff670109870 0x7ff6701111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 --2- 172.21.15.39:0/3718578289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff670108f20 0x7ff670109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 -- 172.21.15.39:0/3718578289 >> 172.21.15.39:0/3718578289 conn(0x7ff670100040 msgr2=0x7ff670102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 -- 172.21.15.39:0/3718578289 shutdown_connections 2024-05-04T03:13:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.066+0000 7ff6773da700 1 -- 172.21.15.39:0/3718578289 wait complete. 2024-05-04T03:13:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.067+0000 7ff6773da700 1 Processor -- start 2024-05-04T03:13:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.067+0000 7ff6773da700 1 -- start start 2024-05-04T03:13:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff6773da700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701045f0 0x7ff67019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff6773da700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670108f20 0x7ff67019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff6773da700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff670109870 0x7ff67019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff6773da700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff670114350 con 0x7ff670108f20 2024-05-04T03:13:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff6773da700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701141d0 con 0x7ff670109870 2024-05-04T03:13:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff6773da700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701144d0 con 0x7ff6701045f0 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670108f20 0x7ff67019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670108f20 0x7ff67019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50616/0 (socket says 172.21.15.39:50616) 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 -- 172.21.15.39:0/4081259211 learned_addr learned my addr 172.21.15.39:0/4081259211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff675176700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701045f0 0x7ff67019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff675977700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff670109870 0x7ff67019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 -- 172.21.15.39:0/4081259211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701045f0 msgr2=0x7ff67019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701045f0 0x7ff67019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 -- 172.21.15.39:0/4081259211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff670109870 msgr2=0x7ff67019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff670109870 0x7ff67019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 -- 172.21.15.39:0/4081259211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67019fc10 con 0x7ff670108f20 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff675176700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701045f0 0x7ff67019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff675977700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff670109870 0x7ff67019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.068+0000 7ff674975700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670108f20 0x7ff67019b2e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00c8f0 tx=0x7ff66c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.069+0000 7ff6667fc700 1 -- 172.21.15.39:0/4081259211 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c007b00 con 0x7ff670108f20 2024-05-04T03:13:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.069+0000 7ff6667fc700 1 -- 172.21.15.39:0/4081259211 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff66c007ca0 con 0x7ff670108f20 2024-05-04T03:13:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.069+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67019ff00 con 0x7ff670108f20 2024-05-04T03:13:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.069+0000 7ff6667fc700 1 -- 172.21.15.39:0/4081259211 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c018660 con 0x7ff670108f20 2024-05-04T03:13:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.069+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6700759f0 con 0x7ff670108f20 2024-05-04T03:13:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.070+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff670105aa0 con 0x7ff670108f20 2024-05-04T03:13:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.072+0000 7ff6667fc700 1 -- 172.21.15.39:0/4081259211 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff66c00f450 con 0x7ff670108f20 2024-05-04T03:13:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.072+0000 7ff6667fc700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff65c0b2880 0x7ff65c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.072+0000 7ff675176700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff65c0b2880 0x7ff65c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.072+0000 7ff6667fc700 1 -- 172.21.15.39:0/4081259211 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff66c0e2300 con 0x7ff670108f20 2024-05-04T03:13:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.073+0000 7ff675176700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff65c0b2880 0x7ff65c0b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff660000c00 tx=0x7ff660005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.076+0000 7ff6667fc700 1 -- 172.21.15.39:0/4081259211 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff66c014030 con 0x7ff670108f20 2024-05-04T03:13:23.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.343+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff67019c0a0 con 0x7ff670108f20 2024-05-04T03:13:23.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.344+0000 7ff6667fc700 1 -- 172.21.15.39:0/4081259211 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11937 (secure 0 0 0) 0x7ff66c0aa060 con 0x7ff670108f20 2024-05-04T03:13:23.345 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:13:23.345 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:11:36.224039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24472,24506,24466,24490,24455]},"id":1}]} 2024-05-04T03:13:23.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff65c0b2880 msgr2=0x7ff65c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff65c0b2880 0x7ff65c0b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff660000c00 tx=0x7ff660005940 comp rx=0 tx=0).stop 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670108f20 msgr2=0x7ff67019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670108f20 0x7ff67019b2e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00c8f0 tx=0x7ff66c00ccb0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 shutdown_connections 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff65c0b2880 0x7ff65c0b4d90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff670109870 0x7ff67019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6701045f0 0x7ff67019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 --2- 172.21.15.39:0/4081259211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670108f20 0x7ff67019b2e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 >> 172.21.15.39:0/4081259211 conn(0x7ff670100040 msgr2=0x7ff670102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 shutdown_connections 2024-05-04T03:13:23.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:23.346+0000 7ff6773da700 1 -- 172.21.15.39:0/4081259211 wait complete. 2024-05-04T03:13:23.349 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-05-04T03:13:23.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:23 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4081259211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:13:23.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:23 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4081259211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:13:23.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:23 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4081259211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:13:24.108 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 750d580f --release --await' 2024-05-04T03:13:24.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id 750d580f --release --await 2024-05-04T03:13:24.399 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:13:24.766 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:24 smithi039 ceph-mon[29715]: pgmap v464: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:24 smithi066 ceph-mon[33264]: pgmap v464: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:24.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:24 smithi167 ceph-mon[33614]: pgmap v464: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:26.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 -- 172.21.15.39:0/3665778724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300108f10 msgr2=0x7f6300109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 --2- 172.21.15.39:0/3665778724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300108f10 0x7f6300109320 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f62f00099c0 tx=0x7f62f0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 -- 172.21.15.39:0/3665778724 shutdown_connections 2024-05-04T03:13:26.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 --2- 172.21.15.39:0/3665778724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6300109860 0x7f63001111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 --2- 172.21.15.39:0/3665778724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6300104630 0x7f6300104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 --2- 172.21.15.39:0/3665778724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300108f10 0x7f6300109320 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 -- 172.21.15.39:0/3665778724 >> 172.21.15.39:0/3665778724 conn(0x7f63001000a0 msgr2=0x7f63001024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:26.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 -- 172.21.15.39:0/3665778724 shutdown_connections 2024-05-04T03:13:26.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.181+0000 7f6307cd4700 1 -- 172.21.15.39:0/3665778724 wait complete. 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6307cd4700 1 Processor -- start 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6307cd4700 1 -- start start 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6307cd4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300104630 0x7f630019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6307cd4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6300108f10 0x7f630019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6307cd4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6300109860 0x7f630019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6307cd4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6300114320 con 0x7f6300104630 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6307cd4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63001141a0 con 0x7f6300108f10 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6307cd4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63001144a0 con 0x7f6300109860 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6305a70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300104630 0x7f630019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6305a70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300104630 0x7f630019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40620/0 (socket says 172.21.15.39:40620) 2024-05-04T03:13:26.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6305a70700 1 -- 172.21.15.39:0/1142588525 learned_addr learned my addr 172.21.15.39:0/1142588525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f630526f700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6300108f10 0x7f630019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.182+0000 7f6306271700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6300109860 0x7f630019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f6305a70700 1 -- 172.21.15.39:0/1142588525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6300109860 msgr2=0x7f630019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f6305a70700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6300109860 0x7f630019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f6305a70700 1 -- 172.21.15.39:0/1142588525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6300108f10 msgr2=0x7f630019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f6305a70700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6300108f10 0x7f630019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f6305a70700 1 -- 172.21.15.39:0/1142588525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630019fc40 con 0x7f6300104630 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f6305a70700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300104630 0x7f630019ad70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00d950 tx=0x7f62fc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f62f6ffd700 1 -- 172.21.15.39:0/1142588525 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc016070 con 0x7f6300104630 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f62f6ffd700 1 -- 172.21.15.39:0/1142588525 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62fc010460 con 0x7f6300104630 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f6307cd4700 1 -- 172.21.15.39:0/1142588525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630019ff30 con 0x7f6300104630 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f62f6ffd700 1 -- 172.21.15.39:0/1142588525 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc00f620 con 0x7f6300104630 2024-05-04T03:13:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.183+0000 7f6307cd4700 1 -- 172.21.15.39:0/1142588525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001a7850 con 0x7f6300104630 2024-05-04T03:13:26.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.186+0000 7f62f6ffd700 1 -- 172.21.15.39:0/1142588525 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f62fc019070 con 0x7f6300104630 2024-05-04T03:13:26.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.187+0000 7f62f6ffd700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f62ec0b2880 0x7f62ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.187+0000 7f630526f700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f62ec0b2880 0x7f62ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.187+0000 7f62f6ffd700 1 -- 172.21.15.39:0/1142588525 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f62fc0e21c0 con 0x7f6300104630 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.187+0000 7f63166e8b80 1 Processor -- start 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f63166e8b80 1 -- start start 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f630526f700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f62ec0b2880 0x7f62ec0b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009990 tx=0x7f62f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363a5d0 0x556c8363a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c836453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c836458e0 0x556c83647d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f63166e8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556c8355d960 con 0x556c8363a5d0 2024-05-04T03:13:26.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f63166e8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c834fc8c0 con 0x556c8363af20 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f63166e8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c83559920 con 0x556c836458e0 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6305a70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363a5d0 0x556c8363a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6305a70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363a5d0 0x556c8363a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40628/0 (socket says 172.21.15.39:40628) 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6305a70700 1 -- 172.21.15.39:0/2482536392 learned_addr learned my addr 172.21.15.39:0/2482536392 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6306271700 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c836458e0 0x556c83647d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f630526f700 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c836453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6305a70700 1 -- 172.21.15.39:0/2482536392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c836458e0 msgr2=0x556c83647d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6305a70700 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c836458e0 0x556c83647d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6305a70700 1 -- 172.21.15.39:0/2482536392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 msgr2=0x556c836453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6305a70700 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c836453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6305a70700 1 -- 172.21.15.39:0/2482536392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c834aa9d0 con 0x556c8363a5d0 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f630526f700 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c836453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.188+0000 7f6306271700 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c836458e0 0x556c83647d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:26.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.189+0000 7f6305a70700 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363a5d0 0x556c8363a9e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0a5870 tx=0x7f62fc0a9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52078ca19326a5ec server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:26.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.189+0000 7f62ebfff700 1 -- 172.21.15.39:0/2482536392 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc020070 con 0x556c8363a5d0 2024-05-04T03:13:26.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.189+0000 7f62ebfff700 1 -- 172.21.15.39:0/2482536392 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62fc0adba0 con 0x556c8363a5d0 2024-05-04T03:13:26.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.189+0000 7f62ebfff700 1 -- 172.21.15.39:0/2482536392 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc010930 con 0x556c8363a5d0 2024-05-04T03:13:26.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 -- 172.21.15.39:0/2482536392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363a5d0 msgr2=0x556c8363a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363a5d0 0x556c8363a9e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0a5870 tx=0x7f62fc0a9380 comp rx=0 tx=0).stop 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 -- 172.21.15.39:0/2482536392 shutdown_connections 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363a5d0 0x556c8363a9e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c836458e0 0x556c83647d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 --2- 172.21.15.39:0/2482536392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c836453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 -- 172.21.15.39:0/2482536392 >> 172.21.15.39:0/2482536392 conn(0x556c834bdf40 msgr2=0x556c83636600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 -- 172.21.15.39:0/2482536392 shutdown_connections 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 -- 172.21.15.39:0/2482536392 wait complete. 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.190+0000 7f63166e8b80 1 -- start start 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f63166e8b80 1 Processor -- start 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c836428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c83642e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c836458e0 0x556c83643360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f63166e8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556c8364c9c0 con 0x556c836458e0 2024-05-04T03:13:26.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f63166e8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c834c3d30 con 0x556c8363af20 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f63166e8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c8364cb40 con 0x556c8363a5d0 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f6306271700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c836458e0 0x556c83643360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f63166e8b80 10 client.0 _ll_register_callbacks cb 0x556c834f5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f630526f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c83642e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f6306271700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c836458e0 0x556c83643360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40632/0 (socket says 172.21.15.39:40632) 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f6306271700 1 -- 172.21.15.39:0/3826820353 learned_addr learned my addr 172.21.15.39:0/3826820353 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.191+0000 7f6305a70700 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c836428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f6306271700 1 -- 172.21.15.39:0/3826820353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 msgr2=0x556c836428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f6306271700 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c836428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f6306271700 1 -- 172.21.15.39:0/3826820353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 msgr2=0x556c83642e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f6306271700 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c83642e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f6306271700 1 -- 172.21.15.39:0/3826820353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c834f9790 con 0x556c836458e0 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f6306271700 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c836458e0 0x556c83643360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f62f80078f0 tx=0x7f62f8007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6424a75134387f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:26.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f62e97fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f62e97fa700 1 -- 172.21.15.39:0/3826820353 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8004bb0 con 0x556c836458e0 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f62e97fa700 1 -- 172.21.15.39:0/3826820353 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f62f8012040 con 0x556c836458e0 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f63166e8b80 1 -- 172.21.15.39:0/3826820353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62fc0adba0 con 0x556c836458e0 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f62e97fa700 1 -- 172.21.15.39:0/3826820353 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62f8004d50 con 0x556c836458e0 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f63166e8b80 10 client.15654 fetch_fsmap learned FSMap version 30 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f63166e8b80 1 -- 172.21.15.39:0/3826820353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c836576d0 con 0x556c836458e0 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.192+0000 7f62e97fa700 1 -- 172.21.15.39:0/3826820353 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62fc0adba0 con 0x556c836458e0 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f62e97fa700 1 -- 172.21.15.39:0/3826820353 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62f8016760 con 0x556c836458e0 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f63166e8b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f63166e8b80 10 client.15654 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f63166e8b80 20 client.15654 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f63166e8b80 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c83659d60 0x556c8365c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f63166e8b80 4 client.15654 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f63166e8b80 1 -- 172.21.15.39:0/3826820353 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c8356b340 con 0x556c83659d60 2024-05-04T03:13:26.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f6305a70700 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c83659d60 0x556c8365c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f6305a70700 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c83659d60 0x556c8365c1d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4374260d6f0cf28b server_cookie=e7546eca931d6575 in_seq=0 out_seq=0 2024-05-04T03:13:26.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.193+0000 7f62e97fa700 10 client.15654 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.194+0000 7f62e97fa700 1 -- 172.21.15.39:0/3826820353 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c8356b340 con 0x556c83659d60 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.194+0000 7f62e97fa700 10 client.15654 handle_command_reply: tid=0 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.194+0000 7f63166e8b80 1 client.15654 shutdown 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.194+0000 7f63166e8b80 1 -- 172.21.15.39:0/3826820353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c836458e0 msgr2=0x556c83643360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.194+0000 7f63166e8b80 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c836458e0 0x556c83643360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f62f80078f0 tx=0x7f62f8007c00 comp rx=0 tx=0).stop 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.194+0000 7f63166e8b80 1 -- 172.21.15.39:0/3826820353 shutdown_connections 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f63166e8b80 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c83659d60 0x556c8365c1d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f63166e8b80 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c836458e0 0x556c83643360 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f63166e8b80 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c836428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f62e97fa700 0 client.15654 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f63166e8b80 1 --2- 172.21.15.39:0/3826820353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c83642e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f63166e8b80 1 -- 172.21.15.39:0/3826820353 >> 172.21.15.39:0/3826820353 conn(0x556c834bdf40 msgr2=0x556c8363c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:26.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f63166e8b80 1 -- 172.21.15.39:0/3826820353 shutdown_connections 2024-05-04T03:13:26.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f63166e8b80 1 -- 172.21.15.39:0/3826820353 wait complete. 2024-05-04T03:13:26.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.195+0000 7f63166e8b80 20 client.15654 trim_cache size 0 max 16384 2024-05-04T03:13:26.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f63166e8b80 1 Processor -- start 2024-05-04T03:13:26.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f63166e8b80 1 -- start start 2024-05-04T03:13:26.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c834bdf40 0x556c8363e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c83655a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363af20 0x556c83661990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f63166e8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f63001ac9a0 con 0x556c8363af20 2024-05-04T03:13:26.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f63166e8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63001ae5a0 con 0x556c834bdf40 2024-05-04T03:13:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f63166e8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63001ad070 con 0x556c8363a5d0 2024-05-04T03:13:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6306271700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363af20 0x556c83661990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6306271700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363af20 0x556c83661990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40638/0 (socket says 172.21.15.39:40638) 2024-05-04T03:13:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6306271700 1 -- 172.21.15.39:0/2646991091 learned_addr learned my addr 172.21.15.39:0/2646991091 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6305a70700 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c834bdf40 0x556c8363e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f630526f700 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c83655a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6306271700 1 -- 172.21.15.39:0/2646991091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 msgr2=0x556c83655a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6306271700 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c83655a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6306271700 1 -- 172.21.15.39:0/2646991091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c834bdf40 msgr2=0x556c8363e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6306271700 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c834bdf40 0x556c8363e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f6306271700 1 -- 172.21.15.39:0/2646991091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c83644e10 con 0x556c8363af20 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.215+0000 7f630526f700 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c83655a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.216+0000 7f6306271700 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363af20 0x556c83661990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f62f8010230 tx=0x7f62f80093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ed4627177dec0b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.216+0000 7f62eb7fe700 1 -- 172.21.15.39:0/2646991091 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8004a30 con 0x556c8363af20 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.216+0000 7f62eb7fe700 1 -- 172.21.15.39:0/2646991091 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62f8016910 con 0x556c8363af20 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.216+0000 7f62eb7fe700 1 -- 172.21.15.39:0/2646991091 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8015cc0 con 0x556c8363af20 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.216+0000 7f63166e8b80 1 -- 172.21.15.39:0/2646991091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363af20 msgr2=0x556c83661990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.216+0000 7f63166e8b80 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363af20 0x556c83661990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f62f8010230 tx=0x7f62f80093b0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 -- 172.21.15.39:0/2646991091 shutdown_connections 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c8363af20 0x556c83661990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c834bdf40 0x556c8363e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 --2- 172.21.15.39:0/2646991091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c83655a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 -- 172.21.15.39:0/2646991091 >> 172.21.15.39:0/2646991091 conn(0x556c83641d50 msgr2=0x556c83635920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 -- 172.21.15.39:0/2646991091 shutdown_connections 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 -- 172.21.15.39:0/2646991091 wait complete. 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 -- start start 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 Processor -- start 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.217+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c834bdf40 0x556c8364eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c8364f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f63166e8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c8364f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f63166e8b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f63001ade10 con 0x556c834bdf40 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f63166e8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f63001ad740 con 0x556c8363af20 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f63166e8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63001aa960 con 0x556c8363a5d0 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f63166e8b80 10 client.0 _ll_register_callbacks cb 0x556c834f5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c834bdf40 0x556c8364eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c834bdf40 0x556c8364eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40640/0 (socket says 172.21.15.39:40640) 2024-05-04T03:13:26.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 -- 172.21.15.39:0/1398499061 learned_addr learned my addr 172.21.15.39:0/1398499061 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f630526f700 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c8364f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6306271700 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c8364f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 -- 172.21.15.39:0/1398499061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 msgr2=0x556c8364f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c8364f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 -- 172.21.15.39:0/1398499061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 msgr2=0x556c8364f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c8364f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 -- 172.21.15.39:0/1398499061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c836c4550 con 0x556c834bdf40 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f6305a70700 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c834bdf40 0x556c8364eda0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f62fc000c00 tx=0x7f62fc010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24586c11c11816f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f62e97fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f62e97fa700 1 -- 172.21.15.39:0/1398499061 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc020040 con 0x556c834bdf40 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f62e97fa700 1 -- 172.21.15.39:0/1398499061 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f62fc0a5a80 con 0x556c834bdf40 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.218+0000 7f63166e8b80 1 -- 172.21.15.39:0/1398499061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62f8016910 con 0x556c834bdf40 2024-05-04T03:13:26.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f62e97fa700 1 -- 172.21.15.39:0/1398499061 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62fc089050 con 0x556c834bdf40 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f63166e8b80 10 client.15660 fetch_fsmap learned FSMap version 30 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f63166e8b80 1 -- 172.21.15.39:0/1398499061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c836c55c0 con 0x556c834bdf40 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f62e97fa700 1 -- 172.21.15.39:0/1398499061 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62f8016910 con 0x556c834bdf40 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f62e97fa700 1 -- 172.21.15.39:0/1398499061 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f62fc08c020 con 0x556c834bdf40 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f63166e8b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f63166e8b80 10 client.15660 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f63166e8b80 20 client.15660 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f63166e8b80 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c836458e0 0x556c836c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f63166e8b80 4 client.15660 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "750d580f", "release": true, "await": true}] 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f63166e8b80 1 -- 172.21.15.39:0/1398499061 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "750d580f", "release": true, "await": true}) -- 0x556c835653e0 con 0x556c836458e0 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.219+0000 7f630526f700 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c836458e0 0x556c836c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.220+0000 7f630526f700 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c836458e0 0x556c836c7960 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee1cc6458de9f1 server_cookie=f680f47dbe66f569 in_seq=0 out_seq=0 2024-05-04T03:13:26.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.220+0000 7f62e97fa700 10 client.15660 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:26.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.224+0000 7f62e97fa700 1 -- 172.21.15.39:0/1398499061 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x556c835653e0 con 0x556c836458e0 2024-05-04T03:13:26.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.224+0000 7f62e97fa700 10 client.15660 handle_command_reply: tid=0 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.224+0000 7f63166e8b80 1 client.15660 shutdown 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 -- 172.21.15.39:0/1398499061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c834bdf40 msgr2=0x556c8364eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c834bdf40 0x556c8364eda0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f62fc000c00 tx=0x7f62fc010f30 comp rx=0 tx=0).stop 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 -- 172.21.15.39:0/1398499061 shutdown_connections 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c836458e0 0x556c836c7960 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c834bdf40 0x556c8364eda0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c8363af20 0x556c8364f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 --2- 172.21.15.39:0/1398499061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c8363a5d0 0x556c8364f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f62e97fa700 0 client.15660 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:26.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 -- 172.21.15.39:0/1398499061 >> 172.21.15.39:0/1398499061 conn(0x556c83641d50 msgr2=0x556c83636350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:26.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 -- 172.21.15.39:0/1398499061 shutdown_connections 2024-05-04T03:13:26.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 1 -- 172.21.15.39:0/1398499061 wait complete. 2024-05-04T03:13:26.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f63166e8b80 20 client.15660 trim_cache size 0 max 16384 2024-05-04T03:13:26.229 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:13:26.229 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-05-04T03:13:26.230 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:13:26.230 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 8, 2024-05-04T03:13:26.230 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:13:26.230 INFO:teuthology.orchestra.run.smithi039.stdout: "750d580f": { 2024-05-04T03:13:26.230 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 8, 2024-05-04T03:13:26.230 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:13:26.230 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 89.2, 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:13:26.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:13:26.232 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:13:26.232 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:13:26.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.225+0000 7f6307cd4700 1 -- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f62ec0b2880 msgr2=0x7f62ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f62ec0b2880 0x7f62ec0b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009990 tx=0x7f62f001a040 comp rx=0 tx=0).stop 2024-05-04T03:13:26.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 -- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300104630 msgr2=0x7f630019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300104630 0x7f630019ad70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00d950 tx=0x7f62fc00dd10 comp rx=0 tx=0).stop 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 -- 172.21.15.39:0/1142588525 shutdown_connections 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f62ec0b2880 0x7f62ec0b4d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6300104630 0x7f630019ad70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6300109860 0x7f630019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 --2- 172.21.15.39:0/1142588525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6300108f10 0x7f630019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 -- 172.21.15.39:0/1142588525 >> 172.21.15.39:0/1142588525 conn(0x7f63001000a0 msgr2=0x7f630010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 -- 172.21.15.39:0/1142588525 shutdown_connections 2024-05-04T03:13:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:13:26.226+0000 7f6307cd4700 1 -- 172.21.15.39:0/1142588525 wait complete. 2024-05-04T03:13:26.392 INFO:teuthology.orchestra.run.smithi039.stderr:2300000 of 3200000 tuples (71%) done (elapsed 96.37 s, remaining 37.71 s) 2024-05-04T03:13:26.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:26 smithi066 ceph-mon[33264]: pgmap v465: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-04T03:13:26.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:26 smithi167 ceph-mon[33614]: pgmap v465: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-04T03:13:26.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:26 smithi039 ceph-mon[29715]: pgmap v465: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-04T03:13:27.050 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '750d580f', seconds elapsed: 42.0 2024-05-04T03:13:27.661 INFO:teuthology.orchestra.run.smithi039.stderr:2400000 of 3200000 tuples (75%) done (elapsed 97.63 s, remaining 32.54 s) 2024-05-04T03:13:27.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:27 smithi066 ceph-mon[33264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:13:27.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:27 smithi167 ceph-mon[33614]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:13:27.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:27 smithi039 ceph-mon[29715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:13:28.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:28 smithi066 ceph-mon[33264]: pgmap v466: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:28.903 INFO:teuthology.orchestra.run.smithi039.stderr:2500000 of 3200000 tuples (78%) done (elapsed 98.88 s, remaining 27.69 s) 2024-05-04T03:13:28.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:28 smithi167 ceph-mon[33614]: pgmap v466: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:28.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:28 smithi039 ceph-mon[29715]: pgmap v466: 97 pgs: 97 active+clean; 566 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-04T03:13:29.692 INFO:teuthology.orchestra.run.smithi039.stderr:2600000 of 3200000 tuples (81%) done (elapsed 99.67 s, remaining 23.00 s) 2024-05-04T03:13:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:30 smithi066 ceph-mon[33264]: pgmap v467: 97 pgs: 97 active+clean; 582 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.5 MiB/s wr, 12 op/s 2024-05-04T03:13:30.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:30 smithi167 ceph-mon[33614]: pgmap v467: 97 pgs: 97 active+clean; 582 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.5 MiB/s wr, 12 op/s 2024-05-04T03:13:30.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:30 smithi039 ceph-mon[29715]: pgmap v467: 97 pgs: 97 active+clean; 582 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.5 MiB/s wr, 12 op/s 2024-05-04T03:13:30.956 INFO:teuthology.orchestra.run.smithi039.stderr:2700000 of 3200000 tuples (84%) done (elapsed 100.93 s, remaining 18.69 s) 2024-05-04T03:13:31.804 INFO:teuthology.orchestra.run.smithi039.stderr:2800000 of 3200000 tuples (87%) done (elapsed 101.78 s, remaining 14.54 s) 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 -- 172.21.15.39:0/3971879325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954102080 msgr2=0x7f8954106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 --2- 172.21.15.39:0/3971879325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954102080 0x7f8954106cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8948009900 tx=0x7f894802a410 comp rx=0 tx=0).stop 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 -- 172.21.15.39:0/3971879325 shutdown_connections 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 --2- 172.21.15.39:0/3971879325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954109b90 0x7f895410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 --2- 172.21.15.39:0/3971879325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8954107230 0x7f8954109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 --2- 172.21.15.39:0/3971879325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954102080 0x7f8954106cf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 -- 172.21.15.39:0/3971879325 >> 172.21.15.39:0/3971879325 conn(0x7f89540fd4c0 msgr2=0x7f89540ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 -- 172.21.15.39:0/3971879325 shutdown_connections 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.824+0000 7f895ab6b640 1 -- 172.21.15.39:0/3971879325 wait complete. 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f895ab6b640 1 Processor -- start 2024-05-04T03:13:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f895ab6b640 1 -- start start 2024-05-04T03:13:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f895ab6b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8954102080 0x7f89541044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f895ab6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954107230 0x7f8954102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f895ab6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954109b90 0x7f89541030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f895ab6b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f895410f690 con 0x7f8954107230 2024-05-04T03:13:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f895ab6b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f895410f510 con 0x7f8954109b90 2024-05-04T03:13:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f895ab6b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f895410f810 con 0x7f8954102080 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f8953fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954107230 0x7f8954102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f8953fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954107230 0x7f8954102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40664/0 (socket says 172.21.15.39:40664) 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f8959108640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954109b90 0x7f89541030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f8953fff640 1 -- 172.21.15.39:0/2806114187 learned_addr learned my addr 172.21.15.39:0/2806114187 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.825+0000 7f8958907640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8954102080 0x7f89541044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8953fff640 1 -- 172.21.15.39:0/2806114187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8954102080 msgr2=0x7f89541044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8953fff640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8954102080 0x7f89541044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8953fff640 1 -- 172.21.15.39:0/2806114187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954109b90 msgr2=0x7f89541030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8953fff640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954109b90 0x7f89541030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8953fff640 1 -- 172.21.15.39:0/2806114187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89541037d0 con 0x7f8954107230 2024-05-04T03:13:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8953fff640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954107230 0x7f8954102b60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f894000d950 tx=0x7f894000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8951ffb640 1 -- 172.21.15.39:0/2806114187 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940017070 con 0x7f8954107230 2024-05-04T03:13:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f895ab6b640 1 -- 172.21.15.39:0/2806114187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89541a43d0 con 0x7f8954107230 2024-05-04T03:13:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8951ffb640 1 -- 172.21.15.39:0/2806114187 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8940009c30 con 0x7f8954107230 2024-05-04T03:13:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f8951ffb640 1 -- 172.21.15.39:0/2806114187 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894000f620 con 0x7f8954107230 2024-05-04T03:13:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.826+0000 7f895ab6b640 1 -- 172.21.15.39:0/2806114187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89541a4890 con 0x7f8954107230 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8951ffb640 1 -- 172.21.15.39:0/2806114187 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8940010460 con 0x7f8954107230 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8961a1d740 1 Processor -- start 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8961a1d740 1 -- start start 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4eda81b0 0x55de4f149a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed7f2f0 0x55de4f149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8961a1d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55de4f0ae5d0 con 0x55de4eda81b0 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8961a1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55de4f067890 con 0x55de4ed7f2f0 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8961a1d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55de4f0aa590 con 0x55de4ed874d0 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.829+0000 7f8959108640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4eda81b0 0x55de4f149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4eda81b0 0x55de4f149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40668/0 (socket says 172.21.15.39:40668) 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 -- 172.21.15.39:0/865496433 learned_addr learned my addr 172.21.15.39:0/865496433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8958907640 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed7f2f0 0x55de4f149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8953fff640 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8951ffb640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89440b27e0 0x7f89440b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 -- 172.21.15.39:0/865496433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 msgr2=0x55de4f150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 -- 172.21.15.39:0/865496433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed7f2f0 msgr2=0x55de4f149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed7f2f0 0x55de4f149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 -- 172.21.15.39:0/865496433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de4edc0210 con 0x55de4eda81b0 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8958907640 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed7f2f0 0x55de4f149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8951ffb640 1 -- 172.21.15.39:0/2806114187 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f89400e1d80 con 0x7f8954107230 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89440b27e0 0x7f89440b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.830+0000 7f8959108640 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4eda81b0 0x55de4f149a60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f894c00afd0 tx=0x7f894c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4fa1cc8ed221f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.831+0000 7f893effd640 1 -- 172.21.15.39:0/865496433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c024430 con 0x55de4eda81b0 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.831+0000 7f893effd640 1 -- 172.21.15.39:0/865496433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f894c012110 con 0x55de4eda81b0 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.831+0000 7f893effd640 1 -- 172.21.15.39:0/865496433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c005570 con 0x55de4eda81b0 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.831+0000 7f8959108640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89440b27e0 0x7f89440b4cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f894c020000 tx=0x7f894c01c030 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.831+0000 7f8961a1d740 1 -- 172.21.15.39:0/865496433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4eda81b0 msgr2=0x55de4f149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.831+0000 7f8961a1d740 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4eda81b0 0x55de4f149a60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f894c00afd0 tx=0x7f894c00f850 comp rx=0 tx=0).stop 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 -- 172.21.15.39:0/865496433 shutdown_connections 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed7f2f0 0x55de4f149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 --2- 172.21.15.39:0/865496433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4eda81b0 0x55de4f149a60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 -- 172.21.15.39:0/865496433 >> 172.21.15.39:0/865496433 conn(0x55de4ed9c300 msgr2=0x55de4f1453a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 -- 172.21.15.39:0/865496433 shutdown_connections 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 -- 172.21.15.39:0/865496433 wait complete. 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 -- start start 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.832+0000 7f8961a1d740 1 Processor -- start 2024-05-04T03:13:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f153700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f153c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8961a1d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55de4f151930 con 0x55de4ed7f2f0 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8961a1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55de4f06b370 con 0x55de4eda81b0 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8961a1d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55de4f151ab0 con 0x55de4ed874d0 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8961a1d740 10 client.0 _ll_register_callbacks cb 0x55de4edbfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40672/0 (socket says 172.21.15.39:40672) 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 -- 172.21.15.39:0/724969821 learned_addr learned my addr 172.21.15.39:0/724969821 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8953fff640 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8959108640 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 -- 172.21.15.39:0/724969821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 msgr2=0x55de4f153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 -- 172.21.15.39:0/724969821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 msgr2=0x55de4f15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f15aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 -- 172.21.15.39:0/724969821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de4ee8c620 con 0x55de4ed7f2f0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8958907640 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f153700 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8948031460 tx=0x7f8948031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12f13faaed8af07 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.833+0000 7f8933fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8933fff640 1 -- 172.21.15.39:0/724969821 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948035040 con 0x55de4ed7f2f0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 1 -- 172.21.15.39:0/724969821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f894c012110 con 0x55de4ed7f2f0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8933fff640 1 -- 172.21.15.39:0/724969821 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8948004240 con 0x55de4ed7f2f0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8933fff640 1 -- 172.21.15.39:0/724969821 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8948004440 con 0x55de4ed7f2f0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 10 client.15672 fetch_fsmap learned FSMap version 31 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 1 -- 172.21.15.39:0/724969821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55de4f15db40 con 0x55de4ed7f2f0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8933fff640 1 -- 172.21.15.39:0/724969821 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f894c012110 con 0x55de4ed7f2f0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8933fff640 1 -- 172.21.15.39:0/724969821 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f894803c790 con 0x55de4ed7f2f0 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 10 client.15672 fetch_fsmap finished waiting for FSMap version 31 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 20 client.15672 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55de4ed66d80 0x55de4f1612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 4 client.15672 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.834+0000 7f8961a1d740 1 -- 172.21.15.39:0/724969821 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de4f0bc1e0 con 0x55de4ed66d80 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.835+0000 7f8953fff640 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55de4ed66d80 0x55de4f1612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.835+0000 7f8953fff640 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55de4ed66d80 0x55de4f1612a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17bacaaff1b700de server_cookie=2bda8e1703e387ee in_seq=0 out_seq=0 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.835+0000 7f8933fff640 10 client.15672 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8933fff640 1 -- 172.21.15.39:0/724969821 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de4f0bc1e0 con 0x55de4ed66d80 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8933fff640 10 client.15672 handle_command_reply: tid=0 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 client.15672 shutdown 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 -- 172.21.15.39:0/724969821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 msgr2=0x55de4f153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f153700 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8948031460 tx=0x7f8948031a70 comp rx=0 tx=0).stop 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 -- 172.21.15.39:0/724969821 shutdown_connections 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55de4ed66d80 0x55de4f1612a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 --2- 172.21.15.39:0/724969821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f153700 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 -- 172.21.15.39:0/724969821 >> 172.21.15.39:0/724969821 conn(0x55de4ed9c300 msgr2=0x55de4f1443a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8933fff640 0 client.15672 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 -- 172.21.15.39:0/724969821 shutdown_connections 2024-05-04T03:13:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 1 -- 172.21.15.39:0/724969821 wait complete. 2024-05-04T03:13:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.836+0000 7f8961a1d740 20 client.15672 trim_cache size 0 max 16384 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.854+0000 7f8961a1d740 1 Processor -- start 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.854+0000 7f8961a1d740 1 -- start start 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.854+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.854+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed874d0 0x55de4f153810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.854+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4eda81b0 0x55de4f15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.854+0000 7f8961a1d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89541ad4a0 con 0x55de4ed7f2f0 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.854+0000 7f8961a1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89541af010 con 0x55de4ed874d0 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.854+0000 7f8961a1d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89541adb70 con 0x55de4eda81b0 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f1532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40678/0 (socket says 172.21.15.39:40678) 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 -- 172.21.15.39:0/3532070799 learned_addr learned my addr 172.21.15.39:0/3532070799 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8953fff640 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed874d0 0x55de4f153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8959108640 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4eda81b0 0x55de4f15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 -- 172.21.15.39:0/3532070799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4eda81b0 msgr2=0x55de4f15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4eda81b0 0x55de4f15aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 -- 172.21.15.39:0/3532070799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed874d0 msgr2=0x55de4f153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed874d0 0x55de4f153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 -- 172.21.15.39:0/3532070799 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de4f1495f0 con 0x55de4ed7f2f0 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8953fff640 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed874d0 0x55de4f153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f8958907640 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f1532d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f89480057c0 tx=0x7f894802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1148693e8cc9baf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.855+0000 7f893e7fc640 1 -- 172.21.15.39:0/3532070799 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948057070 con 0x55de4ed7f2f0 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f893e7fc640 1 -- 172.21.15.39:0/3532070799 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f894803c9d0 con 0x55de4ed7f2f0 2024-05-04T03:13:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f893e7fc640 1 -- 172.21.15.39:0/3532070799 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948035050 con 0x55de4ed7f2f0 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f8961a1d740 1 -- 172.21.15.39:0/3532070799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 msgr2=0x55de4f1532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f8961a1d740 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f1532d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f89480057c0 tx=0x7f894802a410 comp rx=0 tx=0).stop 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f8961a1d740 1 -- 172.21.15.39:0/3532070799 shutdown_connections 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f8961a1d740 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4eda81b0 0x55de4f15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f8961a1d740 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4ed874d0 0x55de4f153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f8961a1d740 1 --2- 172.21.15.39:0/3532070799 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f1532d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.856+0000 7f8961a1d740 1 -- 172.21.15.39:0/3532070799 >> 172.21.15.39:0/3532070799 conn(0x55de4ed9c300 msgr2=0x55de4f14db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 -- 172.21.15.39:0/3532070799 shutdown_connections 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 -- 172.21.15.39:0/3532070799 wait complete. 2024-05-04T03:13:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 -- start start 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 Processor -- start 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f157f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f1c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89541ae910 con 0x55de4ed7f2f0 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89541ae240 con 0x55de4eda81b0 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.857+0000 7f8961a1d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89541ab480 con 0x55de4ed874d0 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8961a1d740 10 client.0 _ll_register_callbacks cb 0x7f8948005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40692/0 (socket says 172.21.15.39:40692) 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 -- 172.21.15.39:0/425314655 learned_addr learned my addr 172.21.15.39:0/425314655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8953fff640 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8959108640 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f1c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 -- 172.21.15.39:0/425314655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 msgr2=0x55de4f1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f1584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 -- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 msgr2=0x55de4f1c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f1c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 -- 172.21.15.39:0/425314655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de4f15b490 con 0x55de4ed7f2f0 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8959108640 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f1c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8958907640 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f157f90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8948009900 tx=0x7f8948000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2371da91855a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8933fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8933fff640 1 -- 172.21.15.39:0/425314655 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948061060 con 0x55de4ed7f2f0 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8933fff640 1 -- 172.21.15.39:0/425314655 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f894803c740 con 0x55de4ed7f2f0 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.858+0000 7f8961a1d740 1 -- 172.21.15.39:0/425314655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f894803c9d0 con 0x55de4ed7f2f0 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8933fff640 1 -- 172.21.15.39:0/425314655 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8948035070 con 0x55de4ed7f2f0 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8961a1d740 10 client.15678 fetch_fsmap learned FSMap version 31 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8961a1d740 1 -- 172.21.15.39:0/425314655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55de4f1c89f0 con 0x55de4ed7f2f0 2024-05-04T03:13:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8933fff640 1 -- 172.21.15.39:0/425314655 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f894803c9d0 con 0x55de4ed7f2f0 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8933fff640 1 -- 172.21.15.39:0/425314655 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f8948051ab0 con 0x55de4ed7f2f0 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8961a1d740 10 client.15678 fetch_fsmap finished waiting for FSMap version 31 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8961a1d740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8961a1d740 20 client.15678 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8961a1d740 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55de4ed3c1c0 0x55de4f1cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8961a1d740 4 client.15678 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8961a1d740 1 -- 172.21.15.39:0/425314655 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de4f0b6050 con 0x55de4ed3c1c0 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.859+0000 7f8953fff640 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55de4ed3c1c0 0x55de4f1cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.860+0000 7f8933fff640 10 client.15678 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.860+0000 7f8953fff640 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55de4ed3c1c0 0x55de4f1cb380 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=839e0ab5eb2c4b83 server_cookie=b8056a93f71d54bb in_seq=0 out_seq=0 2024-05-04T03:13:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8933fff640 1 -- 172.21.15.39:0/425314655 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de4f0b6050 con 0x55de4ed3c1c0 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8933fff640 10 client.15678 handle_command_reply: tid=0 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 client.15678 shutdown 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 -- 172.21.15.39:0/425314655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 msgr2=0x55de4f157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f157f90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8948009900 tx=0x7f8948000bc0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 -- 172.21.15.39:0/425314655 shutdown_connections 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55de4ed3c1c0 0x55de4f1cb380 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de4eda81b0 0x55de4f1c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de4ed874d0 0x55de4f1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8933fff640 0 client.15678 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 --2- 172.21.15.39:0/425314655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de4ed7f2f0 0x55de4f157f90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 -- 172.21.15.39:0/425314655 >> 172.21.15.39:0/425314655 conn(0x55de4ed9c300 msgr2=0x55de4f153240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 -- 172.21.15.39:0/425314655 shutdown_connections 2024-05-04T03:13:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 1 -- 172.21.15.39:0/425314655 wait complete. 2024-05-04T03:13:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.864+0000 7f8961a1d740 20 client.15678 trim_cache size 0 max 16384 2024-05-04T03:13:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:13:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:13:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:13:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:13:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 -- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89440b27e0 msgr2=0x7f89440b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89440b27e0 0x7f89440b4cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f894c020000 tx=0x7f894c01c030 comp rx=0 tx=0).stop 2024-05-04T03:13:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 -- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954107230 msgr2=0x7f8954102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954107230 0x7f8954102b60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f894000d950 tx=0x7f894000de10 comp rx=0 tx=0).stop 2024-05-04T03:13:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 -- 172.21.15.39:0/2806114187 shutdown_connections 2024-05-04T03:13:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f89440b27e0 0x7f89440b4cf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954109b90 0x7f89541030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8954107230 0x7f8954102b60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 --2- 172.21.15.39:0/2806114187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8954102080 0x7f89541044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 -- 172.21.15.39:0/2806114187 >> 172.21.15.39:0/2806114187 conn(0x7f89540fd4c0 msgr2=0x7f89541054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 -- 172.21.15.39:0/2806114187 shutdown_connections 2024-05-04T03:13:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:31.865+0000 7f895ab6b640 1 -- 172.21.15.39:0/2806114187 wait complete. 2024-05-04T03:13:32.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:32 smithi066 ceph-mon[33264]: pgmap v468: 97 pgs: 97 active+clean; 606 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 13 op/s 2024-05-04T03:13:32.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:32 smithi167 ceph-mon[33614]: pgmap v468: 97 pgs: 97 active+clean; 606 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 13 op/s 2024-05-04T03:13:32.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:32 smithi039 ceph-mon[29715]: pgmap v468: 97 pgs: 97 active+clean; 606 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 MiB/s wr, 13 op/s 2024-05-04T03:13:32.975 INFO:teuthology.orchestra.run.smithi039.stderr:2900000 of 3200000 tuples (90%) done (elapsed 102.95 s, remaining 10.65 s) 2024-05-04T03:13:34.178 INFO:teuthology.orchestra.run.smithi039.stderr:3000000 of 3200000 tuples (93%) done (elapsed 104.15 s, remaining 6.94 s) 2024-05-04T03:13:34.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:34 smithi066 ceph-mon[33264]: pgmap v469: 97 pgs: 97 active+clean; 672 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-05-04T03:13:34.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:34 smithi167 ceph-mon[33614]: pgmap v469: 97 pgs: 97 active+clean; 672 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-05-04T03:13:34.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:34 smithi039 ceph-mon[29715]: pgmap v469: 97 pgs: 97 active+clean; 672 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-05-04T03:13:35.074 INFO:teuthology.orchestra.run.smithi039.stderr:3100000 of 3200000 tuples (96%) done (elapsed 105.05 s, remaining 3.39 s) 2024-05-04T03:13:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:3200000 of 3200000 tuples (100%) done (elapsed 106.29 s, remaining 0.00 s) 2024-05-04T03:13:36.411 INFO:teuthology.orchestra.run.smithi039.stderr:vacuuming... 2024-05-04T03:13:36.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:36 smithi066 ceph-mon[33264]: pgmap v470: 97 pgs: 97 active+clean; 704 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-04T03:13:36.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:36 smithi167 ceph-mon[33614]: pgmap v470: 97 pgs: 97 active+clean; 704 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-04T03:13:36.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:36 smithi039 ceph-mon[29715]: pgmap v470: 97 pgs: 97 active+clean; 704 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 MiB/s wr, 19 op/s 2024-05-04T03:13:37.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:37 smithi066 ceph-mon[33264]: pgmap v471: 97 pgs: 97 active+clean; 748 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-05-04T03:13:37.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:37 smithi167 ceph-mon[33614]: pgmap v471: 97 pgs: 97 active+clean; 748 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-05-04T03:13:37.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:37 smithi039 ceph-mon[29715]: pgmap v471: 97 pgs: 97 active+clean; 748 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-05-04T03:13:40.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:40 smithi167 ceph-mon[33614]: pgmap v472: 97 pgs: 97 active+clean; 776 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 32 op/s 2024-05-04T03:13:40.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:40 smithi039 ceph-mon[29715]: pgmap v472: 97 pgs: 97 active+clean; 776 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 32 op/s 2024-05-04T03:13:40.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:40 smithi066 ceph-mon[33264]: pgmap v472: 97 pgs: 97 active+clean; 776 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 32 op/s 2024-05-04T03:13:42.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:42 smithi167 ceph-mon[33614]: pgmap v473: 97 pgs: 97 active+clean; 780 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 42 MiB/s wr, 35 op/s 2024-05-04T03:13:42.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:42 smithi039 ceph-mon[29715]: pgmap v473: 97 pgs: 97 active+clean; 780 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 42 MiB/s wr, 35 op/s 2024-05-04T03:13:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:42 smithi066 ceph-mon[33264]: pgmap v473: 97 pgs: 97 active+clean; 780 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 42 MiB/s wr, 35 op/s 2024-05-04T03:13:44.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:44 smithi167 ceph-mon[33614]: pgmap v474: 97 pgs: 97 active+clean; 780 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 48 MiB/s wr, 37 op/s 2024-05-04T03:13:44.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:44 smithi039 ceph-mon[29715]: pgmap v474: 97 pgs: 97 active+clean; 780 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 48 MiB/s wr, 37 op/s 2024-05-04T03:13:44.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:44 smithi066 ceph-mon[33264]: pgmap v474: 97 pgs: 97 active+clean; 780 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 48 MiB/s wr, 37 op/s 2024-05-04T03:13:44.801 INFO:teuthology.orchestra.run.smithi039.stderr:creating primary keys... 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 -- 172.21.15.39:0/3562410511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc103da0 msgr2=0x7f5bcc1041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/3562410511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc103da0 0x7f5bcc1041b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009900 tx=0x7f5bbc02a410 comp rx=0 tx=0).stop 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 -- 172.21.15.39:0/3562410511 shutdown_connections 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/3562410511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bcc108aa0 0x7f5bcc10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/3562410511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc103da0 0x7f5bcc1041b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/3562410511 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bcc102b60 0x7f5bcc103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 -- 172.21.15.39:0/3562410511 >> 172.21.15.39:0/3562410511 conn(0x7f5bcc0fdd50 msgr2=0x7f5bcc100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 -- 172.21.15.39:0/3562410511 shutdown_connections 2024-05-04T03:13:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.970+0000 7f5bd3dce640 1 -- 172.21.15.39:0/3562410511 wait complete. 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.971+0000 7f5bd3dce640 1 Processor -- start 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.971+0000 7f5bd3dce640 1 -- start start 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd3dce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bcc102b60 0x7f5bcc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd3dce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bcc103da0 0x7f5bcc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd3dce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc108aa0 0x7f5bcc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd3dce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc10e540 con 0x7f5bcc108aa0 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd3dce640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc10e3c0 con 0x7f5bcc103da0 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd3dce640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc10e6c0 con 0x7f5bcc102b60 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd236b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc108aa0 0x7f5bcc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd236b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc108aa0 0x7f5bcc1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33836/0 (socket says 172.21.15.39:33836) 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1b6a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bcc102b60 0x7f5bcc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd236b640 1 -- 172.21.15.39:0/1333690699 learned_addr learned my addr 172.21.15.39:0/1333690699 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1369640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bcc103da0 0x7f5bcc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/1333690699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bcc103da0 msgr2=0x7f5bcc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bcc103da0 0x7f5bcc1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc108aa0 msgr2=0x7f5bcc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc108aa0 0x7f5bcc1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/1333690699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc1a0cb0 con 0x7f5bcc102b60 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1369640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bcc103da0 0x7f5bcc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd236b640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc108aa0 0x7f5bcc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.972+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bcc102b60 0x7f5bcc199170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc102430 tx=0x7f5bc800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.973+0000 7f5bc2ffd640 1 -- 172.21.15.39:0/1333690699 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800cc40 con 0x7f5bcc102b60 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.973+0000 7f5bc2ffd640 1 -- 172.21.15.39:0/1333690699 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bc80044b0 con 0x7f5bcc102b60 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.973+0000 7f5bd3dce640 1 -- 172.21.15.39:0/1333690699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc1a0fa0 con 0x7f5bcc102b60 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.973+0000 7f5bc2ffd640 1 -- 172.21.15.39:0/1333690699 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8003e10 con 0x7f5bcc102b60 2024-05-04T03:13:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.973+0000 7f5bd3dce640 1 -- 172.21.15.39:0/1333690699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc1a1460 con 0x7f5bcc102b60 2024-05-04T03:13:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.976+0000 7f5bc2ffd640 1 -- 172.21.15.39:0/1333690699 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5bc8003fb0 con 0x7f5bcc102b60 2024-05-04T03:13:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.977+0000 7f5bc2ffd640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5bb80b2860 0x7f5bb80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.977+0000 7f5bdac80740 1 Processor -- start 2024-05-04T03:13:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.977+0000 7f5bd1369640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5bb80b2860 0x7f5bb80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.977+0000 7f5bc2ffd640 1 -- 172.21.15.39:0/1333690699 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5bc80e2650 con 0x7f5bcc102b60 2024-05-04T03:13:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.977+0000 7f5bdac80740 1 -- start start 2024-05-04T03:13:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.977+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa741b0 0x561f2ae17c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd1369640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5bb80b2860 0x7f5bb80b4d70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc006010 tx=0x7f5bbc031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae1f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd1b6a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa741b0 0x561f2ae17c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd1b6a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa741b0 0x561f2ae17c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33846/0 (socket says 172.21.15.39:33846) 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/2109332724 learned_addr learned my addr 172.21.15.39:0/2109332724 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bdac80740 1 -- 172.21.15.39:0/2109332724 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561f2ad7a5d0 con 0x561f2aa741b0 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bdac80740 1 -- 172.21.15.39:0/2109332724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561f2ad33890 con 0x561f2aa4b2f0 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bdac80740 1 -- 172.21.15.39:0/2109332724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f2ad76590 con 0x561f2aa534d0 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd236b640 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae1f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd1369640 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd236b640 1 -- 172.21.15.39:0/2109332724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 msgr2=0x561f2ae181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd236b640 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd236b640 1 -- 172.21.15.39:0/2109332724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa741b0 msgr2=0x561f2ae17c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd236b640 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa741b0 0x561f2ae17c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.978+0000 7f5bd236b640 1 -- 172.21.15.39:0/2109332724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2aa8c210 con 0x561f2aa534d0 2024-05-04T03:13:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.979+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa741b0 0x561f2ae17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:13:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.979+0000 7f5bd236b640 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae1f0f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40117e0 tx=0x7f5bc4011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa9c5a87738954de server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.979+0000 7f5bb7fff640 1 -- 172.21.15.39:0/2109332724 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4011310 con 0x561f2aa534d0 2024-05-04T03:13:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.979+0000 7f5bb7fff640 1 -- 172.21.15.39:0/2109332724 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bc400f9a0 con 0x561f2aa534d0 2024-05-04T03:13:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.979+0000 7f5bb7fff640 1 -- 172.21.15.39:0/2109332724 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4005580 con 0x561f2aa534d0 2024-05-04T03:13:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 -- 172.21.15.39:0/2109332724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 msgr2=0x561f2ae1f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae1f0f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40117e0 tx=0x7f5bc4011ca0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 -- 172.21.15.39:0/2109332724 shutdown_connections 2024-05-04T03:13:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae1f0f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2109332724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa741b0 0x561f2ae17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 -- 172.21.15.39:0/2109332724 >> 172.21.15.39:0/2109332724 conn(0x561f2aa68300 msgr2=0x561f2ae113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 -- 172.21.15.39:0/2109332724 shutdown_connections 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.980+0000 7f5bdac80740 1 -- 172.21.15.39:0/2109332724 wait complete. 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 1 -- start start 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 1 Processor -- start 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 0x561f2ae21ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa741b0 0x561f2ae28c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561f2ae1fb40 con 0x561f2aa4b2f0 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561f2ad37370 con 0x561f2aa534d0 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f2ae1fcc0 con 0x561f2aa741b0 2024-05-04T03:13:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bdac80740 10 client.0 _ll_register_callbacks cb 0x561f2aa8ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bd1b6a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bd1b6a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae21760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33848/0 (socket says 172.21.15.39:33848) 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/2674282526 learned_addr learned my addr 172.21.15.39:0/2674282526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.981+0000 7f5bd236b640 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa741b0 0x561f2ae28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bd1369640 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 0x561f2ae21ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/2674282526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa741b0 msgr2=0x561f2ae28c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa741b0 0x561f2ae28c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/2674282526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 msgr2=0x561f2ae21ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 0x561f2ae21ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/2674282526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2ab58620 con 0x561f2aa4b2f0 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bd236b640 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa741b0 0x561f2ae28c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae21760 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80a6ef0 tx=0x7f5bc80b2420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a08f3171afba7a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bb57fa640 1 -- 172.21.15.39:0/2674282526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80a9c80 con 0x561f2aa4b2f0 2024-05-04T03:13:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bb57fa640 1 -- 172.21.15.39:0/2674282526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5bc80a6270 con 0x561f2aa4b2f0 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bdac80740 1 -- 172.21.15.39:0/2674282526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc400f9a0 con 0x561f2aa4b2f0 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bb57fa640 1 -- 172.21.15.39:0/2674282526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5bc80ae030 con 0x561f2aa4b2f0 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bdac80740 10 client.15690 fetch_fsmap learned FSMap version 31 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.982+0000 7f5bdac80740 1 -- 172.21.15.39:0/2674282526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f2ae2bc00 con 0x561f2aa4b2f0 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bb57fa640 1 -- 172.21.15.39:0/2674282526 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bc400f9a0 con 0x561f2aa4b2f0 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bb57fa640 1 -- 172.21.15.39:0/2674282526 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f5bc80a5680 con 0x561f2aa4b2f0 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bdac80740 10 client.15690 fetch_fsmap finished waiting for FSMap version 31 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bdac80740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bdac80740 20 client.15690 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561f2aa32d80 0x561f2ae2f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bdac80740 4 client.15690 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:13:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bdac80740 1 -- 172.21.15.39:0/2674282526 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f2ad881e0 con 0x561f2aa32d80 2024-05-04T03:13:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.983+0000 7f5bd1369640 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561f2aa32d80 0x561f2ae2f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.984+0000 7f5bb57fa640 10 client.15690 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:44.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.984+0000 7f5bd1369640 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561f2aa32d80 0x561f2ae2f320 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b99cbfa0dd83f01 server_cookie=dde99b01afdc48b2 in_seq=0 out_seq=0 2024-05-04T03:13:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bb57fa640 1 -- 172.21.15.39:0/2674282526 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f2ad881e0 con 0x561f2aa32d80 2024-05-04T03:13:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bb57fa640 10 client.15690 handle_command_reply: tid=0 2024-05-04T03:13:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 client.15690 shutdown 2024-05-04T03:13:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 -- 172.21.15.39:0/2674282526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 msgr2=0x561f2ae21760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae21760 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80a6ef0 tx=0x7f5bc80b2420 comp rx=0 tx=0).stop 2024-05-04T03:13:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 -- 172.21.15.39:0/2674282526 shutdown_connections 2024-05-04T03:13:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561f2aa32d80 0x561f2ae2f320 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa741b0 0x561f2ae28c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bb57fa640 0 client.15690 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 0x561f2ae21ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 --2- 172.21.15.39:0/2674282526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae21760 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.985+0000 7f5bdac80740 1 -- 172.21.15.39:0/2674282526 >> 172.21.15.39:0/2674282526 conn(0x561f2aa68300 msgr2=0x561f2ae14300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.986+0000 7f5bdac80740 1 -- 172.21.15.39:0/2674282526 shutdown_connections 2024-05-04T03:13:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.986+0000 7f5bdac80740 1 -- 172.21.15.39:0/2674282526 wait complete. 2024-05-04T03:13:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:44.986+0000 7f5bdac80740 20 client.15690 trim_cache size 0 max 16384 2024-05-04T03:13:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bdac80740 1 Processor -- start 2024-05-04T03:13:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bdac80740 1 -- start start 2024-05-04T03:13:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa32d80 0x561f2ae11230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae28b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bdac80740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1a90e0 con 0x561f2aa32d80 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bdac80740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1aac50 con 0x561f2aa4b2f0 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bdac80740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1a97b0 con 0x561f2aa534d0 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bd1b6a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa32d80 0x561f2ae11230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bd236b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bd1b6a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa32d80 0x561f2ae11230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33858/0 (socket says 172.21.15.39:33858) 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bd236b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae290d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:50918/0 (socket says 172.21.15.39:50918) 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.010+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/3026964261 learned_addr learned my addr 172.21.15.39:0/3026964261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd1369640 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae28b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/3026964261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 msgr2=0x561f2ae290d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae290d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/3026964261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 msgr2=0x561f2ae28b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae28b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd1b6a640 1 -- 172.21.15.39:0/3026964261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2ae17820 con 0x561f2aa32d80 2024-05-04T03:13:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd1369640 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae28b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:13:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd236b640 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:13:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa32d80 0x561f2ae11230 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80e2970 tx=0x7f5bc800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e43b4d6451d1ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bb77fe640 1 -- 172.21.15.39:0/3026964261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc808a030 con 0x561f2aa32d80 2024-05-04T03:13:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bb77fe640 1 -- 172.21.15.39:0/3026964261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bc80a9c80 con 0x561f2aa32d80 2024-05-04T03:13:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.011+0000 7f5bb77fe640 1 -- 172.21.15.39:0/3026964261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80b2ae0 con 0x561f2aa32d80 2024-05-04T03:13:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.012+0000 7f5bdac80740 1 -- 172.21.15.39:0/3026964261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa32d80 msgr2=0x561f2ae11230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.012+0000 7f5bdac80740 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa32d80 0x561f2ae11230 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80e2970 tx=0x7f5bc800cf90 comp rx=0 tx=0).stop 2024-05-04T03:13:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 -- 172.21.15.39:0/3026964261 shutdown_connections 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa534d0 0x561f2ae290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa4b2f0 0x561f2ae28b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 --2- 172.21.15.39:0/3026964261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa32d80 0x561f2ae11230 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 -- 172.21.15.39:0/3026964261 >> 172.21.15.39:0/3026964261 conn(0x561f2aa68300 msgr2=0x561f2ae10b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 -- 172.21.15.39:0/3026964261 shutdown_connections 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 -- 172.21.15.39:0/3026964261 wait complete. 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 -- start start 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.013+0000 7f5bdac80740 1 Processor -- start 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa32d80 0x561f2ae23d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae24270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bdac80740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 0x561f2ae93700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bdac80740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1aa550 con 0x561f2aa4b2f0 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bdac80740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1a9e80 con 0x561f2aa534d0 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bdac80740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1a70c0 con 0x561f2aa32d80 2024-05-04T03:13:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bdac80740 10 client.0 _ll_register_callbacks cb 0x7f5bc80a93f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae24270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae24270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33862/0 (socket says 172.21.15.39:33862) 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 -- 172.21.15.39:0/798167753 learned_addr learned my addr 172.21.15.39:0/798167753 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd236b640 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 0x561f2ae93700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa32d80 0x561f2ae23d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 -- 172.21.15.39:0/798167753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa32d80 msgr2=0x561f2ae23d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa32d80 0x561f2ae23d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 -- 172.21.15.39:0/798167753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 msgr2=0x561f2ae93700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 0x561f2ae93700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 -- 172.21.15.39:0/798167753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2ae29240 con 0x561f2aa4b2f0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bd1369640 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae24270 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc02a930 tx=0x7f5bbc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4cf01fad1e3858 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.014+0000 7f5bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bb57fa640 1 -- 172.21.15.39:0/798167753 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0522d0 con 0x561f2aa4b2f0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bb57fa640 1 -- 172.21.15.39:0/798167753 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5bbc043040 con 0x561f2aa4b2f0 2024-05-04T03:13:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bdac80740 1 -- 172.21.15.39:0/798167753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc80a9c80 con 0x561f2aa4b2f0 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bb57fa640 1 -- 172.21.15.39:0/798167753 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5bbc052470 con 0x561f2aa4b2f0 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bdac80740 10 client.15696 fetch_fsmap learned FSMap version 31 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bdac80740 1 -- 172.21.15.39:0/798167753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f2ae94c00 con 0x561f2aa4b2f0 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bb57fa640 1 -- 172.21.15.39:0/798167753 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bc80a9c80 con 0x561f2aa4b2f0 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bb57fa640 1 -- 172.21.15.39:0/798167753 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7f5bbc003ad0 con 0x561f2aa4b2f0 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bdac80740 10 client.15696 fetch_fsmap finished waiting for FSMap version 31 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bdac80740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.015+0000 7f5bdac80740 20 client.15696 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.016+0000 7f5bdac80740 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561f2aa741b0 0x561f2ae97600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.016+0000 7f5bdac80740 4 client.15696 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.016+0000 7f5bdac80740 1 -- 172.21.15.39:0/798167753 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f2ad82050 con 0x561f2aa741b0 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.016+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561f2aa741b0 0x561f2ae97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.016+0000 7f5bd1b6a640 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561f2aa741b0 0x561f2ae97600 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc248b6c35ebf31b server_cookie=8cac45ed51971afe in_seq=0 out_seq=0 2024-05-04T03:13:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.016+0000 7f5bb57fa640 10 client.15696 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bb57fa640 1 -- 172.21.15.39:0/798167753 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f2ad82050 con 0x561f2aa741b0 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bb57fa640 10 client.15696 handle_command_reply: tid=0 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 client.15696 shutdown 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 -- 172.21.15.39:0/798167753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 msgr2=0x561f2ae24270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae24270 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc02a930 tx=0x7f5bbc009260 comp rx=0 tx=0).stop 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 -- 172.21.15.39:0/798167753 shutdown_connections 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561f2aa741b0 0x561f2ae97600 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f2aa534d0 0x561f2ae93700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f2aa4b2f0 0x561f2ae24270 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 --2- 172.21.15.39:0/798167753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f2aa32d80 0x561f2ae23d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 -- 172.21.15.39:0/798167753 >> 172.21.15.39:0/798167753 conn(0x561f2aa68300 msgr2=0x561f2ae21ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bb57fa640 0 client.15696 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 -- 172.21.15.39:0/798167753 shutdown_connections 2024-05-04T03:13:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.024+0000 7f5bdac80740 1 -- 172.21.15.39:0/798167753 wait complete. 2024-05-04T03:13:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bdac80740 20 client.15696 trim_cache size 0 max 16384 2024-05-04T03:13:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:13:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:13:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:13:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 -- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5bb80b2860 msgr2=0x7f5bb80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5bb80b2860 0x7f5bb80b4d70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc006010 tx=0x7f5bbc031420 comp rx=0 tx=0).stop 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 -- 172.21.15.39:0/1333690699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bcc102b60 msgr2=0x7f5bcc199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bcc102b60 0x7f5bcc199170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc102430 tx=0x7f5bc800ee50 comp rx=0 tx=0).stop 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 -- 172.21.15.39:0/1333690699 shutdown_connections 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f5bb80b2860 0x7f5bb80b4d70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bcc108aa0 0x7f5bcc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bcc103da0 0x7f5bcc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 --2- 172.21.15.39:0/1333690699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bcc102b60 0x7f5bcc199170 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 -- 172.21.15.39:0/1333690699 >> 172.21.15.39:0/1333690699 conn(0x7f5bcc0fdd50 msgr2=0x7f5bcc104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 -- 172.21.15.39:0/1333690699 shutdown_connections 2024-05-04T03:13:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:45.025+0000 7f5bd3dce640 1 -- 172.21.15.39:0/1333690699 wait complete. 2024-05-04T03:13:46.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:46 smithi167 ceph-mon[33614]: pgmap v475: 97 pgs: 97 active+clean; 784 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 MiB/s wr, 32 op/s 2024-05-04T03:13:46.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:46 smithi039 ceph-mon[29715]: pgmap v475: 97 pgs: 97 active+clean; 784 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 MiB/s wr, 32 op/s 2024-05-04T03:13:46.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:46 smithi066 ceph-mon[33264]: pgmap v475: 97 pgs: 97 active+clean; 784 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 43 MiB/s wr, 32 op/s 2024-05-04T03:13:48.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:48 smithi167 ceph-mon[33614]: pgmap v476: 97 pgs: 97 active+clean; 805 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 MiB/s wr, 36 op/s 2024-05-04T03:13:48.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:48 smithi039 ceph-mon[29715]: pgmap v476: 97 pgs: 97 active+clean; 805 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 MiB/s wr, 36 op/s 2024-05-04T03:13:48.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:48 smithi066 ceph-mon[33264]: pgmap v476: 97 pgs: 97 active+clean; 805 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 MiB/s wr, 36 op/s 2024-05-04T03:13:49.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:49 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:49.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:49 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:49.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:49 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:50.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:50 smithi167 ceph-mon[33614]: pgmap v477: 97 pgs: 97 active+clean; 859 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 MiB/s wr, 42 op/s 2024-05-04T03:13:50.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:50 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:50.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:50 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:50.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:50 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:13:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:50 smithi039 ceph-mon[29715]: pgmap v477: 97 pgs: 97 active+clean; 859 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 MiB/s wr, 42 op/s 2024-05-04T03:13:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:50 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:50 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:50 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:13:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:50 smithi066 ceph-mon[33264]: pgmap v477: 97 pgs: 97 active+clean; 859 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 MiB/s wr, 42 op/s 2024-05-04T03:13:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:50 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:50 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:50 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:13:51.630 INFO:teuthology.orchestra.run.smithi039.stderr:done in 121.70 s (drop tables 0.01 s, create tables 0.06 s, client-side generate 106.41 s, vacuum 8.39 s, primary keys 6.83 s). 2024-05-04T03:13:51.634 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-04T03:13:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:starting vacuum...end. 2024-05-04T03:13:52.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:52 smithi167 ceph-mon[33614]: pgmap v478: 97 pgs: 97 active+clean; 894 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 46 MiB/s wr, 37 op/s 2024-05-04T03:13:52.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:52 smithi039 ceph-mon[29715]: pgmap v478: 97 pgs: 97 active+clean; 894 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 46 MiB/s wr, 37 op/s 2024-05-04T03:13:52.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:52 smithi066 ceph-mon[33264]: pgmap v478: 97 pgs: 97 active+clean; 894 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 46 MiB/s wr, 37 op/s 2024-05-04T03:13:54.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:54 smithi167 ceph-mon[33614]: pgmap v479: 97 pgs: 97 active+clean; 946 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 MiB/s wr, 41 op/s 2024-05-04T03:13:54.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:54 smithi039 ceph-mon[29715]: pgmap v479: 97 pgs: 97 active+clean; 946 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 MiB/s wr, 41 op/s 2024-05-04T03:13:54.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:54 smithi066 ceph-mon[33264]: pgmap v479: 97 pgs: 97 active+clean; 946 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 MiB/s wr, 41 op/s 2024-05-04T03:13:56.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:56 smithi039 ceph-mon[29715]: pgmap v480: 97 pgs: 97 active+clean; 951 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 MiB/s wr, 38 op/s 2024-05-04T03:13:56.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:56 smithi066 ceph-mon[33264]: pgmap v480: 97 pgs: 97 active+clean; 951 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 MiB/s wr, 38 op/s 2024-05-04T03:13:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:56 smithi167 ceph-mon[33614]: pgmap v480: 97 pgs: 97 active+clean; 951 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 MiB/s wr, 38 op/s 2024-05-04T03:13:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 5.0 s, 1665.0 tps, lat 56.065 ms stddev 75.137 2024-05-04T03:13:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 -- 172.21.15.39:0/769820305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0f01088b0 msgr2=0x7fe0f010acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 --2- 172.21.15.39:0/769820305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0f01088b0 0x7fe0f010acd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009ca0 tx=0x7fe0e802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 -- 172.21.15.39:0/769820305 shutdown_connections 2024-05-04T03:13:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 --2- 172.21.15.39:0/769820305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0f01088b0 0x7fe0f010acd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 --2- 172.21.15.39:0/769820305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0f0103bb0 0x7fe0f0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 --2- 172.21.15.39:0/769820305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0f0102970 0x7fe0f0103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 -- 172.21.15.39:0/769820305 >> 172.21.15.39:0/769820305 conn(0x7fe0f00fdb60 msgr2=0x7fe0f0100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 -- 172.21.15.39:0/769820305 shutdown_connections 2024-05-04T03:13:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.176+0000 7fe0f6181640 1 -- 172.21.15.39:0/769820305 wait complete. 2024-05-04T03:13:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f6181640 1 Processor -- start 2024-05-04T03:13:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f6181640 1 -- start start 2024-05-04T03:13:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f6181640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0f0102970 0x7fe0f019c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f6181640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0f0103bb0 0x7fe0f019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f6181640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0f01088b0 0x7fe0f01a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f6181640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0111690 con 0x7fe0f0103bb0 2024-05-04T03:13:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f6181640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0111510 con 0x7fe0f01088b0 2024-05-04T03:13:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f6181640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0111810 con 0x7fe0f0102970 2024-05-04T03:13:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0f0103bb0 0x7fe0f019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f5980640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0f01088b0 0x7fe0f01a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0f0103bb0 0x7fe0f019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41384/0 (socket says 172.21.15.39:41384) 2024-05-04T03:13:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f497e640 1 -- 172.21.15.39:0/3736411516 learned_addr learned my addr 172.21.15.39:0/3736411516 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.177+0000 7fe0f517f640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0f0102970 0x7fe0f019c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.178+0000 7fe0f497e640 1 -- 172.21.15.39:0/3736411516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0f0102970 msgr2=0x7fe0f019c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.178+0000 7fe0f497e640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0f0102970 0x7fe0f019c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.178+0000 7fe0f497e640 1 -- 172.21.15.39:0/3736411516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0f01088b0 msgr2=0x7fe0f01a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.178+0000 7fe0f497e640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0f01088b0 0x7fe0f01a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.178+0000 7fe0f497e640 1 -- 172.21.15.39:0/3736411516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01a3d80 con 0x7fe0f0103bb0 2024-05-04T03:13:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.178+0000 7fe0f497e640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0f0103bb0 0x7fe0f019c780 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000ea60 tx=0x7fe0e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.178+0000 7fe0e67fc640 1 -- 172.21.15.39:0/3736411516 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000cde0 con 0x7fe0f0103bb0 2024-05-04T03:13:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.179+0000 7fe0e67fc640 1 -- 172.21.15.39:0/3736411516 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0e00035f0 con 0x7fe0f0103bb0 2024-05-04T03:13:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.179+0000 7fe0e67fc640 1 -- 172.21.15.39:0/3736411516 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0010690 con 0x7fe0f0103bb0 2024-05-04T03:13:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.179+0000 7fe0f6181640 1 -- 172.21.15.39:0/3736411516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01a4070 con 0x7fe0f0103bb0 2024-05-04T03:13:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.179+0000 7fe0f6181640 1 -- 172.21.15.39:0/3736411516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01a4510 con 0x7fe0f0103bb0 2024-05-04T03:13:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.183+0000 7fe0e67fc640 1 -- 172.21.15.39:0/3736411516 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe0e0010830 con 0x7fe0f0103bb0 2024-05-04T03:13:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.183+0000 7fe0fe36c740 1 Processor -- start 2024-05-04T03:13:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0fe36c740 1 -- start start 2024-05-04T03:13:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae45d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae45d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae4648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0fe36c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5563ae3b95d0 con 0x5563ae0924d0 2024-05-04T03:13:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0fe36c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5563ae372890 con 0x5563ae0b31b0 2024-05-04T03:13:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0fe36c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5563ae3b5590 con 0x5563ae08a2f0 2024-05-04T03:13:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0f5980640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae45d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0f5980640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae45d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:60824/0 (socket says 172.21.15.39:60824) 2024-05-04T03:13:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0f497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae4648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0f5980640 1 -- 172.21.15.39:0/2013314972 learned_addr learned my addr 172.21.15.39:0/2013314972 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.184+0000 7fe0f517f640 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae45d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0f517f640 1 -- 172.21.15.39:0/2013314972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 msgr2=0x5563ae45d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0f517f640 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae45d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0f517f640 1 -- 172.21.15.39:0/2013314972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 msgr2=0x5563ae4648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0f517f640 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae4648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0f517f640 1 -- 172.21.15.39:0/2013314972 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ae0cb210 con 0x5563ae08a2f0 2024-05-04T03:13:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0f497e640 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae4648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:13:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0e67fc640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0dc0b2930 0x7fe0dc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0f517f640 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae45d9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0102240 tx=0x7fe0ec005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36cc163d4660a1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.185+0000 7fe0f5980640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0dc0b2930 0x7fe0dc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.186+0000 7fe0e67fc640 1 -- 172.21.15.39:0/3736411516 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe0e00e23b0 con 0x7fe0f0103bb0 2024-05-04T03:13:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.186+0000 7fe0f5980640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0dc0b2930 0x7fe0dc0b4e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80094e0 tx=0x7fe0e8031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.187+0000 7fe0db7fe640 1 -- 172.21.15.39:0/2013314972 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec009d90 con 0x5563ae08a2f0 2024-05-04T03:13:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.187+0000 7fe0db7fe640 1 -- 172.21.15.39:0/2013314972 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0ec0035f0 con 0x5563ae08a2f0 2024-05-04T03:13:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.187+0000 7fe0db7fe640 1 -- 172.21.15.39:0/2013314972 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0044c0 con 0x5563ae08a2f0 2024-05-04T03:13:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 -- 172.21.15.39:0/2013314972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 msgr2=0x5563ae45d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae45d9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0102240 tx=0x7fe0ec005f90 comp rx=0 tx=0).stop 2024-05-04T03:13:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 -- 172.21.15.39:0/2013314972 shutdown_connections 2024-05-04T03:13:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae4648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae45d9c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/2013314972 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae45d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 -- 172.21.15.39:0/2013314972 >> 172.21.15.39:0/2013314972 conn(0x5563ae0a7300 msgr2=0x5563ae458dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 -- 172.21.15.39:0/2013314972 shutdown_connections 2024-05-04T03:13:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.188+0000 7fe0fe36c740 1 -- 172.21.15.39:0/2013314972 wait complete. 2024-05-04T03:13:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.189+0000 7fe0fe36c740 1 -- start start 2024-05-04T03:13:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.189+0000 7fe0fe36c740 1 Processor -- start 2024-05-04T03:13:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae08a2f0 0x5563ae4670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 0x5563ae4675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0f517f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae08a2f0 0x5563ae4670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0f517f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae08a2f0 0x5563ae4670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41410/0 (socket says 172.21.15.39:41410) 2024-05-04T03:13:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0f517f640 1 -- 172.21.15.39:0/3932747656 learned_addr learned my addr 172.21.15.39:0/3932747656 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae46e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5563ae465310 con 0x5563ae08a2f0 2024-05-04T03:13:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5563ae376370 con 0x5563ae0b31b0 2024-05-04T03:13:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.190+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5563ae465490 con 0x5563ae0924d0 2024-05-04T03:13:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0f5980640 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae46e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0f497e640 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 0x5563ae4675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0fe36c740 10 client.0 _ll_register_callbacks cb 0x5563ae0caa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0f5980640 1 -- 172.21.15.39:0/3932747656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 msgr2=0x5563ae4675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0f5980640 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 0x5563ae4675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0f5980640 1 -- 172.21.15.39:0/3932747656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae08a2f0 msgr2=0x5563ae4670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0f5980640 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae08a2f0 0x5563ae4670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0f5980640 1 -- 172.21.15.39:0/3932747656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ae197620 con 0x5563ae0b31b0 2024-05-04T03:13:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0f517f640 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae08a2f0 0x5563ae4670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:13:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.191+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ec0035f0 con 0x5563ae0b31b0 2024-05-04T03:13:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.192+0000 7fe0f5980640 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae46e550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8031920 tx=0x7fe0e802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42d15ab8721bbab server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.192+0000 7fe0d8ff9640 10 client.25355 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:13:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.192+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/3932747656 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8004040 con 0x5563ae0b31b0 2024-05-04T03:13:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.192+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/3932747656 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe0e80041e0 con 0x5563ae0b31b0 2024-05-04T03:13:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.192+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/3932747656 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0e8004440 con 0x5563ae0b31b0 2024-05-04T03:13:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.192+0000 7fe0fe36c740 10 client.25355 fetch_fsmap learned FSMap version 31 2024-05-04T03:13:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.192+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=31}) -- 0x5563ae471410 con 0x5563ae0b31b0 2024-05-04T03:13:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.193+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/3932747656 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7fe0e8051070 con 0x5563ae0b31b0 2024-05-04T03:13:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.193+0000 7fe0fe36c740 10 client.25355 fetch_fsmap finished waiting for FSMap version 31 2024-05-04T03:13:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.193+0000 7fe0fe36c740 10 client.25355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.193+0000 7fe0fe36c740 20 client.25355 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.193+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5563ae071d80 0x5563ae474a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.193+0000 7fe0fe36c740 4 client.25355 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:13:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.193+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563ae3c71e0 con 0x5563ae071d80 2024-05-04T03:13:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.193+0000 7fe0f517f640 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5563ae071d80 0x5563ae474a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.194+0000 7fe0f517f640 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5563ae071d80 0x5563ae474a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d92de5ab470c2b server_cookie=469745fe508c846d in_seq=0 out_seq=0 2024-05-04T03:13:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.194+0000 7fe0d8ff9640 10 client.25355 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.194+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/3932747656 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5563ae3c71e0 con 0x5563ae071d80 2024-05-04T03:13:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.194+0000 7fe0d8ff9640 10 client.25355 handle_command_reply: tid=0 2024-05-04T03:13:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 client.25355 shutdown 2024-05-04T03:13:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 msgr2=0x5563ae46e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae46e550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8031920 tx=0x7fe0e802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 shutdown_connections 2024-05-04T03:13:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5563ae071d80 0x5563ae474a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae0b31b0 0x5563ae46e550 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0d8ff9640 0 client.25355 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 0x5563ae4675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3932747656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae08a2f0 0x5563ae4670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.195+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 >> 172.21.15.39:0/3932747656 conn(0x5563ae0a7300 msgr2=0x5563ae45bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.196+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 shutdown_connections 2024-05-04T03:13:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.196+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3932747656 wait complete. 2024-05-04T03:13:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.196+0000 7fe0fe36c740 20 client.25355 trim_cache size 0 max 16384 2024-05-04T03:13:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.224+0000 7fe0fe36c740 1 Processor -- start 2024-05-04T03:13:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.224+0000 7fe0fe36c740 1 -- start start 2024-05-04T03:13:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.224+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae071d80 0x5563ae458d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae46e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae45bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0fe36c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f01a8e70 con 0x5563ae0924d0 2024-05-04T03:13:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0fe36c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f01aa9e0 con 0x5563ae071d80 2024-05-04T03:13:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0fe36c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f01a9540 con 0x5563ae08a2f0 2024-05-04T03:13:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f517f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae071d80 0x5563ae458d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f517f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae071d80 0x5563ae458d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:60840/0 (socket says 172.21.15.39:60840) 2024-05-04T03:13:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f517f640 1 -- 172.21.15.39:0/3080360101 learned_addr learned my addr 172.21.15.39:0/3080360101 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f5980640 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae45bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f497e640 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae46e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f517f640 1 -- 172.21.15.39:0/3080360101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 msgr2=0x5563ae46e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f517f640 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae46e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f517f640 1 -- 172.21.15.39:0/3080360101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 msgr2=0x5563ae45bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f517f640 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae45bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.225+0000 7fe0f517f640 1 -- 172.21.15.39:0/3080360101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ae4648e0 con 0x5563ae071d80 2024-05-04T03:13:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.226+0000 7fe0f517f640 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae071d80 0x5563ae458d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec003db0 tx=0x7fe0ec005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b673bc4dfa04a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.226+0000 7fe0daffd640 1 -- 172.21.15.39:0/3080360101 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec032070 con 0x5563ae071d80 2024-05-04T03:13:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.226+0000 7fe0daffd640 1 -- 172.21.15.39:0/3080360101 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0ec003b40 con 0x5563ae071d80 2024-05-04T03:13:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.226+0000 7fe0daffd640 1 -- 172.21.15.39:0/3080360101 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec023ab0 con 0x5563ae071d80 2024-05-04T03:13:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.227+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3080360101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae071d80 msgr2=0x5563ae458d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.227+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae071d80 0x5563ae458d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec003db0 tx=0x7fe0ec005d30 comp rx=0 tx=0).stop 2024-05-04T03:13:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.227+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3080360101 shutdown_connections 2024-05-04T03:13:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.227+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae0924d0 0x5563ae45bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.227+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae08a2f0 0x5563ae46e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.227+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/3080360101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae071d80 0x5563ae458d70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.227+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3080360101 >> 172.21.15.39:0/3080360101 conn(0x5563ae0a7300 msgr2=0x5563ae458660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.228+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3080360101 shutdown_connections 2024-05-04T03:13:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.228+0000 7fe0fe36c740 1 -- 172.21.15.39:0/3080360101 wait complete. 2024-05-04T03:13:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.228+0000 7fe0fe36c740 1 -- start start 2024-05-04T03:13:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.229+0000 7fe0fe36c740 1 Processor -- start 2024-05-04T03:13:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.229+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae071d80 0x5563ae46ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.229+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae08a2f0 0x5563ae46bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0fe36c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 0x5563ae46ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f517f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae071d80 0x5563ae46ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f517f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae071d80 0x5563ae46ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41426/0 (socket says 172.21.15.39:41426) 2024-05-04T03:13:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0fe36c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f01aa2e0 con 0x5563ae071d80 2024-05-04T03:13:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f517f640 1 -- 172.21.15.39:0/555661958 learned_addr learned my addr 172.21.15.39:0/555661958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:13:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f01a9c10 con 0x5563ae08a2f0 2024-05-04T03:13:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f01a6e50 con 0x5563ae0924d0 2024-05-04T03:13:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0fe36c740 10 client.0 _ll_register_callbacks cb 0x7fe0e8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:13:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f497e640 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae08a2f0 0x5563ae46bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f5980640 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 0x5563ae46ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f517f640 1 -- 172.21.15.39:0/555661958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 msgr2=0x5563ae46ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f517f640 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 0x5563ae46ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f517f640 1 -- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae08a2f0 msgr2=0x5563ae46bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:13:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f517f640 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae08a2f0 0x5563ae46bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f517f640 1 -- 172.21.15.39:0/555661958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ae46d440 con 0x5563ae071d80 2024-05-04T03:13:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.230+0000 7fe0f497e640 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae08a2f0 0x5563ae46bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:13:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0f517f640 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae071d80 0x5563ae46ba10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x5563ae45e4c0 tx=0x7fe0ec005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db25564673f47528 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:13:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/555661958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec03a020 con 0x5563ae071d80 2024-05-04T03:13:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/555661958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe0ec0050d0 con 0x5563ae071d80 2024-05-04T03:13:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ec003b40 con 0x5563ae071d80 2024-05-04T03:13:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/555661958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0ec032040 con 0x5563ae071d80 2024-05-04T03:13:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0fe36c740 10 client.15714 fetch_fsmap learned FSMap version 31 2024-05-04T03:13:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5563ae46e6b0 con 0x5563ae071d80 2024-05-04T03:13:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.231+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/555661958 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0ec003b40 con 0x5563ae071d80 2024-05-04T03:13:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.232+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/555661958 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 31) ==== 8358+0+0 (secure 0 0 0) 0x7fe0ec035030 con 0x5563ae071d80 2024-05-04T03:13:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.232+0000 7fe0fe36c740 10 client.15714 fetch_fsmap finished waiting for FSMap version 31 2024-05-04T03:13:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.232+0000 7fe0fe36c740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:13:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.232+0000 7fe0fe36c740 20 client.15714 populate_metadata read hostname 'smithi039' 2024-05-04T03:13:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.232+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5563ae0b31b0 0x5563ae4df260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.232+0000 7fe0fe36c740 4 client.15714 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:13:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.232+0000 7fe0f497e640 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5563ae0b31b0 0x5563ae4df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.233+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563ae3c1050 con 0x5563ae0b31b0 2024-05-04T03:13:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.233+0000 7fe0d8ff9640 10 client.15714 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.234+0000 7fe0f497e640 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5563ae0b31b0 0x5563ae4df260 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec42f4ca8ed3631 server_cookie=38991617b28f684b in_seq=0 out_seq=0 2024-05-04T03:13:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0d8ff9640 1 -- 172.21.15.39:0/555661958 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563ae3c1050 con 0x5563ae0b31b0 2024-05-04T03:13:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0d8ff9640 10 client.15714 handle_command_reply: tid=0 2024-05-04T03:13:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 client.15714 shutdown 2024-05-04T03:13:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae071d80 msgr2=0x5563ae46ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae071d80 0x5563ae46ba10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x5563ae45e4c0 tx=0x7fe0ec005f10 comp rx=0 tx=0).stop 2024-05-04T03:13:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 shutdown_connections 2024-05-04T03:13:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5563ae0b31b0 0x5563ae4df260 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0d8ff9640 0 client.15714 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:13:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5563ae0924d0 0x5563ae46ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5563ae08a2f0 0x5563ae46bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 --2- 172.21.15.39:0/555661958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563ae071d80 0x5563ae46ba10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 >> 172.21.15.39:0/555661958 conn(0x5563ae0a7300 msgr2=0x5563ae45bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.242+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 shutdown_connections 2024-05-04T03:13:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.243+0000 7fe0fe36c740 1 -- 172.21.15.39:0/555661958 wait complete. 2024-05-04T03:13:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.243+0000 7fe0fe36c740 20 client.15714 trim_cache size 0 max 16384 2024-05-04T03:13:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:13:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:13:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:13:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:13:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 -- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0dc0b2930 msgr2=0x7fe0dc0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0dc0b2930 0x7fe0dc0b4e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80094e0 tx=0x7fe0e8031000 comp rx=0 tx=0).stop 2024-05-04T03:13:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 -- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0f0103bb0 msgr2=0x7fe0f019c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0f0103bb0 0x7fe0f019c780 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000ea60 tx=0x7fe0e000ef20 comp rx=0 tx=0).stop 2024-05-04T03:13:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 -- 172.21.15.39:0/3736411516 shutdown_connections 2024-05-04T03:13:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe0dc0b2930 0x7fe0dc0b4e40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe0f01088b0 0x7fe0f01a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0f0103bb0 0x7fe0f019c780 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 --2- 172.21.15.39:0/3736411516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0f0102970 0x7fe0f019c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.244+0000 7fe0f6181640 1 -- 172.21.15.39:0/3736411516 >> 172.21.15.39:0/3736411516 conn(0x7fe0f00fdb60 msgr2=0x7fe0f0104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.245+0000 7fe0f6181640 1 -- 172.21.15.39:0/3736411516 shutdown_connections 2024-05-04T03:13:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:13:58.245+0000 7fe0f6181640 1 -- 172.21.15.39:0/3736411516 wait complete. 2024-05-04T03:13:58.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:13:58 smithi039 ceph-mon[29715]: pgmap v481: 97 pgs: 97 active+clean; 945 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 73 op/s 2024-05-04T03:13:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:13:58 smithi066 ceph-mon[33264]: pgmap v481: 97 pgs: 97 active+clean; 945 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 73 op/s 2024-05-04T03:13:58.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:13:58 smithi167 ceph-mon[33614]: pgmap v481: 97 pgs: 97 active+clean; 945 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 73 op/s 2024-05-04T03:14:00.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:00 smithi039 ceph-mon[29715]: pgmap v482: 97 pgs: 97 active+clean; 938 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 32 MiB/s wr, 99 op/s 2024-05-04T03:14:00.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:00 smithi066 ceph-mon[33264]: pgmap v482: 97 pgs: 97 active+clean; 938 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 32 MiB/s wr, 99 op/s 2024-05-04T03:14:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:00 smithi167 ceph-mon[33614]: pgmap v482: 97 pgs: 97 active+clean; 938 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 32 MiB/s wr, 99 op/s 2024-05-04T03:14:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 10.0 s, 1506.2 tps, lat 66.203 ms stddev 19.395 2024-05-04T03:14:02.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:02 smithi039 ceph-mon[29715]: pgmap v483: 97 pgs: 97 active+clean; 938 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 22 MiB/s wr, 280 op/s 2024-05-04T03:14:02.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:02 smithi066 ceph-mon[33264]: pgmap v483: 97 pgs: 97 active+clean; 938 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 22 MiB/s wr, 280 op/s 2024-05-04T03:14:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:02 smithi167 ceph-mon[33614]: pgmap v483: 97 pgs: 97 active+clean; 938 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 22 MiB/s wr, 280 op/s 2024-05-04T03:14:04.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:04 smithi066 ceph-mon[33264]: pgmap v484: 97 pgs: 97 active+clean; 938 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 20 MiB/s wr, 540 op/s 2024-05-04T03:14:04.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:04 smithi167 ceph-mon[33614]: pgmap v484: 97 pgs: 97 active+clean; 938 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 20 MiB/s wr, 540 op/s 2024-05-04T03:14:04.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:04 smithi039 ceph-mon[29715]: pgmap v484: 97 pgs: 97 active+clean; 938 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 20 MiB/s wr, 540 op/s 2024-05-04T03:14:06.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:06 smithi066 ceph-mon[33264]: pgmap v485: 97 pgs: 97 active+clean; 938 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 725 op/s 2024-05-04T03:14:06.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:06 smithi167 ceph-mon[33614]: pgmap v485: 97 pgs: 97 active+clean; 938 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 725 op/s 2024-05-04T03:14:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:06 smithi039 ceph-mon[29715]: pgmap v485: 97 pgs: 97 active+clean; 938 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 725 op/s 2024-05-04T03:14:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 15.0 s, 1438.8 tps, lat 69.412 ms stddev 18.311 2024-05-04T03:14:08.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:08 smithi066 ceph-mon[33264]: pgmap v486: 97 pgs: 97 active+clean; 938 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 929 op/s 2024-05-04T03:14:08.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:08 smithi066 ceph-mon[33264]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:14:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:08 smithi167 ceph-mon[33614]: pgmap v486: 97 pgs: 97 active+clean; 938 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 929 op/s 2024-05-04T03:14:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:08 smithi167 ceph-mon[33614]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:14:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:08 smithi039 ceph-mon[29715]: pgmap v486: 97 pgs: 97 active+clean; 938 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 929 op/s 2024-05-04T03:14:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:08 smithi039 ceph-mon[29715]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-04T03:14:10.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:10 smithi066 ceph-mon[33264]: pgmap v487: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-05-04T03:14:10.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:10 smithi167 ceph-mon[33614]: pgmap v487: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-05-04T03:14:10.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:10 smithi039 ceph-mon[29715]: pgmap v487: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 13 MiB/s wr, 1.33k op/s 2024-05-04T03:14:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1232002786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 msgr2=0x7fe9e8106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1232002786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 0x7fe9e8106b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009900 tx=0x7fe9dc02a410 comp rx=0 tx=0).stop 2024-05-04T03:14:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1232002786 shutdown_connections 2024-05-04T03:14:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1232002786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9e810bdb0 0x7fe9e810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1232002786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e81070b0 0x7fe9e81074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1232002786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 0x7fe9e8106b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1232002786 >> 172.21.15.39:0/1232002786 conn(0x7fe9e8101040 msgr2=0x7fe9e8103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1232002786 shutdown_connections 2024-05-04T03:14:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.401+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1232002786 wait complete. 2024-05-04T03:14:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.402+0000 7fe9eed9d640 1 Processor -- start 2024-05-04T03:14:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.402+0000 7fe9eed9d640 1 -- start start 2024-05-04T03:14:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 0x7fe9e807ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9eed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9e81070b0 0x7fe9e807bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9eed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e810bdb0 0x7fe9e80785a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9eed9d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8111820 con 0x7fe9e81070b0 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9eed9d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81116a0 con 0x7fe9e8105e70 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9eed9d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81119a0 con 0x7fe9e810bdb0 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9ed33a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e810bdb0 0x7fe9e80785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9e81070b0 0x7fe9e807bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9ed33a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e810bdb0 0x7fe9e80785a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:41818/0 (socket says 172.21.15.39:41818) 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.403+0000 7fe9ed33a640 1 -- 172.21.15.39:0/1180037819 learned_addr learned my addr 172.21.15.39:0/1180037819 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.404+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 0x7fe9e807ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.404+0000 7fe9e7fff640 1 -- 172.21.15.39:0/1180037819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e810bdb0 msgr2=0x7fe9e80785a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.404+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e810bdb0 0x7fe9e80785a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.404+0000 7fe9e7fff640 1 -- 172.21.15.39:0/1180037819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 msgr2=0x7fe9e807ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.404+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 0x7fe9e807ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.404+0000 7fe9e7fff640 1 -- 172.21.15.39:0/1180037819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e8078dc0 con 0x7fe9e81070b0 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.404+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 0x7fe9e807ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:14:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.404+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9e81070b0 0x7fe9e807bf90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400e9f0 tx=0x7fe9d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.405+0000 7fe9e5ffb640 1 -- 172.21.15.39:0/1180037819 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400ccc0 con 0x7fe9e81070b0 2024-05-04T03:14:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.405+0000 7fe9e5ffb640 1 -- 172.21.15.39:0/1180037819 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9d4005020 con 0x7fe9e81070b0 2024-05-04T03:14:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.405+0000 7fe9e5ffb640 1 -- 172.21.15.39:0/1180037819 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400ce60 con 0x7fe9e81070b0 2024-05-04T03:14:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.405+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1180037819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e80790b0 con 0x7fe9e81070b0 2024-05-04T03:14:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.405+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1180037819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e81a8bd0 con 0x7fe9e81070b0 2024-05-04T03:14:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.409+0000 7fe9e5ffb640 1 -- 172.21.15.39:0/1180037819 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe9d4004a60 con 0x7fe9e81070b0 2024-05-04T03:14:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.409+0000 7fe9f5c55740 1 Processor -- start 2024-05-04T03:14:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9f5c55740 1 -- start start 2024-05-04T03:14:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930eb51b0 0x563931256a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e8c2f0 0x563931256fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e944d0 0x56393125dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5639311bb5d0 con 0x563930e944d0 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563931174890 con 0x563930e8c2f0 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5639311b7590 con 0x563930eb51b0 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e944d0 0x56393125dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e944d0 0x56393125dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60296/0 (socket says 172.21.15.39:60296) 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9e7fff640 1 -- 172.21.15.39:0/324749084 learned_addr learned my addr 172.21.15.39:0/324749084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9ed33a640 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930eb51b0 0x563931256a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.410+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e8c2f0 0x563931256fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9e5ffb640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe9d80b28e0 0x7fe9d80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9e7fff640 1 -- 172.21.15.39:0/324749084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930eb51b0 msgr2=0x563931256a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930eb51b0 0x563931256a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9e7fff640 1 -- 172.21.15.39:0/324749084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e8c2f0 msgr2=0x563931256fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e8c2f0 0x563931256fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9e7fff640 1 -- 172.21.15.39:0/324749084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563930ecd210 con 0x563930e944d0 2024-05-04T03:14:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e8c2f0 0x563931256fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:14:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9ed33a640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe9d80b28e0 0x7fe9d80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9e5ffb640 1 -- 172.21.15.39:0/1180037819 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe9d40e21f0 con 0x7fe9e81070b0 2024-05-04T03:14:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.411+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e944d0 0x56393125dee0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe9d408f3d0 tx=0x7fe9d406ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb92bb58ee4c1408 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.412+0000 7fe9d2ffd640 1 -- 172.21.15.39:0/324749084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d401c070 con 0x563930e944d0 2024-05-04T03:14:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.412+0000 7fe9ed33a640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe9d80b28e0 0x7fe9d80b4df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x563931257c30 tx=0x7fe9e0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.412+0000 7fe9d2ffd640 1 -- 172.21.15.39:0/324749084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9d408fd60 con 0x563930e944d0 2024-05-04T03:14:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.412+0000 7fe9d2ffd640 1 -- 172.21.15.39:0/324749084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40a8f30 con 0x563930e944d0 2024-05-04T03:14:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 -- 172.21.15.39:0/324749084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e944d0 msgr2=0x56393125dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e944d0 0x56393125dee0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe9d408f3d0 tx=0x7fe9d406ed00 comp rx=0 tx=0).stop 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 -- 172.21.15.39:0/324749084 shutdown_connections 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e944d0 0x56393125dee0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e8c2f0 0x563931256fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/324749084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930eb51b0 0x563931256a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 -- 172.21.15.39:0/324749084 >> 172.21.15.39:0/324749084 conn(0x563930ea9300 msgr2=0x5639312523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 -- 172.21.15.39:0/324749084 shutdown_connections 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.414+0000 7fe9f5c55740 1 -- 172.21.15.39:0/324749084 wait complete. 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.415+0000 7fe9f5c55740 1 -- start start 2024-05-04T03:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.415+0000 7fe9f5c55740 1 Processor -- start 2024-05-04T03:14:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x563931260660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e944d0 0x563931260ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9ecb39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x563931260660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9ecb39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x563931260660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:41840/0 (socket says 172.21.15.39:41840) 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9ecb39640 1 -- 172.21.15.39:0/3586055893 learned_addr learned my addr 172.21.15.39:0/3586055893 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e944d0 0x563931260ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930eb51b0 0x563931267b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56393125e920 con 0x563930eb51b0 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563931178370 con 0x563930e944d0 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56393125eaa0 con 0x563930e8c2f0 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.416+0000 7fe9ed33a640 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930eb51b0 0x563931267b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9f5c55740 10 client.0 _ll_register_callbacks cb 0x563930ecca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9e7fff640 1 -- 172.21.15.39:0/3586055893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 msgr2=0x563931260660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x563931260660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9e7fff640 1 -- 172.21.15.39:0/3586055893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930eb51b0 msgr2=0x563931267b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930eb51b0 0x563931267b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9e7fff640 1 -- 172.21.15.39:0/3586055893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563930f99620 con 0x563930e944d0 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e944d0 0x563931260ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400c6a0 tx=0x7fe9d4005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d750f8ceacd0c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.417+0000 7fe9c7fff640 1 -- 172.21.15.39:0/3586055893 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d401c070 con 0x563930e944d0 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.418+0000 7fe9c7fff640 1 -- 172.21.15.39:0/3586055893 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe9d406e8e0 con 0x563930e944d0 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.418+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639311c91e0 con 0x563930e944d0 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.418+0000 7fe9c7fff640 1 -- 172.21.15.39:0/3586055893 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe9d40e5ca0 con 0x563930e944d0 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.418+0000 7fe9f5c55740 10 client.25373 fetch_fsmap learned FSMap version 32 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.418+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56393126a9f0 con 0x563930e944d0 2024-05-04T03:14:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.418+0000 7fe9c7fff640 1 -- 172.21.15.39:0/3586055893 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9d40cc0d0 con 0x563930e944d0 2024-05-04T03:14:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.418+0000 7fe9c7fff640 1 -- 172.21.15.39:0/3586055893 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe9d40b2190 con 0x563930e944d0 2024-05-04T03:14:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.419+0000 7fe9f5c55740 10 client.25373 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:14:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.419+0000 7fe9f5c55740 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:14:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.419+0000 7fe9f5c55740 20 client.25373 populate_metadata read hostname 'smithi039' 2024-05-04T03:14:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.419+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563930e73d80 0x56393126dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.419+0000 7fe9f5c55740 4 client.25373 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:14:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.419+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639311c91e0 con 0x563930e73d80 2024-05-04T03:14:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.419+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563930e73d80 0x56393126dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.420+0000 7fe9c7fff640 10 client.25373 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.421+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563930e73d80 0x56393126dd50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ee83000dfd33f3 server_cookie=2b3f8f0e069433b5 in_seq=0 out_seq=0 2024-05-04T03:14:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9c7fff640 1 -- 172.21.15.39:0/3586055893 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639311c91e0 con 0x563930e73d80 2024-05-04T03:14:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9c7fff640 10 client.25373 handle_command_reply: tid=0 2024-05-04T03:14:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 client.25373 shutdown 2024-05-04T03:14:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e944d0 msgr2=0x563931260ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e944d0 0x563931260ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400c6a0 tx=0x7fe9d4005bd0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 shutdown_connections 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563930e73d80 0x56393126dd50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930eb51b0 0x563931267b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930e944d0 0x563931260ba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/3586055893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x563931260660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 >> 172.21.15.39:0/3586055893 conn(0x563930ea9300 msgr2=0x5639312513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9c7fff640 0 client.25373 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 shutdown_connections 2024-05-04T03:14:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.422+0000 7fe9f5c55740 1 -- 172.21.15.39:0/3586055893 wait complete. 2024-05-04T03:14:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.423+0000 7fe9f5c55740 20 client.25373 trim_cache size 0 max 16384 2024-05-04T03:14:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.449+0000 7fe9f5c55740 1 Processor -- start 2024-05-04T03:14:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9f5c55740 1 -- start start 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e8c2f0 0x56393124cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e944d0 0x56393125aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x56393125b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b1820 con 0x563930e8c2f0 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b3360 con 0x563930eb51b0 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b1ef0 con 0x563930e944d0 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9ecb39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e8c2f0 0x56393124cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9ed33a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x56393125b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9ed33a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x56393125b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:54534/0 (socket says 172.21.15.39:54534) 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9ed33a640 1 -- 172.21.15.39:0/274886621 learned_addr learned my addr 172.21.15.39:0/274886621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/274886621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e944d0 0x56393125aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9e7fff640 1 -- 172.21.15.39:0/274886621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 msgr2=0x56393125b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/274886621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x56393125b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.450+0000 7fe9e7fff640 1 -- 172.21.15.39:0/274886621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e8c2f0 msgr2=0x56393124cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.451+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/274886621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e8c2f0 0x56393124cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.451+0000 7fe9e7fff640 1 -- 172.21.15.39:0/274886621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56393125e810 con 0x563930e944d0 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.451+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/274886621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e944d0 0x56393125aad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40a6a30 tx=0x7fe9d400e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=386a7f98ce14ea65 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.451+0000 7fe9d27fc640 1 -- 172.21.15.39:0/274886621 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d401c030 con 0x563930e944d0 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.451+0000 7fe9d27fc640 1 -- 172.21.15.39:0/274886621 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9d40a92d0 con 0x563930e944d0 2024-05-04T03:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.451+0000 7fe9d27fc640 1 -- 172.21.15.39:0/274886621 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40a9ec0 con 0x563930e944d0 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 -- 172.21.15.39:0/274886621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e944d0 msgr2=0x56393125aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/274886621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e944d0 0x56393125aad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40a6a30 tx=0x7fe9d400e940 comp rx=0 tx=0).stop 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 -- 172.21.15.39:0/274886621 shutdown_connections 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/274886621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x56393125b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/274886621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e944d0 0x56393125aad0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40a6a30 tx=0x7fe9d400e940 comp rx=0 tx=0).stop 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/274886621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e8c2f0 0x56393124cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 -- 172.21.15.39:0/274886621 >> 172.21.15.39:0/274886621 conn(0x563930ea9300 msgr2=0x563931260630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 -- 172.21.15.39:0/274886621 shutdown_connections 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.453+0000 7fe9f5c55740 1 -- 172.21.15.39:0/274886621 wait complete. 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.454+0000 7fe9f5c55740 1 -- start start 2024-05-04T03:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.454+0000 7fe9f5c55740 1 Processor -- start 2024-05-04T03:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.454+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x5639312645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x563931264b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9f5c55740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e491c0 0x563931265a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b2c90 con 0x563930e491c0 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81b25c0 con 0x563930eb51b0 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9f5c55740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e81af800 con 0x563930e8c2f0 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ecb39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x5639312645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x563931264b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ecb39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x5639312645c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:41864/0 (socket says 172.21.15.39:41864) 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ecb39640 1 -- 172.21.15.39:0/2164448340 learned_addr learned my addr 172.21.15.39:0/2164448340 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ed33a640 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e491c0 0x563931265a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9f5c55740 10 client.0 _ll_register_callbacks cb 0x7fe9d40ea170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ecb39640 1 -- 172.21.15.39:0/2164448340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 msgr2=0x563931264b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x563931264b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ecb39640 1 -- 172.21.15.39:0/2164448340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e491c0 msgr2=0x563931265a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e491c0 0x563931265a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.455+0000 7fe9ecb39640 1 -- 172.21.15.39:0/2164448340 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563931265ff0 con 0x563930e8c2f0 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.456+0000 7fe9ecb39640 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x5639312645c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc02a450 tx=0x7fe9dc009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf3fe10b98c9074 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.456+0000 7fe9c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.456+0000 7fe9c7fff640 1 -- 172.21.15.39:0/2164448340 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc035040 con 0x563930e8c2f0 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.456+0000 7fe9c7fff640 1 -- 172.21.15.39:0/2164448340 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe9dc004040 con 0x563930e8c2f0 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.456+0000 7fe9f5c55740 1 -- 172.21.15.39:0/2164448340 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d40a92d0 con 0x563930e8c2f0 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.456+0000 7fe9c7fff640 1 -- 172.21.15.39:0/2164448340 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe9dc031e30 con 0x563930e8c2f0 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.456+0000 7fe9f5c55740 10 client.25381 fetch_fsmap learned FSMap version 32 2024-05-04T03:14:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.456+0000 7fe9f5c55740 1 -- 172.21.15.39:0/2164448340 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563931267330 con 0x563930e8c2f0 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9c7fff640 1 -- 172.21.15.39:0/2164448340 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9dc004240 con 0x563930e8c2f0 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9c7fff640 1 -- 172.21.15.39:0/2164448340 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe9dc003760 con 0x563930e8c2f0 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9f5c55740 10 client.25381 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9f5c55740 10 client.25381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9f5c55740 20 client.25381 populate_metadata read hostname 'smithi039' 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5639312d6660 0x5639312d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9f5c55740 4 client.25381 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9f5c55740 1 -- 172.21.15.39:0/2164448340 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe9d40a92d0 con 0x5639312d6660 2024-05-04T03:14:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.457+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5639312d6660 0x5639312d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.458+0000 7fe9c7fff640 10 client.25381 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.458+0000 7fe9e7fff640 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5639312d6660 0x5639312d8a80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f358575a8c4ba3f server_cookie=e1651240df92017a in_seq=0 out_seq=0 2024-05-04T03:14:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.466+0000 7fe9c7fff640 1 -- 172.21.15.39:0/2164448340 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe9d40a92d0 con 0x5639312d6660 2024-05-04T03:14:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.466+0000 7fe9c7fff640 10 client.25381 handle_command_reply: tid=0 2024-05-04T03:14:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.466+0000 7fe9f5c55740 1 client.25381 shutdown 2024-05-04T03:14:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 -- 172.21.15.39:0/2164448340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 msgr2=0x5639312645c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x5639312645c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc02a450 tx=0x7fe9dc009880 comp rx=0 tx=0).stop 2024-05-04T03:14:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 -- 172.21.15.39:0/2164448340 shutdown_connections 2024-05-04T03:14:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5639312d6660 0x5639312d8a80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563930e491c0 0x563931265a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563930eb51b0 0x563931264b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 --2- 172.21.15.39:0/2164448340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563930e8c2f0 0x5639312645c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9c7fff640 0 client.25381 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 -- 172.21.15.39:0/2164448340 >> 172.21.15.39:0/2164448340 conn(0x563930ea9300 msgr2=0x56393125aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 -- 172.21.15.39:0/2164448340 shutdown_connections 2024-05-04T03:14:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.467+0000 7fe9f5c55740 1 -- 172.21.15.39:0/2164448340 wait complete. 2024-05-04T03:14:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.468+0000 7fe9f5c55740 20 client.25381 trim_cache size 0 max 16384 2024-05-04T03:14:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:14:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:14:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:14:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:14:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe9d80b28e0 msgr2=0x7fe9d80b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe9d80b28e0 0x7fe9d80b4df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x563931257c30 tx=0x7fe9e0009500 comp rx=0 tx=0).stop 2024-05-04T03:14:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9e81070b0 msgr2=0x7fe9e807bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9e81070b0 0x7fe9e807bf90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400e9f0 tx=0x7fe9d400eeb0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1180037819 shutdown_connections 2024-05-04T03:14:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe9d80b28e0 0x7fe9d80b4df0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e810bdb0 0x7fe9e80785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9e81070b0 0x7fe9e807bf90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 --2- 172.21.15.39:0/1180037819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9e8105e70 0x7fe9e807ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.469+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1180037819 >> 172.21.15.39:0/1180037819 conn(0x7fe9e8101040 msgr2=0x7fe9e8107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.470+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1180037819 shutdown_connections 2024-05-04T03:14:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:11.470+0000 7fe9eed9d640 1 -- 172.21.15.39:0/1180037819 wait complete. 2024-05-04T03:14:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 20.0 s, 1410.4 tps, lat 70.990 ms stddev 22.476 2024-05-04T03:14:12.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:12 smithi066 ceph-mon[33264]: pgmap v488: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:14:12.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:12 smithi167 ceph-mon[33614]: pgmap v488: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:14:12.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:12 smithi039 ceph-mon[29715]: pgmap v488: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:14:14.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:14 smithi066 ceph-mon[33264]: pgmap v489: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:14 smithi167 ceph-mon[33614]: pgmap v489: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:14 smithi039 ceph-mon[29715]: pgmap v489: 97 pgs: 97 active+clean; 938 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:16.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:16 smithi039 ceph-mon[29715]: pgmap v490: 97 pgs: 97 active+clean; 942 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.62k op/s 2024-05-04T03:14:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:16 smithi066 ceph-mon[33264]: pgmap v490: 97 pgs: 97 active+clean; 942 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.62k op/s 2024-05-04T03:14:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:16 smithi167 ceph-mon[33614]: pgmap v490: 97 pgs: 97 active+clean; 942 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.62k op/s 2024-05-04T03:14:16.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 25.0 s, 1402.8 tps, lat 71.232 ms stddev 68.813 2024-05-04T03:14:18.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:18 smithi066 ceph-mon[33264]: pgmap v491: 97 pgs: 97 active+clean; 950 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-05-04T03:14:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:18 smithi167 ceph-mon[33614]: pgmap v491: 97 pgs: 97 active+clean; 950 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-05-04T03:14:18.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:18 smithi039 ceph-mon[29715]: pgmap v491: 97 pgs: 97 active+clean; 950 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 1.67k op/s 2024-05-04T03:14:20.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:20 smithi066 ceph-mon[33264]: pgmap v492: 97 pgs: 97 active+clean; 955 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 1.82k op/s 2024-05-04T03:14:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:20 smithi167 ceph-mon[33614]: pgmap v492: 97 pgs: 97 active+clean; 955 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 1.82k op/s 2024-05-04T03:14:20.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:20 smithi039 ceph-mon[29715]: pgmap v492: 97 pgs: 97 active+clean; 955 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 1.82k op/s 2024-05-04T03:14:21.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 30.0 s, 1452.0 tps, lat 68.866 ms stddev 17.281 2024-05-04T03:14:22.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:22 smithi066 ceph-mon[33264]: pgmap v493: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-05-04T03:14:22.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:22 smithi167 ceph-mon[33614]: pgmap v493: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-05-04T03:14:22.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:22 smithi039 ceph-mon[29715]: pgmap v493: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.64k op/s 2024-05-04T03:14:24.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:24 smithi167 ceph-mon[33614]: pgmap v494: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.83k op/s 2024-05-04T03:14:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:24 smithi066 ceph-mon[33264]: pgmap v494: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.83k op/s 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 -- 172.21.15.39:0/328931017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd581070d0 msgr2=0x7efd581074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 --2- 172.21.15.39:0/328931017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd581070d0 0x7efd581074e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efd48009900 tx=0x7efd4802a3f0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 -- 172.21.15.39:0/328931017 shutdown_connections 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 --2- 172.21.15.39:0/328931017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd5810bdd0 0x7efd5810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 --2- 172.21.15.39:0/328931017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd581070d0 0x7efd581074e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 --2- 172.21.15.39:0/328931017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58105e90 0x7efd58106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 -- 172.21.15.39:0/328931017 >> 172.21.15.39:0/328931017 conn(0x7efd58101060 msgr2=0x7efd58103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 -- 172.21.15.39:0/328931017 shutdown_connections 2024-05-04T03:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.631+0000 7efd5e092640 1 -- 172.21.15.39:0/328931017 wait complete. 2024-05-04T03:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.632+0000 7efd5e092640 1 Processor -- start 2024-05-04T03:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.632+0000 7efd5e092640 1 -- start start 2024-05-04T03:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd5e092640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58105e90 0x7efd5807a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd5e092640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd581070d0 0x7efd5807ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd5e092640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5810bdd0 0x7efd58077490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd5e092640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd581118a0 con 0x7efd5810bdd0 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd5e092640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd58111720 con 0x7efd58105e90 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd5e092640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd58111a20 con 0x7efd581070d0 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd56ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd581070d0 0x7efd5807ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd56ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd581070d0 0x7efd5807ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:54356/0 (socket says 172.21.15.39:54356) 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd56ffd640 1 -- 172.21.15.39:0/1077500880 learned_addr learned my addr 172.21.15.39:0/1077500880 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd57fff640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5810bdd0 0x7efd58077490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd577fe640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58105e90 0x7efd5807a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd56ffd640 1 -- 172.21.15.39:0/1077500880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58105e90 msgr2=0x7efd5807a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd56ffd640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58105e90 0x7efd5807a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.633+0000 7efd56ffd640 1 -- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5810bdd0 msgr2=0x7efd58077490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd56ffd640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5810bdd0 0x7efd58077490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd56ffd640 1 -- 172.21.15.39:0/1077500880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd58077cb0 con 0x7efd581070d0 2024-05-04T03:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd57fff640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5810bdd0 0x7efd58077490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd56ffd640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd581070d0 0x7efd5807ae80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd48031460 tx=0x7efd48003bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd54ff9640 1 -- 172.21.15.39:0/1077500880 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4803f070 con 0x7efd581070d0 2024-05-04T03:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd54ff9640 1 -- 172.21.15.39:0/1077500880 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd48031b90 con 0x7efd581070d0 2024-05-04T03:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd54ff9640 1 -- 172.21.15.39:0/1077500880 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48035030 con 0x7efd581070d0 2024-05-04T03:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd5e092640 1 -- 172.21.15.39:0/1077500880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd58077f40 con 0x7efd581070d0 2024-05-04T03:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.634+0000 7efd5e092640 1 -- 172.21.15.39:0/1077500880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd581a8b90 con 0x7efd581070d0 2024-05-04T03:14:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.638+0000 7efd6501d740 1 Processor -- start 2024-05-04T03:14:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.638+0000 7efd54ff9640 1 -- 172.21.15.39:0/1077500880 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efd48042070 con 0x7efd581070d0 2024-05-04T03:14:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd6501d740 1 -- start start 2024-05-04T03:14:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd6501d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd57fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd6501d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd57fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35138/0 (socket says 172.21.15.39:35138) 2024-05-04T03:14:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd6501d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd6501d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b1c78395d0 con 0x55b1c75331b0 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd6501d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b1c77f2890 con 0x55b1c750a2f0 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd6501d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b1c7835590 con 0x55b1c75124d0 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78d4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:34096/0 (socket says 172.21.15.39:34096) 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.639+0000 7efd57fff640 1 -- 172.21.15.39:0/2358402256 learned_addr learned my addr 172.21.15.39:0/2358402256 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd56ffd640 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd57fff640 1 -- 172.21.15.39:0/2358402256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 msgr2=0x55b1c78dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd57fff640 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd57fff640 1 -- 172.21.15.39:0/2358402256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 msgr2=0x55b1c78d4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd54ff9640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd400b28b0 0x7efd400b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd57fff640 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78d4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd57fff640 1 -- 172.21.15.39:0/2358402256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c754b210 con 0x55b1c75331b0 2024-05-04T03:14:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd56ffd640 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd577fe640 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd56ffd640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd400b28b0 0x7efd400b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.640+0000 7efd54ff9640 1 -- 172.21.15.39:0/1077500880 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efd48102aa0 con 0x7efd581070d0 2024-05-04T03:14:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.641+0000 7efd57fff640 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78d4a60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efd50011790 tx=0x7efd50011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e007fa20d757d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.641+0000 7efd56ffd640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd400b28b0 0x7efd400b4dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efd48005ab0 tx=0x7efd48106580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.641+0000 7efd3dffb640 1 -- 172.21.15.39:0/2358402256 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50004d20 con 0x55b1c75331b0 2024-05-04T03:14:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.641+0000 7efd3dffb640 1 -- 172.21.15.39:0/2358402256 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd50003970 con 0x55b1c75331b0 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.642+0000 7efd3dffb640 1 -- 172.21.15.39:0/2358402256 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50004400 con 0x55b1c75331b0 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.642+0000 7efd6501d740 1 -- 172.21.15.39:0/2358402256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 msgr2=0x55b1c78d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.642+0000 7efd6501d740 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78d4a60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efd50011790 tx=0x7efd50011c50 comp rx=0 tx=0).stop 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.643+0000 7efd6501d740 1 -- 172.21.15.39:0/2358402256 shutdown_connections 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.643+0000 7efd6501d740 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.643+0000 7efd6501d740 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.643+0000 7efd6501d740 1 --2- 172.21.15.39:0/2358402256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78d4a60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.643+0000 7efd6501d740 1 -- 172.21.15.39:0/2358402256 >> 172.21.15.39:0/2358402256 conn(0x55b1c7527300 msgr2=0x55b1c78d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.643+0000 7efd6501d740 1 -- 172.21.15.39:0/2358402256 shutdown_connections 2024-05-04T03:14:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.643+0000 7efd6501d740 1 -- 172.21.15.39:0/2358402256 wait complete. 2024-05-04T03:14:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.644+0000 7efd6501d740 1 -- start start 2024-05-04T03:14:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.644+0000 7efd6501d740 1 Processor -- start 2024-05-04T03:14:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd6501d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78de750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd6501d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd6501d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd57fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78de750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd57fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78de750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:34108/0 (socket says 172.21.15.39:34108) 2024-05-04T03:14:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd57fff640 1 -- 172.21.15.39:0/378587208 learned_addr learned my addr 172.21.15.39:0/378587208 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b1c78dc980 con 0x55b1c75331b0 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b1c77f6370 con 0x55b1c750a2f0 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b1c78dcb00 con 0x55b1c75124d0 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd577fe640 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd57fff640 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.645+0000 7efd6501d740 10 client.0 _ll_register_callbacks cb 0x55b1c754aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.646+0000 7efd577fe640 1 -- 172.21.15.39:0/378587208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 msgr2=0x55b1c78de750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.646+0000 7efd577fe640 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78de750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.646+0000 7efd577fe640 1 -- 172.21.15.39:0/378587208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 msgr2=0x55b1c78e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.646+0000 7efd577fe640 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.646+0000 7efd577fe640 1 -- 172.21.15.39:0/378587208 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c7617620 con 0x55b1c75124d0 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.646+0000 7efd577fe640 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dec90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55b1c78d5e40 tx=0x7efd4c012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5546fedffdbe9f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.646+0000 7efd36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.646+0000 7efd36ffd640 1 -- 172.21.15.39:0/378587208 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c01f060 con 0x55b1c75124d0 2024-05-04T03:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.647+0000 7efd36ffd640 1 -- 172.21.15.39:0/378587208 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efd4c00f8a0 con 0x55b1c75124d0 2024-05-04T03:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.647+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd50003970 con 0x55b1c75124d0 2024-05-04T03:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.647+0000 7efd36ffd640 1 -- 172.21.15.39:0/378587208 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efd4c00faa0 con 0x55b1c75124d0 2024-05-04T03:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.647+0000 7efd6501d740 10 client.25393 fetch_fsmap learned FSMap version 32 2024-05-04T03:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.647+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b1c78e8b90 con 0x55b1c75124d0 2024-05-04T03:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.647+0000 7efd36ffd640 1 -- 172.21.15.39:0/378587208 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd50003970 con 0x55b1c75124d0 2024-05-04T03:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.648+0000 7efd36ffd640 1 -- 172.21.15.39:0/378587208 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7efd4c019030 con 0x55b1c75124d0 2024-05-04T03:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.648+0000 7efd6501d740 10 client.25393 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.648+0000 7efd6501d740 10 client.25393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.648+0000 7efd6501d740 20 client.25393 populate_metadata read hostname 'smithi039' 2024-05-04T03:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.648+0000 7efd6501d740 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b1c74f1d80 0x55b1c78ec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.648+0000 7efd6501d740 4 client.25393 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.648+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1c78471e0 con 0x55b1c74f1d80 2024-05-04T03:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.648+0000 7efd57fff640 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b1c74f1d80 0x55b1c78ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.649+0000 7efd57fff640 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b1c74f1d80 0x55b1c78ec300 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18bae090c6ef77ef server_cookie=b14cc1e4876abc9b in_seq=0 out_seq=0 2024-05-04T03:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.649+0000 7efd36ffd640 10 client.25393 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.650+0000 7efd36ffd640 1 -- 172.21.15.39:0/378587208 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1c78471e0 con 0x55b1c74f1d80 2024-05-04T03:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.650+0000 7efd36ffd640 10 client.25393 handle_command_reply: tid=0 2024-05-04T03:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.650+0000 7efd6501d740 1 client.25393 shutdown 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.650+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 msgr2=0x55b1c78dec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.650+0000 7efd6501d740 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dec90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55b1c78d5e40 tx=0x7efd4c012820 comp rx=0 tx=0).stop 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 shutdown_connections 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b1c74f1d80 0x55b1c78ec300 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75124d0 0x55b1c78dec90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 1 --2- 172.21.15.39:0/378587208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c750a2f0 0x55b1c78de750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 >> 172.21.15.39:0/378587208 conn(0x55b1c7527300 msgr2=0x55b1c78cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd36ffd640 0 client.25393 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 shutdown_connections 2024-05-04T03:14:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 1 -- 172.21.15.39:0/378587208 wait complete. 2024-05-04T03:14:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.651+0000 7efd6501d740 20 client.25393 trim_cache size 0 max 16384 2024-05-04T03:14:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.679+0000 7efd6501d740 1 Processor -- start 2024-05-04T03:14:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.679+0000 7efd6501d740 1 -- start start 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.680+0000 7efd6501d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c750a2f0 0x55b1c78d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.680+0000 7efd6501d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.680+0000 7efd6501d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.680+0000 7efd577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.680+0000 7efd577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:34118/0 (socket says 172.21.15.39:34118) 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.680+0000 7efd577fe640 1 -- 172.21.15.39:0/3701627961 learned_addr learned my addr 172.21.15.39:0/3701627961 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.680+0000 7efd57fff640 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c750a2f0 0x55b1c78d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd6501d740 1 -- 172.21.15.39:0/3701627961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd581b1950 con 0x55b1c75331b0 2024-05-04T03:14:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd6501d740 1 -- 172.21.15.39:0/3701627961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd581b3490 con 0x55b1c75124d0 2024-05-04T03:14:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd6501d740 1 -- 172.21.15.39:0/3701627961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd581b2020 con 0x55b1c750a2f0 2024-05-04T03:14:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd57fff640 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd577fe640 1 -- 172.21.15.39:0/3701627961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c750a2f0 msgr2=0x55b1c78d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd577fe640 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c750a2f0 0x55b1c78d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd577fe640 1 -- 172.21.15.39:0/3701627961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 msgr2=0x55b1c78e6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd577fe640 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78e6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.681+0000 7efd577fe640 1 -- 172.21.15.39:0/3701627961 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c78ea050 con 0x55b1c75124d0 2024-05-04T03:14:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.682+0000 7efd577fe640 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e5b10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd4c025d00 tx=0x7efd4c0105e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6ca58c461f805e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.682+0000 7efd3d7fa640 1 -- 172.21.15.39:0/3701627961 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c02a070 con 0x55b1c75124d0 2024-05-04T03:14:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.683+0000 7efd3d7fa640 1 -- 172.21.15.39:0/3701627961 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd4c00fe40 con 0x55b1c75124d0 2024-05-04T03:14:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.683+0000 7efd3d7fa640 1 -- 172.21.15.39:0/3701627961 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c005290 con 0x55b1c75124d0 2024-05-04T03:14:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.684+0000 7efd6501d740 1 -- 172.21.15.39:0/3701627961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 msgr2=0x55b1c78e5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.684+0000 7efd6501d740 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e5b10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efd4c025d00 tx=0x7efd4c0105e0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.684+0000 7efd6501d740 1 -- 172.21.15.39:0/3701627961 shutdown_connections 2024-05-04T03:14:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.685+0000 7efd6501d740 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c75331b0 0x55b1c78e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.685+0000 7efd6501d740 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e5b10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.685+0000 7efd6501d740 1 --2- 172.21.15.39:0/3701627961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c750a2f0 0x55b1c78d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.685+0000 7efd6501d740 1 -- 172.21.15.39:0/3701627961 >> 172.21.15.39:0/3701627961 conn(0x55b1c7527300 msgr2=0x55b1c78cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.685+0000 7efd6501d740 1 -- 172.21.15.39:0/3701627961 shutdown_connections 2024-05-04T03:14:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.685+0000 7efd6501d740 1 -- 172.21.15.39:0/3701627961 wait complete. 2024-05-04T03:14:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.685+0000 7efd6501d740 1 -- start start 2024-05-04T03:14:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.686+0000 7efd6501d740 1 Processor -- start 2024-05-04T03:14:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd6501d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c750a2f0 0x55b1c78e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd6501d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd6501d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75331b0 0x55b1c78e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd6501d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd581b2dc0 con 0x55b1c750a2f0 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd6501d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd581b26f0 con 0x55b1c75124d0 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd6501d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd581af930 con 0x55b1c75331b0 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd6501d740 10 client.0 _ll_register_callbacks cb 0x7efd4c00d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd57fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c750a2f0 0x55b1c78e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd57fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c750a2f0 0x55b1c78e37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35160/0 (socket says 172.21.15.39:35160) 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd57fff640 1 -- 172.21.15.39:0/1778785698 learned_addr learned my addr 172.21.15.39:0/1778785698 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.687+0000 7efd57fff640 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75331b0 0x55b1c78e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd577fe640 1 -- 172.21.15.39:0/1778785698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75331b0 msgr2=0x55b1c78e4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd577fe640 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75331b0 0x55b1c78e4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd577fe640 1 -- 172.21.15.39:0/1778785698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c750a2f0 msgr2=0x55b1c78e37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd577fe640 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c750a2f0 0x55b1c78e37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd577fe640 1 -- 172.21.15.39:0/1778785698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c78e5190 con 0x55b1c75124d0 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd57fff640 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75331b0 0x55b1c78e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd577fe640 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e3d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd4c013fd0 tx=0x7efd4c025600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef528a9e6ae2724 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd36ffd640 1 -- 172.21.15.39:0/1778785698 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c02a070 con 0x55b1c75124d0 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd36ffd640 1 -- 172.21.15.39:0/1778785698 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efd4c025a00 con 0x55b1c75124d0 2024-05-04T03:14:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.688+0000 7efd6501d740 1 -- 172.21.15.39:0/1778785698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4c00fe40 con 0x55b1c75124d0 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd36ffd640 1 -- 172.21.15.39:0/1778785698 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efd4c01f040 con 0x55b1c75124d0 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd6501d740 10 client.25397 fetch_fsmap learned FSMap version 32 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd6501d740 1 -- 172.21.15.39:0/1778785698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b1c78e6400 con 0x55b1c75124d0 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd36ffd640 1 -- 172.21.15.39:0/1778785698 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd4c00fe40 con 0x55b1c75124d0 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd36ffd640 1 -- 172.21.15.39:0/1778785698 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7efd4c019070 con 0x55b1c75124d0 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd6501d740 10 client.25397 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd6501d740 10 client.25397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd6501d740 20 client.25397 populate_metadata read hostname 'smithi039' 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd6501d740 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b1c74c71c0 0x55b1c7956bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd6501d740 4 client.25397 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:14:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd6501d740 1 -- 172.21.15.39:0/1778785698 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1c7841050 con 0x55b1c74c71c0 2024-05-04T03:14:24.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:24 smithi039 ceph-mon[29715]: pgmap v494: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 1.83k op/s 2024-05-04T03:14:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.689+0000 7efd57fff640 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b1c74c71c0 0x55b1c7956bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.691+0000 7efd36ffd640 10 client.25397 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.691+0000 7efd57fff640 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b1c74c71c0 0x55b1c7956bd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c8a1eee2df85d0 server_cookie=2f8410ab50b5481d in_seq=0 out_seq=0 2024-05-04T03:14:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.699+0000 7efd36ffd640 1 -- 172.21.15.39:0/1778785698 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1c7841050 con 0x55b1c74c71c0 2024-05-04T03:14:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.699+0000 7efd36ffd640 10 client.25397 handle_command_reply: tid=0 2024-05-04T03:14:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.699+0000 7efd6501d740 1 client.25397 shutdown 2024-05-04T03:14:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 -- 172.21.15.39:0/1778785698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 msgr2=0x55b1c78e3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e3d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd4c013fd0 tx=0x7efd4c025600 comp rx=0 tx=0).stop 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 -- 172.21.15.39:0/1778785698 shutdown_connections 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b1c74c71c0 0x55b1c7956bd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1c75331b0 0x55b1c78e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b1c75124d0 0x55b1c78e3d10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd36ffd640 0 client.25397 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 --2- 172.21.15.39:0/1778785698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1c750a2f0 0x55b1c78e37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 -- 172.21.15.39:0/1778785698 >> 172.21.15.39:0/1778785698 conn(0x55b1c7527300 msgr2=0x55b1c78ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 -- 172.21.15.39:0/1778785698 shutdown_connections 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 1 -- 172.21.15.39:0/1778785698 wait complete. 2024-05-04T03:14:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.700+0000 7efd6501d740 20 client.25397 trim_cache size 0 max 16384 2024-05-04T03:14:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:14:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:14:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:14:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:14:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 -- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd400b28b0 msgr2=0x7efd400b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd400b28b0 0x7efd400b4dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efd48005ab0 tx=0x7efd48106580 comp rx=0 tx=0).stop 2024-05-04T03:14:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 -- 172.21.15.39:0/1077500880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd581070d0 msgr2=0x7efd5807ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd581070d0 0x7efd5807ae80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd48031460 tx=0x7efd48003bb0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 -- 172.21.15.39:0/1077500880 shutdown_connections 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd400b28b0 0x7efd400b4dc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5810bdd0 0x7efd58077490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd581070d0 0x7efd5807ae80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 --2- 172.21.15.39:0/1077500880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd58105e90 0x7efd5807a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.701+0000 7efd5e092640 1 -- 172.21.15.39:0/1077500880 >> 172.21.15.39:0/1077500880 conn(0x7efd58101060 msgr2=0x7efd58107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.702+0000 7efd5e092640 1 -- 172.21.15.39:0/1077500880 shutdown_connections 2024-05-04T03:14:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:24.702+0000 7efd5e092640 1 -- 172.21.15.39:0/1077500880 wait complete. 2024-05-04T03:14:26.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:26 smithi066 ceph-mon[33264]: pgmap v495: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:26.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:26 smithi167 ceph-mon[33614]: pgmap v495: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:26.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:26 smithi039 ceph-mon[29715]: pgmap v495: 97 pgs: 97 active+clean; 955 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:26.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 35.0 s, 1378.2 tps, lat 48.923 ms stddev 21.130 2024-05-04T03:14:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:28 smithi066 ceph-mon[33264]: pgmap v496: 97 pgs: 97 active+clean; 955 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:28 smithi167 ceph-mon[33614]: pgmap v496: 97 pgs: 97 active+clean; 955 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:28.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:28 smithi039 ceph-mon[29715]: pgmap v496: 97 pgs: 97 active+clean; 955 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:30 smithi066 ceph-mon[33264]: pgmap v497: 97 pgs: 97 active+clean; 956 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 1.75k op/s 2024-05-04T03:14:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:30 smithi167 ceph-mon[33614]: pgmap v497: 97 pgs: 97 active+clean; 956 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 1.75k op/s 2024-05-04T03:14:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:30 smithi039 ceph-mon[29715]: pgmap v497: 97 pgs: 97 active+clean; 956 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 15 MiB/s wr, 1.75k op/s 2024-05-04T03:14:31.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 40.0 s, 1418.6 tps, lat 93.344 ms stddev 634.212 2024-05-04T03:14:32.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:32 smithi066 ceph-mon[33264]: pgmap v498: 97 pgs: 97 active+clean; 957 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:32.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:32 smithi167 ceph-mon[33614]: pgmap v498: 97 pgs: 97 active+clean; 957 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:32.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:32 smithi039 ceph-mon[29715]: pgmap v498: 97 pgs: 97 active+clean; 957 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:34.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:34 smithi066 ceph-mon[33264]: pgmap v499: 97 pgs: 97 active+clean; 958 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-05-04T03:14:34.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:34 smithi167 ceph-mon[33614]: pgmap v499: 97 pgs: 97 active+clean; 958 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-05-04T03:14:34.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:34 smithi039 ceph-mon[29715]: pgmap v499: 97 pgs: 97 active+clean; 958 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-05-04T03:14:36.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:36 smithi066 ceph-mon[33264]: pgmap v500: 97 pgs: 97 active+clean; 958 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 1.56k op/s 2024-05-04T03:14:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:36 smithi167 ceph-mon[33614]: pgmap v500: 97 pgs: 97 active+clean; 958 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 1.56k op/s 2024-05-04T03:14:36.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:36 smithi039 ceph-mon[29715]: pgmap v500: 97 pgs: 97 active+clean; 958 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 1.56k op/s 2024-05-04T03:14:36.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 45.0 s, 1410.1 tps, lat 64.895 ms stddev 16.412 2024-05-04T03:14:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.858+0000 7ff59032e640 1 -- 172.21.15.39:0/3963249192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff588102550 msgr2=0x7ff588102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.858+0000 7ff59032e640 1 --2- 172.21.15.39:0/3963249192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff588102550 0x7ff588102960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff584009860 tx=0x7ff58402a340 comp rx=0 tx=0).stop 2024-05-04T03:14:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.858+0000 7ff59032e640 1 -- 172.21.15.39:0/3963249192 shutdown_connections 2024-05-04T03:14:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.858+0000 7ff59032e640 1 --2- 172.21.15.39:0/3963249192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5881037f0 0x7ff588103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.858+0000 7ff59032e640 1 --2- 172.21.15.39:0/3963249192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff588102ea0 0x7ff5881032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.858+0000 7ff59032e640 1 --2- 172.21.15.39:0/3963249192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff588102550 0x7ff588102960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.858+0000 7ff59032e640 1 -- 172.21.15.39:0/3963249192 >> 172.21.15.39:0/3963249192 conn(0x7ff5880fdd60 msgr2=0x7ff588100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.858+0000 7ff59032e640 1 -- 172.21.15.39:0/3963249192 shutdown_connections 2024-05-04T03:14:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.859+0000 7ff59032e640 1 -- 172.21.15.39:0/3963249192 wait complete. 2024-05-04T03:14:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.859+0000 7ff59032e640 1 Processor -- start 2024-05-04T03:14:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.859+0000 7ff59032e640 1 -- start start 2024-05-04T03:14:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff59032e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588102550 0x7ff5881a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff59032e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff588102ea0 0x7ff5881a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff59032e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5881037f0 0x7ff5881a7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff59032e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff588104a00 con 0x7ff588102ea0 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff59032e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff588104880 con 0x7ff5881037f0 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff59032e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff588104b80 con 0x7ff588102550 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff58e8cb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5881037f0 0x7ff5881a7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff58e8cb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5881037f0 0x7ff5881a7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:45678/0 (socket says 172.21.15.39:45678) 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff58e8cb640 1 -- 172.21.15.39:0/1006055667 learned_addr learned my addr 172.21.15.39:0/1006055667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.860+0000 7ff58e0ca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588102550 0x7ff5881a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.861+0000 7ff58e0ca640 1 -- 172.21.15.39:0/1006055667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5881037f0 msgr2=0x7ff5881a7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.861+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5881037f0 0x7ff5881a7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.861+0000 7ff58e0ca640 1 -- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff588102ea0 msgr2=0x7ff5881a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.861+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff588102ea0 0x7ff5881a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.861+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff588102ea0 0x7ff5881a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.861+0000 7ff58e0ca640 1 -- 172.21.15.39:0/1006055667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5881a81e0 con 0x7ff588102550 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.861+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff588102ea0 0x7ff5881a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.861+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588102550 0x7ff5881a06a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff584009780 tx=0x7ff584030af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.862+0000 7ff57f7fe640 1 -- 172.21.15.39:0/1006055667 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584037040 con 0x7ff588102550 2024-05-04T03:14:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.862+0000 7ff57f7fe640 1 -- 172.21.15.39:0/1006055667 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff584030da0 con 0x7ff588102550 2024-05-04T03:14:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.862+0000 7ff57f7fe640 1 -- 172.21.15.39:0/1006055667 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584003850 con 0x7ff588102550 2024-05-04T03:14:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.862+0000 7ff59032e640 1 -- 172.21.15.39:0/1006055667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5881a8470 con 0x7ff588102550 2024-05-04T03:14:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.862+0000 7ff59032e640 1 -- 172.21.15.39:0/1006055667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5881a88c0 con 0x7ff588102550 2024-05-04T03:14:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.865+0000 7ff5971da740 1 Processor -- start 2024-05-04T03:14:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.865+0000 7ff57f7fe640 1 -- 172.21.15.39:0/1006055667 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff584042020 con 0x7ff588102550 2024-05-04T03:14:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff5971da740 1 -- start start 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff5971da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a891da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff5971da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a891dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff5971da740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8924ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff5971da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645a88825d0 con 0x5645a85532f0 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff5971da740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5645a883b890 con 0x5645a855b4d0 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff5971da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5645a887e590 con 0x5645a857c1b0 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff58e0ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a891dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff58d8c9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8924ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff58e8cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a891da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff58e0ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a891dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49922/0 (socket says 172.21.15.39:49922) 2024-05-04T03:14:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.866+0000 7ff58e0ca640 1 -- 172.21.15.39:0/2801764596 learned_addr learned my addr 172.21.15.39:0/2801764596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.867+0000 7ff58d8c9640 1 -- 172.21.15.39:0/2801764596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 msgr2=0x5645a891da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.867+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/2801764596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a891da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.867+0000 7ff58d8c9640 1 -- 172.21.15.39:0/2801764596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 msgr2=0x5645a891dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.867+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/2801764596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a891dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.867+0000 7ff58d8c9640 1 -- 172.21.15.39:0/2801764596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645a8594210 con 0x5645a855b4d0 2024-05-04T03:14:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.867+0000 7ff57f7fe640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5740b28e0 0x7ff5740b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.868+0000 7ff58e8cb640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5740b28e0 0x7ff5740b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.868+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/2801764596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8924ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff578005fd0 tx=0x7ff578010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21061b2eabad4c1d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.868+0000 7ff57f7fe640 1 -- 172.21.15.39:0/1006055667 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff584101970 con 0x7ff588102550 2024-05-04T03:14:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.868+0000 7ff57cbfe640 1 -- 172.21.15.39:0/2801764596 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578009e50 con 0x5645a855b4d0 2024-05-04T03:14:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.868+0000 7ff57cbfe640 1 -- 172.21.15.39:0/2801764596 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5780035f0 con 0x5645a855b4d0 2024-05-04T03:14:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.868+0000 7ff57cbfe640 1 -- 172.21.15.39:0/2801764596 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578012690 con 0x5645a855b4d0 2024-05-04T03:14:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.869+0000 7ff58e8cb640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5740b28e0 0x7ff5740b4df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5645a891ec30 tx=0x7ff580009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 -- 172.21.15.39:0/2801764596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 msgr2=0x5645a8924ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 --2- 172.21.15.39:0/2801764596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8924ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff578005fd0 tx=0x7ff578010740 comp rx=0 tx=0).stop 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 -- 172.21.15.39:0/2801764596 shutdown_connections 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 --2- 172.21.15.39:0/2801764596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8924ee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 --2- 172.21.15.39:0/2801764596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a891dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 --2- 172.21.15.39:0/2801764596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a891da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 -- 172.21.15.39:0/2801764596 >> 172.21.15.39:0/2801764596 conn(0x5645a8570300 msgr2=0x5645a89193a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 -- 172.21.15.39:0/2801764596 shutdown_connections 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.870+0000 7ff5971da740 1 -- 172.21.15.39:0/2801764596 wait complete. 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 1 -- start start 2024-05-04T03:14:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 1 Processor -- start 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a85532f0 0x5645a89276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a8927be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a857c1b0 0x5645a892eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645a8925930 con 0x5645a855b4d0 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5645a883f370 con 0x5645a857c1b0 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5645a8925ab0 con 0x5645a85532f0 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff5971da740 10 client.0 _ll_register_callbacks cb 0x5645a8593a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff58e8cb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a857c1b0 0x5645a892eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff58e0ca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a85532f0 0x5645a89276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff58e8cb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a857c1b0 0x5645a892eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:45682/0 (socket says 172.21.15.39:45682) 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff58e8cb640 1 -- 172.21.15.39:0/4264859258 learned_addr learned my addr 172.21.15.39:0/4264859258 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.871+0000 7ff58d8c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a8927be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff58e0ca640 1 -- 172.21.15.39:0/4264859258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a857c1b0 msgr2=0x5645a892eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a857c1b0 0x5645a892eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff58e0ca640 1 -- 172.21.15.39:0/4264859258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 msgr2=0x5645a8927be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a8927be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff58e0ca640 1 -- 172.21.15.39:0/4264859258 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645a8660620 con 0x5645a85532f0 2024-05-04T03:14:37.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a8927be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a85532f0 0x5645a89276a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5840c8c40 tx=0x7ff5841074b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eef0bd0442683bf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff571ffb640 1 -- 172.21.15.39:0/4264859258 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584107020 con 0x5645a85532f0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.872+0000 7ff571ffb640 1 -- 172.21.15.39:0/4264859258 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff5840d1b20 con 0x5645a85532f0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.873+0000 7ff5971da740 1 -- 172.21.15.39:0/4264859258 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5780035f0 con 0x5645a85532f0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.873+0000 7ff571ffb640 1 -- 172.21.15.39:0/4264859258 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff584104e60 con 0x5645a85532f0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.873+0000 7ff5971da740 10 client.25411 fetch_fsmap learned FSMap version 32 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.873+0000 7ff5971da740 1 -- 172.21.15.39:0/4264859258 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5645a8925a00 con 0x5645a85532f0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.873+0000 7ff571ffb640 1 -- 172.21.15.39:0/4264859258 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5780035f0 con 0x5645a85532f0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.874+0000 7ff571ffb640 1 -- 172.21.15.39:0/4264859258 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff58403a070 con 0x5645a85532f0 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.874+0000 7ff5971da740 10 client.25411 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.874+0000 7ff5971da740 10 client.25411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.874+0000 7ff5971da740 20 client.25411 populate_metadata read hostname 'smithi039' 2024-05-04T03:14:37.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.874+0000 7ff5971da740 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645a853ad80 0x5645a8935260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.875+0000 7ff5971da740 4 client.25411 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:14:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.875+0000 7ff5971da740 1 -- 172.21.15.39:0/4264859258 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645a88901e0 con 0x5645a853ad80 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.875+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645a853ad80 0x5645a8935260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.876+0000 7ff571ffb640 10 client.25411 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.876+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645a853ad80 0x5645a8935260 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c43cac8dc5dd7bf server_cookie=815cc3afdcf5af77 in_seq=0 out_seq=0 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.877+0000 7ff571ffb640 1 -- 172.21.15.39:0/4264859258 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645a88901e0 con 0x5645a853ad80 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.877+0000 7ff571ffb640 10 client.25411 handle_command_reply: tid=0 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.877+0000 7ff5971da740 1 client.25411 shutdown 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.877+0000 7ff5971da740 1 -- 172.21.15.39:0/4264859258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a85532f0 msgr2=0x5645a89276a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.877+0000 7ff5971da740 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a85532f0 0x5645a89276a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5840c8c40 tx=0x7ff5841074b0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.878+0000 7ff5971da740 1 -- 172.21.15.39:0/4264859258 shutdown_connections 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.878+0000 7ff5971da740 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645a853ad80 0x5645a8935260 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.878+0000 7ff5971da740 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a857c1b0 0x5645a892eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.878+0000 7ff5971da740 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a8927be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.878+0000 7ff5971da740 1 --2- 172.21.15.39:0/4264859258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a85532f0 0x5645a89276a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.878+0000 7ff5971da740 1 -- 172.21.15.39:0/4264859258 >> 172.21.15.39:0/4264859258 conn(0x5645a8570300 msgr2=0x5645a89183a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.878+0000 7ff571ffb640 0 client.25411 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.879+0000 7ff5971da740 1 -- 172.21.15.39:0/4264859258 shutdown_connections 2024-05-04T03:14:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.879+0000 7ff5971da740 1 -- 172.21.15.39:0/4264859258 wait complete. 2024-05-04T03:14:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.879+0000 7ff5971da740 20 client.25411 trim_cache size 0 max 16384 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.906+0000 7ff5971da740 1 Processor -- start 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.906+0000 7ff5971da740 1 -- start start 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.906+0000 7ff5971da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a8918cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.906+0000 7ff5971da740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8919230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.906+0000 7ff5971da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a891c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.906+0000 7ff5971da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881b1670 con 0x5645a85532f0 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.906+0000 7ff5971da740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881b31b0 con 0x5645a855b4d0 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.906+0000 7ff5971da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881b1d40 con 0x5645a857c1b0 2024-05-04T03:14:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a8918cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a8918cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49940/0 (socket says 172.21.15.39:49940) 2024-05-04T03:14:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 -- 172.21.15.39:0/1482400396 learned_addr learned my addr 172.21.15.39:0/1482400396 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e8cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a891c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8919230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 -- 172.21.15.39:0/1482400396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 msgr2=0x5645a891c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a891c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 -- 172.21.15.39:0/1482400396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 msgr2=0x5645a8919230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8919230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 -- 172.21.15.39:0/1482400396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645a8919d00 con 0x5645a85532f0 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8919230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.907+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a8918cf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff5840c8ed0 tx=0x7ff584009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3571c5c4ab3fb54 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.908+0000 7ff573fff640 1 -- 172.21.15.39:0/1482400396 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840e3050 con 0x5645a85532f0 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.908+0000 7ff573fff640 1 -- 172.21.15.39:0/1482400396 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5840d1b20 con 0x5645a85532f0 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.908+0000 7ff573fff640 1 -- 172.21.15.39:0/1482400396 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840c8d10 con 0x5645a85532f0 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 -- 172.21.15.39:0/1482400396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 msgr2=0x5645a8918cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a8918cf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff5840c8ed0 tx=0x7ff584009550 comp rx=0 tx=0).stop 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 -- 172.21.15.39:0/1482400396 shutdown_connections 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a891c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a855b4d0 0x5645a8919230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 --2- 172.21.15.39:0/1482400396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a85532f0 0x5645a8918cf0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 -- 172.21.15.39:0/1482400396 >> 172.21.15.39:0/1482400396 conn(0x5645a8570300 msgr2=0x5645a8921b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 -- 172.21.15.39:0/1482400396 shutdown_connections 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.909+0000 7ff5971da740 1 -- 172.21.15.39:0/1482400396 wait complete. 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.910+0000 7ff5971da740 1 -- start start 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.910+0000 7ff5971da740 1 Processor -- start 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff5971da740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a85532f0 0x5645a892b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff5971da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a892b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff5971da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a899b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff5971da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881b2ae0 con 0x5645a855b4d0 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff58e0ca640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a85532f0 0x5645a892b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff5971da740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881b2410 con 0x5645a85532f0 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff5971da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881af650 con 0x5645a857c1b0 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff58e8cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a899b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff58e0ca640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a85532f0 0x5645a892b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:45698/0 (socket says 172.21.15.39:45698) 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff58e0ca640 1 -- 172.21.15.39:0/2808893404 learned_addr learned my addr 172.21.15.39:0/2808893404 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a892b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.911+0000 7ff5971da740 10 client.0 _ll_register_callbacks cb 0x7ff584030f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff58e8cb640 1 -- 172.21.15.39:0/2808893404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a85532f0 msgr2=0x5645a892b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff58e8cb640 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a85532f0 0x5645a892b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff58e8cb640 1 -- 172.21.15.39:0/2808893404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 msgr2=0x5645a892b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff58e8cb640 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a892b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff58e8cb640 1 -- 172.21.15.39:0/2808893404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645a899bb40 con 0x5645a857c1b0 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff58d8c9640 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a892b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a85532f0 0x5645a892b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff58e8cb640 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a899b600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff580011eb0 tx=0x7ff580004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5d5fac76f314b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:14:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.912+0000 7ff571ffb640 1 -- 172.21.15.39:0/2808893404 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58000fca0 con 0x5645a857c1b0 2024-05-04T03:14:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.913+0000 7ff5971da740 1 -- 172.21.15.39:0/2808893404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5840d1b20 con 0x5645a857c1b0 2024-05-04T03:14:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.913+0000 7ff571ffb640 1 -- 172.21.15.39:0/2808893404 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff580004b20 con 0x5645a857c1b0 2024-05-04T03:14:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.913+0000 7ff571ffb640 1 -- 172.21.15.39:0/2808893404 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff58000fe40 con 0x5645a857c1b0 2024-05-04T03:14:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.913+0000 7ff5971da740 10 client.25417 fetch_fsmap learned FSMap version 32 2024-05-04T03:14:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.913+0000 7ff5971da740 1 -- 172.21.15.39:0/2808893404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5645a899ce10 con 0x5645a857c1b0 2024-05-04T03:14:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.913+0000 7ff571ffb640 1 -- 172.21.15.39:0/2808893404 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5840d1b20 con 0x5645a857c1b0 2024-05-04T03:14:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.913+0000 7ff571ffb640 1 -- 172.21.15.39:0/2808893404 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff58001e430 con 0x5645a857c1b0 2024-05-04T03:14:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.914+0000 7ff5971da740 10 client.25417 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:14:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.914+0000 7ff5971da740 10 client.25417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:14:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.914+0000 7ff5971da740 20 client.25417 populate_metadata read hostname 'smithi039' 2024-05-04T03:14:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.914+0000 7ff5971da740 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645a85101c0 0x5645a899f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.914+0000 7ff5971da740 4 client.25417 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:14:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.914+0000 7ff5971da740 1 -- 172.21.15.39:0/2808893404 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645a888a050 con 0x5645a85101c0 2024-05-04T03:14:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.914+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645a85101c0 0x5645a899f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.914+0000 7ff571ffb640 10 client.25417 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.915+0000 7ff58e0ca640 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645a85101c0 0x5645a899f860 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d1525677a6c5bb server_cookie=a1d25acceb4797ca in_seq=0 out_seq=0 2024-05-04T03:14:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.922+0000 7ff571ffb640 1 -- 172.21.15.39:0/2808893404 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645a888a050 con 0x5645a85101c0 2024-05-04T03:14:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.922+0000 7ff571ffb640 10 client.25417 handle_command_reply: tid=0 2024-05-04T03:14:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.922+0000 7ff5971da740 1 client.25417 shutdown 2024-05-04T03:14:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.922+0000 7ff5971da740 1 -- 172.21.15.39:0/2808893404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 msgr2=0x5645a899b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.922+0000 7ff5971da740 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a899b600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff580011eb0 tx=0x7ff580004a40 comp rx=0 tx=0).stop 2024-05-04T03:14:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff5971da740 1 -- 172.21.15.39:0/2808893404 shutdown_connections 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff5971da740 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645a85101c0 0x5645a899f860 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff571ffb640 0 client.25417 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff5971da740 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645a857c1b0 0x5645a899b600 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff5971da740 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645a855b4d0 0x5645a892b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff5971da740 1 --2- 172.21.15.39:0/2808893404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645a85532f0 0x5645a892b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff5971da740 1 -- 172.21.15.39:0/2808893404 >> 172.21.15.39:0/2808893404 conn(0x5645a8570300 msgr2=0x5645a8918cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff5971da740 1 -- 172.21.15.39:0/2808893404 shutdown_connections 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.923+0000 7ff5971da740 1 -- 172.21.15.39:0/2808893404 wait complete. 2024-05-04T03:14:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.924+0000 7ff5971da740 20 client.25417 trim_cache size 0 max 16384 2024-05-04T03:14:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:14:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:14:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:14:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 -- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5740b28e0 msgr2=0x7ff5740b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5740b28e0 0x7ff5740b4df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5645a891ec30 tx=0x7ff580009500 comp rx=0 tx=0).stop 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 -- 172.21.15.39:0/1006055667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588102550 msgr2=0x7ff5881a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588102550 0x7ff5881a06a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff584009780 tx=0x7ff584030af0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 -- 172.21.15.39:0/1006055667 shutdown_connections 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5740b28e0 0x7ff5740b4df0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5881037f0 0x7ff5881a7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff588102ea0 0x7ff5881a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 --2- 172.21.15.39:0/1006055667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588102550 0x7ff5881a06a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 -- 172.21.15.39:0/1006055667 >> 172.21.15.39:0/1006055667 conn(0x7ff5880fdd60 msgr2=0x7ff5880fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 -- 172.21.15.39:0/1006055667 shutdown_connections 2024-05-04T03:14:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:37.925+0000 7ff59032e640 1 -- 172.21.15.39:0/1006055667 wait complete. 2024-05-04T03:14:38.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:38 smithi066 ceph-mon[33264]: pgmap v501: 97 pgs: 97 active+clean; 958 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-05-04T03:14:38.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:38 smithi167 ceph-mon[33614]: pgmap v501: 97 pgs: 97 active+clean; 958 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-05-04T03:14:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:38 smithi039 ceph-mon[29715]: pgmap v501: 97 pgs: 97 active+clean; 958 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 1.67k op/s 2024-05-04T03:14:40.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:40 smithi066 ceph-mon[33264]: pgmap v502: 97 pgs: 97 active+clean; 966 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 MiB/s wr, 1.82k op/s 2024-05-04T03:14:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:40 smithi167 ceph-mon[33614]: pgmap v502: 97 pgs: 97 active+clean; 966 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 MiB/s wr, 1.82k op/s 2024-05-04T03:14:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:40 smithi039 ceph-mon[29715]: pgmap v502: 97 pgs: 97 active+clean; 966 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 16 MiB/s wr, 1.82k op/s 2024-05-04T03:14:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 50.0 s, 1469.1 tps, lat 73.911 ms stddev 73.466 2024-05-04T03:14:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:42 smithi066 ceph-mon[33264]: pgmap v503: 97 pgs: 97 active+clean; 975 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 15 MiB/s wr, 1.63k op/s 2024-05-04T03:14:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:42 smithi167 ceph-mon[33614]: pgmap v503: 97 pgs: 97 active+clean; 975 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 15 MiB/s wr, 1.63k op/s 2024-05-04T03:14:42.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:42 smithi039 ceph-mon[29715]: pgmap v503: 97 pgs: 97 active+clean; 975 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 15 MiB/s wr, 1.63k op/s 2024-05-04T03:14:44.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:44 smithi066 ceph-mon[33264]: pgmap v504: 97 pgs: 97 active+clean; 978 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.75k op/s 2024-05-04T03:14:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:44 smithi167 ceph-mon[33614]: pgmap v504: 97 pgs: 97 active+clean; 978 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.75k op/s 2024-05-04T03:14:44.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:44 smithi039 ceph-mon[29715]: pgmap v504: 97 pgs: 97 active+clean; 978 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 1.75k op/s 2024-05-04T03:14:46.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:46 smithi066 ceph-mon[33264]: pgmap v505: 97 pgs: 97 active+clean; 978 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-04T03:14:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:46 smithi167 ceph-mon[33614]: pgmap v505: 97 pgs: 97 active+clean; 978 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-04T03:14:46.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:46 smithi039 ceph-mon[29715]: pgmap v505: 97 pgs: 97 active+clean; 978 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-04T03:14:46.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 55.0 s, 1459.2 tps, lat 68.497 ms stddev 18.907 2024-05-04T03:14:48.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:48 smithi066 ceph-mon[33264]: pgmap v506: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-04T03:14:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:48 smithi167 ceph-mon[33614]: pgmap v506: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-04T03:14:48.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:48 smithi039 ceph-mon[29715]: pgmap v506: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.68k op/s 2024-05-04T03:14:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:50 smithi066 ceph-mon[33264]: pgmap v507: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.70k op/s 2024-05-04T03:14:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:50 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:50 smithi167 ceph-mon[33614]: pgmap v507: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.70k op/s 2024-05-04T03:14:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:50 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:50.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:50 smithi039 ceph-mon[29715]: pgmap v507: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.70k op/s 2024-05-04T03:14:50.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:50 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 -- 172.21.15.39:0/827037925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810f880 msgr2=0x7feae810fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 --2- 172.21.15.39:0/827037925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810f880 0x7feae810fc90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fead8009860 tx=0x7fead802a340 comp rx=0 tx=0).stop 2024-05-04T03:14:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 -- 172.21.15.39:0/827037925 shutdown_connections 2024-05-04T03:14:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 --2- 172.21.15.39:0/827037925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae8114580 0x7feae81169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 --2- 172.21.15.39:0/827037925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810f880 0x7feae810fc90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 --2- 172.21.15.39:0/827037925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae810e640 0x7feae810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 -- 172.21.15.39:0/827037925 >> 172.21.15.39:0/827037925 conn(0x7feae806e700 msgr2=0x7feae810bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 -- 172.21.15.39:0/827037925 shutdown_connections 2024-05-04T03:14:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.090+0000 7feaecdb7640 1 -- 172.21.15.39:0/827037925 wait complete. 2024-05-04T03:14:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feaecdb7640 1 Processor -- start 2024-05-04T03:14:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feaecdb7640 1 -- start start 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feaecdb7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810e640 0x7feae81a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feaecdb7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae810f880 0x7feae81a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feaecdb7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8114580 0x7feae81ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feae659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810e640 0x7feae81a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8114580 0x7feae81ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feae659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810e640 0x7feae81a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36890/0 (socket says 172.21.15.39:36890) 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8114580 0x7feae81ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37794/0 (socket says 172.21.15.39:37794) 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feaecdb7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feae8119ff0 con 0x7feae8114580 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feaecdb7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feae8119e70 con 0x7feae810f880 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feaecdb7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feae811a170 con 0x7feae810e640 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.092+0000 7feae659c640 1 -- 172.21.15.39:0/592546753 learned_addr learned my addr 172.21.15.39:0/592546753 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7feae5d9b640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae810f880 0x7feae81a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7feae659c640 1 -- 172.21.15.39:0/592546753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae810f880 msgr2=0x7feae81a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7feae659c640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae810f880 0x7feae81a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7feae659c640 1 -- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8114580 msgr2=0x7feae81ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7feae659c640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8114580 0x7feae81ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7feae659c640 1 -- 172.21.15.39:0/592546753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae81ac7e0 con 0x7feae810e640 2024-05-04T03:14:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7feae6d9d640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8114580 0x7feae81ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7feae659c640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810e640 0x7feae81a4cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feadc005fd0 tx=0x7feadc00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.093+0000 7fead77fe640 1 -- 172.21.15.39:0/592546753 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc017070 con 0x7feae810e640 2024-05-04T03:14:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.094+0000 7fead77fe640 1 -- 172.21.15.39:0/592546753 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feadc009c30 con 0x7feae810e640 2024-05-04T03:14:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.094+0000 7fead77fe640 1 -- 172.21.15.39:0/592546753 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc003c90 con 0x7feae810e640 2024-05-04T03:14:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.094+0000 7feaecdb7640 1 -- 172.21.15.39:0/592546753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae81acad0 con 0x7feae810e640 2024-05-04T03:14:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.094+0000 7feaecdb7640 1 -- 172.21.15.39:0/592546753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae81acfd0 con 0x7feae810e640 2024-05-04T03:14:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.099+0000 7fead77fe640 1 -- 172.21.15.39:0/592546753 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7feadc003e30 con 0x7feae810e640 2024-05-04T03:14:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.099+0000 7feaf3c6d740 1 Processor -- start 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.099+0000 7feaf3c6d740 1 -- start start 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842454a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 0x55d842454fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d8423b95d0 con 0x55d8420b31b0 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d842372890 con 0x55d8420924d0 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d8423b5590 con 0x55d84208a2f0 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842454a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842454a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37810/0 (socket says 172.21.15.39:37810) 2024-05-04T03:14:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae6d9d640 1 -- 172.21.15.39:0/849235144 learned_addr learned my addr 172.21.15.39:0/849235144 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae659c640 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 0x55d842454fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae5d9b640 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae6d9d640 1 -- 172.21.15.39:0/849235144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 msgr2=0x55d842454fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae6d9d640 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 0x55d842454fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae6d9d640 1 -- 172.21.15.39:0/849235144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 msgr2=0x55d84245bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae6d9d640 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.100+0000 7feae6d9d640 1 -- 172.21.15.39:0/849235144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8420cb210 con 0x55d8420b31b0 2024-05-04T03:14:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.101+0000 7feae6d9d640 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842454a60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feae000e850 tx=0x7feae00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2c598c4aa7257e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.101+0000 7fead77fe640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fead00b2910 0x7fead00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.102+0000 7fead4ff9640 1 -- 172.21.15.39:0/849235144 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae001d070 con 0x55d8420b31b0 2024-05-04T03:14:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.102+0000 7fead4ff9640 1 -- 172.21.15.39:0/849235144 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feae0003e00 con 0x55d8420b31b0 2024-05-04T03:14:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.102+0000 7fead4ff9640 1 -- 172.21.15.39:0/849235144 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0004830 con 0x55d8420b31b0 2024-05-04T03:14:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.102+0000 7fead77fe640 1 -- 172.21.15.39:0/592546753 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7feadc0e1df0 con 0x7feae810e640 2024-05-04T03:14:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.102+0000 7feae659c640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fead00b2910 0x7fead00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.103+0000 7feae659c640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fead00b2910 0x7fead00b4e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feadc0a5440 tx=0x7feadc00bf60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.103+0000 7feaf3c6d740 1 -- 172.21.15.39:0/849235144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 msgr2=0x55d842454a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.103+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842454a60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feae000e850 tx=0x7feae00095c0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.103+0000 7feaf3c6d740 1 -- 172.21.15.39:0/849235144 shutdown_connections 2024-05-04T03:14:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.103+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.103+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 0x55d842454fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.103+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/849235144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842454a60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.103+0000 7feaf3c6d740 1 -- 172.21.15.39:0/849235144 >> 172.21.15.39:0/849235144 conn(0x55d8420a7300 msgr2=0x55d8424503a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.104+0000 7feaf3c6d740 1 -- 172.21.15.39:0/849235144 shutdown_connections 2024-05-04T03:14:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.104+0000 7feaf3c6d740 1 -- 172.21.15.39:0/849235144 wait complete. 2024-05-04T03:14:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.104+0000 7feaf3c6d740 1 -- start start 2024-05-04T03:14:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.104+0000 7feaf3c6d740 1 Processor -- start 2024-05-04T03:14:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.104+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 0x55d84245e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842465b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d84245c930 con 0x55d8420b31b0 2024-05-04T03:14:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d842376370 con 0x55d8420924d0 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d84245cab0 con 0x55d84208a2f0 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feaf3c6d740 10 client.0 _ll_register_callbacks cb 0x55d8420caa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842465b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842465b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37822/0 (socket says 172.21.15.39:37822) 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 -- 172.21.15.39:0/1672022721 learned_addr learned my addr 172.21.15.39:0/1672022721 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae5d9b640 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 0x55d84245e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 -- 172.21.15.39:0/1672022721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 msgr2=0x55d84245e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 0x55d84245e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 -- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 msgr2=0x55d84245ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 -- 172.21.15.39:0/1672022721 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d842197620 con 0x55d8420b31b0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae5d9b640 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.105+0000 7feae6d9d640 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842465b20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feae0004260 tx=0x7feae0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3868c0e4e8fdae5b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feacdffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feacdffb640 1 -- 172.21.15.39:0/1672022721 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae001d020 con 0x55d8420b31b0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feacdffb640 1 -- 172.21.15.39:0/1672022721 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7feae0003b80 con 0x55d8420b31b0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feaf3c6d740 1 -- 172.21.15.39:0/1672022721 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae0003e00 con 0x55d8420b31b0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feacdffb640 1 -- 172.21.15.39:0/1672022721 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feae0004a40 con 0x55d8420b31b0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feaf3c6d740 10 client.15777 fetch_fsmap learned FSMap version 32 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feaf3c6d740 1 -- 172.21.15.39:0/1672022721 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d842468a30 con 0x55d8420b31b0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feacdffb640 1 -- 172.21.15.39:0/1672022721 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feae002f070 con 0x55d8420b31b0 2024-05-04T03:14:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feacdffb640 1 -- 172.21.15.39:0/1672022721 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7feae0032020 con 0x55d8420b31b0 2024-05-04T03:14:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feaf3c6d740 10 client.15777 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:14:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feaf3c6d740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:14:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.106+0000 7feaf3c6d740 20 client.15777 populate_metadata read hostname 'smithi039' 2024-05-04T03:14:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.107+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d842071d80 0x55d84246bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.107+0000 7feaf3c6d740 4 client.15777 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:14:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.107+0000 7feaf3c6d740 1 -- 172.21.15.39:0/1672022721 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feae0003e00 con 0x55d842071d80 2024-05-04T03:14:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.107+0000 7feae5d9b640 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d842071d80 0x55d84246bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.107+0000 7feacdffb640 10 client.15777 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.108+0000 7feae5d9b640 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d842071d80 0x55d84246bf00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee8eead0179e324 server_cookie=15a48a1bf43c97f2 in_seq=0 out_seq=0 2024-05-04T03:14:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.108+0000 7feacdffb640 1 -- 172.21.15.39:0/1672022721 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7feae0003e00 con 0x55d842071d80 2024-05-04T03:14:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.108+0000 7feacdffb640 10 client.15777 handle_command_reply: tid=0 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.108+0000 7feaf3c6d740 1 client.15777 shutdown 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 -- 172.21.15.39:0/1672022721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 msgr2=0x55d842465b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842465b20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feae0004260 tx=0x7feae0009b30 comp rx=0 tx=0).stop 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 -- 172.21.15.39:0/1672022721 shutdown_connections 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d842071d80 0x55d84246bf00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feacdffb640 0 client.15777 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420b31b0 0x55d842465b20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d84245ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/1672022721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84208a2f0 0x55d84245e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 -- 172.21.15.39:0/1672022721 >> 172.21.15.39:0/1672022721 conn(0x55d8420a7300 msgr2=0x55d84244f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 -- 172.21.15.39:0/1672022721 shutdown_connections 2024-05-04T03:14:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 1 -- 172.21.15.39:0/1672022721 wait complete. 2024-05-04T03:14:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.109+0000 7feaf3c6d740 20 client.15777 trim_cache size 0 max 16384 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feaf3c6d740 1 Processor -- start 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feaf3c6d740 1 -- start start 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d84244af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d84208a2f0 0x55d842458ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d842459020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feae81b5d50 con 0x55d84208a2f0 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feae81b7890 con 0x55d8420924d0 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feaf3c6d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feae81b6420 con 0x55d842071d80 2024-05-04T03:14:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d84244af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d84244af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36920/0 (socket says 172.21.15.39:36920) 2024-05-04T03:14:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.134+0000 7feae6d9d640 1 -- 172.21.15.39:0/612009696 learned_addr learned my addr 172.21.15.39:0/612009696 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.135+0000 7feae5d9b640 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d84208a2f0 0x55d842458ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.135+0000 7feae6d9d640 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d842459020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.135+0000 7feae6d9d640 1 -- 172.21.15.39:0/612009696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 msgr2=0x55d842459020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.135+0000 7feae6d9d640 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d842459020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.135+0000 7feae6d9d640 1 -- 172.21.15.39:0/612009696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d84208a2f0 msgr2=0x55d842458ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.135+0000 7feae6d9d640 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d84208a2f0 0x55d842458ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.135+0000 7feae6d9d640 1 -- 172.21.15.39:0/612009696 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d842455bf0 con 0x55d842071d80 2024-05-04T03:14:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.135+0000 7feae6d9d640 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d84244af10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feae0005720 tx=0x7feae0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202127a7c0e0d12c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.136+0000 7feacffff640 1 -- 172.21.15.39:0/612009696 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae001d070 con 0x55d842071d80 2024-05-04T03:14:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.136+0000 7feacffff640 1 -- 172.21.15.39:0/612009696 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feae0003d80 con 0x55d842071d80 2024-05-04T03:14:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.136+0000 7feacffff640 1 -- 172.21.15.39:0/612009696 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0005510 con 0x55d842071d80 2024-05-04T03:14:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 -- 172.21.15.39:0/612009696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 msgr2=0x55d84244af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d84244af10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feae0005720 tx=0x7feae0005fd0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 -- 172.21.15.39:0/612009696 shutdown_connections 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8420924d0 0x55d842459020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d84208a2f0 0x55d842458ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/612009696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d84244af10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 -- 172.21.15.39:0/612009696 >> 172.21.15.39:0/612009696 conn(0x55d8420a7300 msgr2=0x55d842454a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 -- 172.21.15.39:0/612009696 shutdown_connections 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.137+0000 7feaf3c6d740 1 -- 172.21.15.39:0/612009696 wait complete. 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.138+0000 7feaf3c6d740 1 -- start start 2024-05-04T03:14:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.138+0000 7feaf3c6d740 1 Processor -- start 2024-05-04T03:14:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.139+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d842460100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.139+0000 7feaf3c6d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d84208a2f0 0x55d842460640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.139+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d842460100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.139+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d842460100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36936/0 (socket says 172.21.15.39:36936) 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.139+0000 7feae5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d84208a2f0 0x55d842460640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.139+0000 7feae5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d84208a2f0 0x55d842460640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:50164/0 (socket says 172.21.15.39:50164) 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae6d9d640 1 -- 172.21.15.39:0/426242234 learned_addr learned my addr 172.21.15.39:0/426242234 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420924d0 0x55d8424d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feae81b71c0 con 0x55d8420924d0 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feae81b6af0 con 0x55d84208a2f0 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feae81b3d30 con 0x55d842071d80 2024-05-04T03:14:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feaf3c6d740 10 client.0 _ll_register_callbacks cb 0x7feae001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae6d9d640 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420924d0 0x55d8424d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae5d9b640 1 -- 172.21.15.39:0/426242234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 msgr2=0x55d842460100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae5d9b640 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d842460100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae5d9b640 1 -- 172.21.15.39:0/426242234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420924d0 msgr2=0x55d8424d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae5d9b640 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420924d0 0x55d8424d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae5d9b640 1 -- 172.21.15.39:0/426242234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8424d27a0 con 0x55d84208a2f0 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae6d9d640 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d842460100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feae5d9b640 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d84208a2f0 0x55d842460640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fead8009780 tx=0x7fead8030890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86663031246e863 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.140+0000 7feacdffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.141+0000 7feacdffb640 1 -- 172.21.15.39:0/426242234 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead804e020 con 0x55d84208a2f0 2024-05-04T03:14:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.141+0000 7feacdffb640 1 -- 172.21.15.39:0/426242234 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fead8030d70 con 0x55d84208a2f0 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.141+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae0003d80 con 0x55d84208a2f0 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.141+0000 7feacdffb640 1 -- 172.21.15.39:0/426242234 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fead8037040 con 0x55d84208a2f0 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.141+0000 7feaf3c6d740 10 client.25433 fetch_fsmap learned FSMap version 32 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.141+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d8424d39c0 con 0x55d84208a2f0 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.141+0000 7feacdffb640 1 -- 172.21.15.39:0/426242234 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feae0003d80 con 0x55d84208a2f0 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.141+0000 7feacdffb640 1 -- 172.21.15.39:0/426242234 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fead80046c0 con 0x55d84208a2f0 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.142+0000 7feaf3c6d740 10 client.25433 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.142+0000 7feaf3c6d740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.142+0000 7feaf3c6d740 20 client.25433 populate_metadata read hostname 'smithi039' 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.142+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d8420b31b0 0x55d8424d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.142+0000 7feaf3c6d740 4 client.25433 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:14:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.142+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8423c71e0 con 0x55d8420b31b0 2024-05-04T03:14:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.142+0000 7feae6d9d640 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d8420b31b0 0x55d8424d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.143+0000 7feae6d9d640 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d8420b31b0 0x55d8424d63b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48118e1a50cbb209 server_cookie=9ffe2e30511e9b2d in_seq=0 out_seq=0 2024-05-04T03:14:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.143+0000 7feacdffb640 10 client.25433 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.151+0000 7feacdffb640 1 -- 172.21.15.39:0/426242234 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8423c71e0 con 0x55d8420b31b0 2024-05-04T03:14:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.151+0000 7feacdffb640 10 client.25433 handle_command_reply: tid=0 2024-05-04T03:14:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 client.25433 shutdown 2024-05-04T03:14:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d84208a2f0 msgr2=0x55d842460640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d84208a2f0 0x55d842460640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fead8009780 tx=0x7fead8030890 comp rx=0 tx=0).stop 2024-05-04T03:14:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 shutdown_connections 2024-05-04T03:14:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d8420b31b0 0x55d8424d63b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d8420924d0 0x55d8424d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d84208a2f0 0x55d842460640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 --2- 172.21.15.39:0/426242234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d842071d80 0x55d842460100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 >> 172.21.15.39:0/426242234 conn(0x55d8420a7300 msgr2=0x55d842458ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feacdffb640 0 client.25433 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 shutdown_connections 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 1 -- 172.21.15.39:0/426242234 wait complete. 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.152+0000 7feaf3c6d740 20 client.25433 trim_cache size 0 max 16384 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:14:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:14:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.153+0000 7feaecdb7640 1 -- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fead00b2910 msgr2=0x7fead00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.153+0000 7feaecdb7640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fead00b2910 0x7fead00b4e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feadc0a5440 tx=0x7feadc00bf60 comp rx=0 tx=0).stop 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.153+0000 7feaecdb7640 1 -- 172.21.15.39:0/592546753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810e640 msgr2=0x7feae81a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.153+0000 7feaecdb7640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810e640 0x7feae81a4cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feadc005fd0 tx=0x7feadc00dde0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.154+0000 7feaecdb7640 1 -- 172.21.15.39:0/592546753 shutdown_connections 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.154+0000 7feaecdb7640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fead00b2910 0x7fead00b4e20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.154+0000 7feaecdb7640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8114580 0x7feae81ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.154+0000 7feaecdb7640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feae810f880 0x7feae81a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.154+0000 7feaecdb7640 1 --2- 172.21.15.39:0/592546753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feae810e640 0x7feae81a4cb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.154+0000 7feaecdb7640 1 -- 172.21.15.39:0/592546753 >> 172.21.15.39:0/592546753 conn(0x7feae806e700 msgr2=0x7feae8110560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.154+0000 7feaecdb7640 1 -- 172.21.15.39:0/592546753 shutdown_connections 2024-05-04T03:14:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:14:51.154+0000 7feaecdb7640 1 -- 172.21.15.39:0/592546753 wait complete. 2024-05-04T03:14:51.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:51 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:51.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:51 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:51.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:51 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:14:51.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:51 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:51.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:51 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:51.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:51 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:14:51.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:51.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:51.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:14:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 60.0 s, 1382.4 tps, lat 72.267 ms stddev 19.840 2024-05-04T03:14:52.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:52 smithi066 ceph-mon[33264]: pgmap v508: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:52 smithi167 ceph-mon[33614]: pgmap v508: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:52.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:52 smithi039 ceph-mon[29715]: pgmap v508: 97 pgs: 97 active+clean; 978 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:14:54.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:54 smithi066 ceph-mon[33264]: pgmap v509: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:54.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:54 smithi167 ceph-mon[33614]: pgmap v509: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:54.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:54 smithi039 ceph-mon[29715]: pgmap v509: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:14:56.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:56 smithi066 ceph-mon[33264]: pgmap v510: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:14:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:56 smithi167 ceph-mon[33614]: pgmap v510: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:14:56.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:56 smithi039 ceph-mon[29715]: pgmap v510: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:14:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 65.0 s, 1353.4 tps, lat 73.870 ms stddev 30.311 2024-05-04T03:14:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:14:58 smithi066 ceph-mon[33264]: pgmap v511: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:14:58.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:14:58 smithi167 ceph-mon[33614]: pgmap v511: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:14:58.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:14:58 smithi039 ceph-mon[29715]: pgmap v511: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:15:00.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:00 smithi066 ceph-mon[33264]: pgmap v512: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.64k op/s 2024-05-04T03:15:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:00 smithi167 ceph-mon[33614]: pgmap v512: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.64k op/s 2024-05-04T03:15:00.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:00 smithi039 ceph-mon[29715]: pgmap v512: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.64k op/s 2024-05-04T03:15:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 70.0 s, 1517.1 tps, lat 65.939 ms stddev 16.611 2024-05-04T03:15:02.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:02 smithi066 ceph-mon[33264]: pgmap v513: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.63k op/s 2024-05-04T03:15:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:02 smithi167 ceph-mon[33614]: pgmap v513: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.63k op/s 2024-05-04T03:15:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:02 smithi039 ceph-mon[29715]: pgmap v513: 97 pgs: 97 active+clean; 979 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.63k op/s 2024-05-04T03:15:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.313+0000 7f24feffd640 1 -- 172.21.15.39:0/942319052 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e80307b0 con 0x7f2500075a10 2024-05-04T03:15:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.314+0000 7f2506896640 1 -- 172.21.15.39:0/942319052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2500075a10 msgr2=0x7f250010fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.314+0000 7f2506896640 1 --2- 172.21.15.39:0/942319052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2500075a10 0x7f250010fe90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009860 tx=0x7f24e802a340 comp rx=0 tx=0).stop 2024-05-04T03:15:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.315+0000 7f2506896640 1 -- 172.21.15.39:0/942319052 shutdown_connections 2024-05-04T03:15:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.315+0000 7f2506896640 1 --2- 172.21.15.39:0/942319052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25001103d0 0x7f25001127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.315+0000 7f2506896640 1 --2- 172.21.15.39:0/942319052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2500075a10 0x7f250010fe90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.315+0000 7f2506896640 1 --2- 172.21.15.39:0/942319052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25000750c0 0x7f25000754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.315+0000 7f2506896640 1 -- 172.21.15.39:0/942319052 >> 172.21.15.39:0/942319052 conn(0x7f25000fee60 msgr2=0x7f2500101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.315+0000 7f2506896640 1 -- 172.21.15.39:0/942319052 shutdown_connections 2024-05-04T03:15:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.315+0000 7f2506896640 1 -- 172.21.15.39:0/942319052 wait complete. 2024-05-04T03:15:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.316+0000 7f2506896640 1 Processor -- start 2024-05-04T03:15:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.316+0000 7f2506896640 1 -- start start 2024-05-04T03:15:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.317+0000 7f2506896640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000750c0 0x7f25001a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.317+0000 7f2506896640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2500075a10 0x7f25001a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.317+0000 7f2506896640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25001103d0 0x7f25001a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000750c0 0x7f25001a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24fffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000750c0 0x7f25001a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:39968/0 (socket says 172.21.15.39:39968) 2024-05-04T03:15:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24fffff640 1 -- 172.21.15.39:0/828429677 learned_addr learned my addr 172.21.15.39:0/828429677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2500115cb0 con 0x7f25001103d0 2024-05-04T03:15:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2500115b30 con 0x7f25000750c0 2024-05-04T03:15:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2500115e30 con 0x7f2500075a10 2024-05-04T03:15:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2500075a10 0x7f25001a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f2504e33640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25001103d0 0x7f25001a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24fffff640 1 -- 172.21.15.39:0/828429677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2500075a10 msgr2=0x7f25001a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24fffff640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2500075a10 0x7f25001a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24fffff640 1 -- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25001103d0 msgr2=0x7f25001a7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24fffff640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25001103d0 0x7f25001a7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.318+0000 7f24fffff640 1 -- 172.21.15.39:0/828429677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25001a83f0 con 0x7f25000750c0 2024-05-04T03:15:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.319+0000 7f24fffff640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000750c0 0x7f25001a08b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24f800eb90 tx=0x7f24f800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.319+0000 7f24fd7fa640 1 -- 172.21.15.39:0/828429677 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8010630 con 0x7f25000750c0 2024-05-04T03:15:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.319+0000 7f24fd7fa640 1 -- 172.21.15.39:0/828429677 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24f800ee90 con 0x7f25000750c0 2024-05-04T03:15:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.320+0000 7f24fd7fa640 1 -- 172.21.15.39:0/828429677 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8018a50 con 0x7f25000750c0 2024-05-04T03:15:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.320+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25001a86e0 con 0x7f25000750c0 2024-05-04T03:15:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.320+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25001a8bb0 con 0x7f25000750c0 2024-05-04T03:15:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.324+0000 7f250d829740 1 Processor -- start 2024-05-04T03:15:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.324+0000 7f250d829740 1 -- start start 2024-05-04T03:15:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f250d829740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f250d829740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f21fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f250d829740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f28ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f250d829740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557237e865d0 con 0x557237b5f4d0 2024-05-04T03:15:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f250d829740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557237e3f890 con 0x557237b572f0 2024-05-04T03:15:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f250d829740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557237e82590 con 0x557237b801b0 2024-05-04T03:15:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f2504e33640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f2504e33640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f21a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:39920/0 (socket says 172.21.15.39:39920) 2024-05-04T03:15:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f2504e33640 1 -- 172.21.15.39:0/16839284 learned_addr learned my addr 172.21.15.39:0/16839284 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/16839284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f28ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f2504e33640 1 -- 172.21.15.39:0/16839284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 msgr2=0x557237f21fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f2504e33640 1 --2- 172.21.15.39:0/16839284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f21fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f2504e33640 1 -- 172.21.15.39:0/16839284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 msgr2=0x557237f28ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f2504e33640 1 --2- 172.21.15.39:0/16839284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f28ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.325+0000 7f2504e33640 1 -- 172.21.15.39:0/16839284 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557237b98210 con 0x557237b801b0 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.326+0000 7f24fd7fa640 1 -- 172.21.15.39:0/828429677 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f24f8018bf0 con 0x7f25000750c0 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.326+0000 7f2504e33640 1 --2- 172.21.15.39:0/16839284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f21a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009b90 tx=0x7f24f00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d50b81247870c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.326+0000 7f24f67fc640 1 -- 172.21.15.39:0/16839284 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0012450 con 0x557237b801b0 2024-05-04T03:15:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.326+0000 7f24f67fc640 1 -- 172.21.15.39:0/16839284 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24f0012a80 con 0x557237b801b0 2024-05-04T03:15:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.326+0000 7f24f67fc640 1 -- 172.21.15.39:0/16839284 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f000f990 con 0x557237b801b0 2024-05-04T03:15:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.327+0000 7f24fd7fa640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24e00b2930 0x7f24e00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.327+0000 7f24fd7fa640 1 -- 172.21.15.39:0/828429677 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f24f80e26e0 con 0x7f25000750c0 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.327+0000 7f24fffff640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24e00b2930 0x7f24e00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 -- 172.21.15.39:0/16839284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 msgr2=0x557237f21a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 --2- 172.21.15.39:0/16839284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f21a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009b90 tx=0x7f24f00117b0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f24fffff640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24e00b2930 0x7f24e00b4e40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24f80a99b0 tx=0x7f24f8000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 -- 172.21.15.39:0/16839284 shutdown_connections 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 --2- 172.21.15.39:0/16839284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f28ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 --2- 172.21.15.39:0/16839284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f21fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 --2- 172.21.15.39:0/16839284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f21a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 -- 172.21.15.39:0/16839284 >> 172.21.15.39:0/16839284 conn(0x557237b74300 msgr2=0x557237f1d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 -- 172.21.15.39:0/16839284 shutdown_connections 2024-05-04T03:15:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.328+0000 7f250d829740 1 -- 172.21.15.39:0/16839284 wait complete. 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.329+0000 7f250d829740 1 -- start start 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.329+0000 7f250d829740 1 Processor -- start 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f250d829740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f250d829740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f2bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f250d829740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f32be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f250d829740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557237f299d0 con 0x557237b5f4d0 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f250d829740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557237e43370 con 0x557237b572f0 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f250d829740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557237f29b50 con 0x557237b801b0 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f250d829740 10 client.0 _ll_register_callbacks cb 0x557237b97a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:39978/0 (socket says 172.21.15.39:39978) 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 -- 172.21.15.39:0/169053652 learned_addr learned my addr 172.21.15.39:0/169053652 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f32be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f2bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 -- 172.21.15.39:0/169053652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 msgr2=0x557237f32be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f32be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 -- 172.21.15.39:0/169053652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 msgr2=0x557237f2bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f2bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f2504e33640 1 -- 172.21.15.39:0/169053652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557237c64620 con 0x557237b572f0 2024-05-04T03:15:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.330+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f2bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:15:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.331+0000 7f2504e33640 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2b740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009b60 tx=0x7f24f0014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d518a68ab65cb041 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.331+0000 7f24ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:15:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.331+0000 7f24ef7fe640 1 -- 172.21.15.39:0/169053652 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f002e060 con 0x557237b572f0 2024-05-04T03:15:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.331+0000 7f24ef7fe640 1 -- 172.21.15.39:0/169053652 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f24f0004230 con 0x557237b572f0 2024-05-04T03:15:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.331+0000 7f250d829740 1 -- 172.21.15.39:0/169053652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f0012a80 con 0x557237b572f0 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f24ef7fe640 1 -- 172.21.15.39:0/169053652 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24f0004430 con 0x557237b572f0 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f250d829740 10 client.25442 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f250d829740 1 -- 172.21.15.39:0/169053652 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557237f35b10 con 0x557237b572f0 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f24ef7fe640 1 -- 172.21.15.39:0/169053652 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24f000fba0 con 0x557237b572f0 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f24ef7fe640 1 -- 172.21.15.39:0/169053652 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f24f0026040 con 0x557237b572f0 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f250d829740 10 client.25442 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f250d829740 10 client.25442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f250d829740 20 client.25442 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f250d829740 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557237b3ed80 0x557237f38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f250d829740 4 client.25442 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.332+0000 7f250d829740 1 -- 172.21.15.39:0/169053652 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f24f0012a80 con 0x557237b3ed80 2024-05-04T03:15:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.333+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557237b3ed80 0x557237f38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.333+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557237b3ed80 0x557237f38fe0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f8745e9f60611d server_cookie=e5905227b5c5b387 in_seq=0 out_seq=0 2024-05-04T03:15:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.333+0000 7f24ef7fe640 10 client.25442 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.334+0000 7f24ef7fe640 1 -- 172.21.15.39:0/169053652 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f24f0012a80 con 0x557237b3ed80 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.334+0000 7f24ef7fe640 10 client.25442 handle_command_reply: tid=0 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.334+0000 7f250d829740 1 client.25442 shutdown 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 -- 172.21.15.39:0/169053652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 msgr2=0x557237f2b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2b740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009b60 tx=0x7f24f0014040 comp rx=0 tx=0).stop 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 -- 172.21.15.39:0/169053652 shutdown_connections 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557237b3ed80 0x557237f38fe0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f32be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f2bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 --2- 172.21.15.39:0/169053652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2b740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 -- 172.21.15.39:0/169053652 >> 172.21.15.39:0/169053652 conn(0x557237b74300 msgr2=0x557237f1c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f24ef7fe640 0 client.25442 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 -- 172.21.15.39:0/169053652 shutdown_connections 2024-05-04T03:15:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 1 -- 172.21.15.39:0/169053652 wait complete. 2024-05-04T03:15:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.335+0000 7f250d829740 20 client.25442 trim_cache size 0 max 16384 2024-05-04T03:15:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.363+0000 7f250d829740 1 Processor -- start 2024-05-04T03:15:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.363+0000 7f250d829740 1 -- start start 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.363+0000 7f250d829740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.363+0000 7f250d829740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f32b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f250d829740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f3d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f250d829740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24e400e4a0 con 0x557237b5f4d0 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f250d829740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24e400ffe0 con 0x557237b572f0 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f250d829740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24e400eb70 con 0x557237b801b0 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:39990/0 (socket says 172.21.15.39:39990) 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 -- 172.21.15.39:0/1308316864 learned_addr learned my addr 172.21.15.39:0/1308316864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f32b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f3d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 -- 172.21.15.39:0/1308316864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 msgr2=0x557237f3d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f3d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 -- 172.21.15.39:0/1308316864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 msgr2=0x557237f32b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f32b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.364+0000 7f2504e33640 1 -- 172.21.15.39:0/1308316864 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557237f21a20 con 0x557237b572f0 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.365+0000 7f2504e33640 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2ba20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24f00124d0 tx=0x7f24f0011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf218e4c0b876aec server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.365+0000 7f24f5ffb640 1 -- 172.21.15.39:0/1308316864 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f002e070 con 0x557237b572f0 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.365+0000 7f24f5ffb640 1 -- 172.21.15.39:0/1308316864 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24f0009db0 con 0x557237b572f0 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.365+0000 7f24f5ffb640 1 -- 172.21.15.39:0/1308316864 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0038040 con 0x557237b572f0 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 -- 172.21.15.39:0/1308316864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 msgr2=0x557237f2ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2ba20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24f00124d0 tx=0x7f24f0011cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 -- 172.21.15.39:0/1308316864 shutdown_connections 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b801b0 0x557237f3d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b5f4d0 0x557237f32b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 --2- 172.21.15.39:0/1308316864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b572f0 0x557237f2ba20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 -- 172.21.15.39:0/1308316864 >> 172.21.15.39:0/1308316864 conn(0x557237b74300 msgr2=0x557237f2b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 -- 172.21.15.39:0/1308316864 shutdown_connections 2024-05-04T03:15:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.366+0000 7f250d829740 1 -- 172.21.15.39:0/1308316864 wait complete. 2024-05-04T03:15:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.367+0000 7f250d829740 1 -- start start 2024-05-04T03:15:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.367+0000 7f250d829740 1 Processor -- start 2024-05-04T03:15:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f250d829740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b572f0 0x557237f32bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f250d829740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b5f4d0 0x557237f9d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f2504e33640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b572f0 0x557237f32bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f250d829740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b801b0 0x557237f9f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f250d829740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24e400f910 con 0x557237b572f0 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f250d829740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24e400f240 con 0x557237b801b0 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f2504e33640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b572f0 0x557237f32bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55036/0 (socket says 172.21.15.39:55036) 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f250d829740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24e400c480 con 0x557237b5f4d0 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f2504e33640 1 -- 172.21.15.39:0/2195337177 learned_addr learned my addr 172.21.15.39:0/2195337177 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.368+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b5f4d0 0x557237f9d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f2504e33640 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b801b0 0x557237f9f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f250d829740 10 client.0 _ll_register_callbacks cb 0x7f24f0009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f24ff7fe640 1 -- 172.21.15.39:0/2195337177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b801b0 msgr2=0x557237f9f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b801b0 0x557237f9f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f24ff7fe640 1 -- 172.21.15.39:0/2195337177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b572f0 msgr2=0x557237f32bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b572f0 0x557237f32bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f24ff7fe640 1 -- 172.21.15.39:0/2195337177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557237f9ff70 con 0x557237b5f4d0 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f2504e33640 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b572f0 0x557237f32bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f24ff7fe640 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b5f4d0 0x557237f9d1d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f24e802a860 tx=0x7f24e8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d643ff803b6a08eb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.369+0000 7f24ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.370+0000 7f24ef7fe640 1 -- 172.21.15.39:0/2195337177 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e804c070 con 0x557237b5f4d0 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.370+0000 7f24ef7fe640 1 -- 172.21.15.39:0/2195337177 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f24e8030ab0 con 0x557237b5f4d0 2024-05-04T03:15:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.370+0000 7f250d829740 1 -- 172.21.15.39:0/2195337177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f0009db0 con 0x557237b5f4d0 2024-05-04T03:15:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.370+0000 7f24ef7fe640 1 -- 172.21.15.39:0/2195337177 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f24e8037040 con 0x557237b5f4d0 2024-05-04T03:15:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.370+0000 7f250d829740 10 client.25453 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.370+0000 7f250d829740 1 -- 172.21.15.39:0/2195337177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557237fa12b0 con 0x557237b5f4d0 2024-05-04T03:15:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.370+0000 7f24ef7fe640 1 -- 172.21.15.39:0/2195337177 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24f0009db0 con 0x557237b5f4d0 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.370+0000 7f24ef7fe640 1 -- 172.21.15.39:0/2195337177 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f24e80484b0 con 0x557237b5f4d0 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f250d829740 10 client.25453 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f250d829740 10 client.25453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f250d829740 20 client.25453 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f250d829740 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557237b141c0 0x557237fa3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f250d829740 4 client.25453 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f250d829740 1 -- 172.21.15.39:0/2195337177 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557237e941e0 con 0x557237b141c0 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f2504e33640 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557237b141c0 0x557237fa3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f24ef7fe640 10 client.25453 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.371+0000 7f2504e33640 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557237b141c0 0x557237fa3cd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d2013e715b23cd server_cookie=da2c8c15dc9330a4 in_seq=0 out_seq=0 2024-05-04T03:15:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.381+0000 7f24ef7fe640 1 -- 172.21.15.39:0/2195337177 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557237e941e0 con 0x557237b141c0 2024-05-04T03:15:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.381+0000 7f24ef7fe640 10 client.25453 handle_command_reply: tid=0 2024-05-04T03:15:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.381+0000 7f250d829740 1 client.25453 shutdown 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.381+0000 7f250d829740 1 -- 172.21.15.39:0/2195337177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b5f4d0 msgr2=0x557237f9d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.381+0000 7f250d829740 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b5f4d0 0x557237f9d1d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f24e802a860 tx=0x7f24e8005dc0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.381+0000 7f250d829740 1 -- 172.21.15.39:0/2195337177 shutdown_connections 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.381+0000 7f250d829740 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557237b141c0 0x557237fa3cd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.382+0000 7f250d829740 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557237b801b0 0x557237f9f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.382+0000 7f250d829740 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557237b5f4d0 0x557237f9d1d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.382+0000 7f24ef7fe640 0 client.25453 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.382+0000 7f250d829740 1 --2- 172.21.15.39:0/2195337177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557237b572f0 0x557237f32bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.382+0000 7f250d829740 1 -- 172.21.15.39:0/2195337177 >> 172.21.15.39:0/2195337177 conn(0x557237b74300 msgr2=0x557237f201f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.382+0000 7f250d829740 1 -- 172.21.15.39:0/2195337177 shutdown_connections 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.382+0000 7f250d829740 1 -- 172.21.15.39:0/2195337177 wait complete. 2024-05-04T03:15:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.382+0000 7f250d829740 20 client.25453 trim_cache size 0 max 16384 2024-05-04T03:15:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:15:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:15:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:15:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24e00b2930 msgr2=0x7f24e00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24e00b2930 0x7f24e00b4e40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24f80a99b0 tx=0x7f24f8000f40 comp rx=0 tx=0).stop 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000750c0 msgr2=0x7f25001a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000750c0 0x7f25001a08b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24f800eb90 tx=0x7f24f800c6a0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 shutdown_connections 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24e00b2930 0x7f24e00b4e40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25001103d0 0x7f25001a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2500075a10 0x7f25001a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 --2- 172.21.15.39:0/828429677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25000750c0 0x7f25001a08b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 >> 172.21.15.39:0/828429677 conn(0x7f25000fee60 msgr2=0x7f2500103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 shutdown_connections 2024-05-04T03:15:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:04.383+0000 7f2506896640 1 -- 172.21.15.39:0/828429677 wait complete. 2024-05-04T03:15:04.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:04 smithi066 ceph-mon[33264]: pgmap v514: 97 pgs: 97 active+clean; 987 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-05-04T03:15:04.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:04 smithi167 ceph-mon[33614]: pgmap v514: 97 pgs: 97 active+clean; 987 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-05-04T03:15:04.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:04 smithi039 ceph-mon[29715]: pgmap v514: 97 pgs: 97 active+clean; 987 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-05-04T03:15:06.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:06 smithi066 ceph-mon[33264]: pgmap v515: 97 pgs: 97 active+clean; 991 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:06.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:06 smithi167 ceph-mon[33614]: pgmap v515: 97 pgs: 97 active+clean; 991 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:06 smithi039 ceph-mon[29715]: pgmap v515: 97 pgs: 97 active+clean; 991 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 75.0 s, 1287.3 tps, lat 77.566 ms stddev 73.914 2024-05-04T03:15:08.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:08 smithi066 ceph-mon[33264]: pgmap v516: 97 pgs: 97 active+clean; 991 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:15:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:08 smithi167 ceph-mon[33614]: pgmap v516: 97 pgs: 97 active+clean; 991 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:15:08.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:08 smithi039 ceph-mon[29715]: pgmap v516: 97 pgs: 97 active+clean; 991 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:15:10.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:10 smithi066 ceph-mon[33264]: pgmap v517: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:10.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:10 smithi167 ceph-mon[33614]: pgmap v517: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:10.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:10 smithi039 ceph-mon[29715]: pgmap v517: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 80.0 s, 1503.2 tps, lat 66.617 ms stddev 17.505 2024-05-04T03:15:12.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:12 smithi066 ceph-mon[33264]: pgmap v518: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-05-04T03:15:12.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:12 smithi167 ceph-mon[33614]: pgmap v518: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-05-04T03:15:12.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:12 smithi039 ceph-mon[29715]: pgmap v518: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-05-04T03:15:14.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:14 smithi066 ceph-mon[33264]: pgmap v519: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:14 smithi167 ceph-mon[33614]: pgmap v519: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:14 smithi039 ceph-mon[29715]: pgmap v519: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.61k op/s 2024-05-04T03:15:15.693 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-04T03:15:15.694 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-04T03:15:15.694 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:16.097 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:16.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:16 smithi066 ceph-mon[33264]: pgmap v520: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 1.48k op/s 2024-05-04T03:15:16.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:16 smithi039 ceph-mon[29715]: pgmap v520: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 1.48k op/s 2024-05-04T03:15:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:16 smithi167 ceph-mon[33614]: pgmap v520: 97 pgs: 97 active+clean; 995 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 1.48k op/s 2024-05-04T03:15:16.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 85.0 s, 1495.6 tps, lat 66.709 ms stddev 18.193 2024-05-04T03:15:17.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.535+0000 7f7b17ad1640 1 -- 172.21.15.39:0/1083243394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 msgr2=0x7f7b10103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.535+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/1083243394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 0x7f7b10103850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009860 tx=0x7f7b0c02c550 comp rx=0 tx=0).stop 2024-05-04T03:15:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.535+0000 7f7b17ad1640 1 -- 172.21.15.39:0/1083243394 shutdown_connections 2024-05-04T03:15:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.535+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/1083243394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b10108a90 0x7f7b1010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.535+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/1083243394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b10103d90 0x7f7b101041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.535+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/1083243394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 0x7f7b10103850 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.535+0000 7f7b17ad1640 1 -- 172.21.15.39:0/1083243394 >> 172.21.15.39:0/1083243394 conn(0x7f7b100fdd40 msgr2=0x7f7b101001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.536+0000 7f7b17ad1640 1 -- 172.21.15.39:0/1083243394 shutdown_connections 2024-05-04T03:15:17.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.536+0000 7f7b17ad1640 1 -- 172.21.15.39:0/1083243394 wait complete. 2024-05-04T03:15:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b17ad1640 1 Processor -- start 2024-05-04T03:15:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b17ad1640 1 -- start start 2024-05-04T03:15:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b17ad1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 0x7f7b1019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b17ad1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b10103d90 0x7f7b1019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b17ad1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b10108a90 0x7f7b101a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b17ad1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10111830 con 0x7f7b10108a90 2024-05-04T03:15:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b17ad1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101116b0 con 0x7f7b10103d90 2024-05-04T03:15:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b17ad1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101119b0 con 0x7f7b10102b50 2024-05-04T03:15:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.537+0000 7f7b1586d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 0x7f7b1019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1586d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 0x7f7b1019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:42478/0 (socket says 172.21.15.39:42478) 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1586d640 1 -- 172.21.15.39:0/441048888 learned_addr learned my addr 172.21.15.39:0/441048888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1506c640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b10103d90 0x7f7b1019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1606e640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b10108a90 0x7f7b101a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1586d640 1 -- 172.21.15.39:0/441048888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b10103d90 msgr2=0x7f7b1019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1586d640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b10103d90 0x7f7b1019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1586d640 1 -- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b10108a90 msgr2=0x7f7b101a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1586d640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b10108a90 0x7f7b101a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.538+0000 7f7b1586d640 1 -- 172.21.15.39:0/441048888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b101a3fe0 con 0x7f7b10102b50 2024-05-04T03:15:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.539+0000 7f7b1586d640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 0x7f7b1019c4a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009780 tx=0x7f7b0c032f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.539+0000 7f7b06ffd640 1 -- 172.21.15.39:0/441048888 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c041070 con 0x7f7b10102b50 2024-05-04T03:15:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.539+0000 7f7b06ffd640 1 -- 172.21.15.39:0/441048888 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b0c005550 con 0x7f7b10102b50 2024-05-04T03:15:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.539+0000 7f7b06ffd640 1 -- 172.21.15.39:0/441048888 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c0046c0 con 0x7f7b10102b50 2024-05-04T03:15:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.539+0000 7f7b17ad1640 1 -- 172.21.15.39:0/441048888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b101a4270 con 0x7f7b10102b50 2024-05-04T03:15:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.539+0000 7f7b17ad1640 1 -- 172.21.15.39:0/441048888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101a4760 con 0x7f7b10102b50 2024-05-04T03:15:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.543+0000 7f7b06ffd640 1 -- 172.21.15.39:0/441048888 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7b0c044070 con 0x7f7b10102b50 2024-05-04T03:15:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.543+0000 7f7b1e987740 1 Processor -- start 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1e987740 1 -- start start 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f304ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c812f0 0x55e9f304bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f3052ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1e987740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e9f2fb05d0 con 0x55e9f2c812f0 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1e987740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e9f2f69890 con 0x55e9f2c894d0 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1e987740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e9f2fac590 con 0x55e9f2caa1b0 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1606e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f304ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1606e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f304ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:42490/0 (socket says 172.21.15.39:42490) 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b1606e640 1 -- 172.21.15.39:0/4055687063 learned_addr learned my addr 172.21.15.39:0/4055687063 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.544+0000 7f7b06ffd640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7afc0b2970 0x7f7afc0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.545+0000 7f7b06ffd640 1 -- 172.21.15.39:0/441048888 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7b0c1030d0 con 0x7f7b10102b50 2024-05-04T03:15:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.545+0000 7f7b1606e640 1 -- 172.21.15.39:0/4055687063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 msgr2=0x55e9f3052ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:15:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.545+0000 7f7b1606e640 1 --2- 172.21.15.39:0/4055687063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f3052ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.545+0000 7f7b1606e640 1 -- 172.21.15.39:0/4055687063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c812f0 msgr2=0x55e9f304bfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:15:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.545+0000 7f7b1606e640 1 --2- 172.21.15.39:0/4055687063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c812f0 0x55e9f304bfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.545+0000 7f7b1606e640 1 -- 172.21.15.39:0/4055687063 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f2cc2210 con 0x55e9f2caa1b0 2024-05-04T03:15:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.545+0000 7f7b1606e640 1 --2- 172.21.15.39:0/4055687063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f304ba60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b08012390 tx=0x7f7b08012850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc2f5c1f21d78dd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.546+0000 7f7afbfff640 1 -- 172.21.15.39:0/4055687063 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080136c0 con 0x55e9f2caa1b0 2024-05-04T03:15:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.546+0000 7f7b1606e640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7afc0b2970 0x7f7afc0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.546+0000 7f7afbfff640 1 -- 172.21.15.39:0/4055687063 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b08013cf0 con 0x55e9f2caa1b0 2024-05-04T03:15:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.546+0000 7f7afbfff640 1 -- 172.21.15.39:0/4055687063 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0801bd50 con 0x55e9f2caa1b0 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.547+0000 7f7b1606e640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7afc0b2970 0x7f7afc0b4e80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b0801c000 tx=0x7f7b0801c590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.547+0000 7f7b1e987740 1 -- 172.21.15.39:0/4055687063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 msgr2=0x55e9f304ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.547+0000 7f7b1e987740 1 --2- 172.21.15.39:0/4055687063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f304ba60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b08012390 tx=0x7f7b08012850 comp rx=0 tx=0).stop 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.547+0000 7f7b1e987740 1 -- 172.21.15.39:0/4055687063 shutdown_connections 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.548+0000 7f7b1e987740 1 --2- 172.21.15.39:0/4055687063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f3052ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.548+0000 7f7b1e987740 1 --2- 172.21.15.39:0/4055687063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c812f0 0x55e9f304bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.548+0000 7f7b1e987740 1 --2- 172.21.15.39:0/4055687063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f304ba60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.548+0000 7f7b1e987740 1 -- 172.21.15.39:0/4055687063 >> 172.21.15.39:0/4055687063 conn(0x55e9f2c9e300 msgr2=0x55e9f30473a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:17.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.548+0000 7f7b1e987740 1 -- 172.21.15.39:0/4055687063 shutdown_connections 2024-05-04T03:15:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.548+0000 7f7b1e987740 1 -- 172.21.15.39:0/4055687063 wait complete. 2024-05-04T03:15:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.549+0000 7f7b1e987740 1 -- start start 2024-05-04T03:15:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.549+0000 7f7b1e987740 1 Processor -- start 2024-05-04T03:15:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.549+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f3066720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.549+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f3066c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1586d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f3066720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1586d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f3066720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:42504/0 (socket says 172.21.15.39:42504) 2024-05-04T03:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1586d640 1 -- 172.21.15.39:0/1489338021 learned_addr learned my addr 172.21.15.39:0/1489338021 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1e987740 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1506c640 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f3066c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e9f3064940 con 0x55e9f2caa1b0 2024-05-04T03:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e9f2f6d370 con 0x55e9f2c894d0 2024-05-04T03:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e9f3064ac0 con 0x55e9f2c812f0 2024-05-04T03:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1606e640 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1e987740 10 client.0 _ll_register_callbacks cb 0x55e9f2cc1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1586d640 1 -- 172.21.15.39:0/1489338021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 msgr2=0x55e9f3066c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1586d640 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f3066c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1586d640 1 -- 172.21.15.39:0/1489338021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 msgr2=0x55e9f306dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1586d640 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1586d640 1 -- 172.21.15.39:0/1489338021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f2d8e620 con 0x55e9f2c812f0 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.550+0000 7f7b1606e640 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7b1586d640 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f3066720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0c7970 tx=0x7f7b0c0c9e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dab938e801b44d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7af97fa640 1 -- 172.21.15.39:0/1489338021 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c0e4060 con 0x55e9f2c812f0 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b08013cf0 con 0x55e9f2c812f0 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7af97fa640 1 -- 172.21.15.39:0/1489338021 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7b0c0ca960 con 0x55e9f2c812f0 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7af97fa640 1 -- 172.21.15.39:0/1489338021 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b0c0eb020 con 0x55e9f2c812f0 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7b1e987740 10 client.25465 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e9f3064a60 con 0x55e9f2c812f0 2024-05-04T03:15:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.551+0000 7f7af97fa640 1 -- 172.21.15.39:0/1489338021 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b08013cf0 con 0x55e9f2c812f0 2024-05-04T03:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.552+0000 7f7af97fa640 1 -- 172.21.15.39:0/1489338021 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7b0c04e060 con 0x55e9f2c812f0 2024-05-04T03:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.552+0000 7f7b1e987740 10 client.25465 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.552+0000 7f7b1e987740 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.552+0000 7f7b1e987740 20 client.25465 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.552+0000 7f7b1e987740 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e9f2c68d80 0x55e9f3074240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.552+0000 7f7b1e987740 4 client.25465 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.552+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9f2fbe1e0 con 0x55e9f2c68d80 2024-05-04T03:15:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.552+0000 7f7b1506c640 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e9f2c68d80 0x55e9f3074240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.553+0000 7f7af97fa640 10 client.25465 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.553+0000 7f7b1506c640 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e9f2c68d80 0x55e9f3074240 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4d9b92630eb773 server_cookie=ca0c62403dde863d in_seq=0 out_seq=0 2024-05-04T03:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7af97fa640 1 -- 172.21.15.39:0/1489338021 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9f2fbe1e0 con 0x55e9f2c68d80 2024-05-04T03:15:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7af97fa640 10 client.25465 handle_command_reply: tid=0 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7b1e987740 1 client.25465 shutdown 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 msgr2=0x55e9f3066720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7b1e987740 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f3066720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0c7970 tx=0x7f7b0c0c9e70 comp rx=0 tx=0).stop 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 shutdown_connections 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7b1e987740 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e9f2c68d80 0x55e9f3074240 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7b1e987740 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.554+0000 7f7b1e987740 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f3066c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.555+0000 7f7b1e987740 1 --2- 172.21.15.39:0/1489338021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f3066720 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.555+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 >> 172.21.15.39:0/1489338021 conn(0x55e9f2c9e300 msgr2=0x55e9f30463a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.555+0000 7f7af97fa640 0 client.25465 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.555+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 shutdown_connections 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.555+0000 7f7b1e987740 1 -- 172.21.15.39:0/1489338021 wait complete. 2024-05-04T03:15:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.555+0000 7f7b1e987740 20 client.25465 trim_cache size 0 max 16384 2024-05-04T03:15:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.581+0000 7f7b1e987740 1 Processor -- start 2024-05-04T03:15:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.581+0000 7f7b1e987740 1 -- start start 2024-05-04T03:15:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c812f0 0x55e9f30501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c894d0 0x55e9f3046080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f306db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1e987740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101a4bd0 con 0x55e9f2c894d0 2024-05-04T03:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1e987740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101a6710 con 0x55e9f2c812f0 2024-05-04T03:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1e987740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101a52a0 con 0x55e9f2caa1b0 2024-05-04T03:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1506c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c894d0 0x55e9f3046080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1506c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c894d0 0x55e9f3046080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54580/0 (socket says 172.21.15.39:54580) 2024-05-04T03:15:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1506c640 1 -- 172.21.15.39:0/3562600092 learned_addr learned my addr 172.21.15.39:0/3562600092 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1606e640 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f306db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.582+0000 7f7b1586d640 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c812f0 0x55e9f30501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.583+0000 7f7b1506c640 1 -- 172.21.15.39:0/3562600092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 msgr2=0x55e9f306db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.583+0000 7f7b1506c640 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f306db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.583+0000 7f7b1506c640 1 -- 172.21.15.39:0/3562600092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c812f0 msgr2=0x55e9f30501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.583+0000 7f7b1506c640 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c812f0 0x55e9f30501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.583+0000 7f7b1506c640 1 -- 172.21.15.39:0/3562600092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f3047f80 con 0x55e9f2c894d0 2024-05-04T03:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.583+0000 7f7b1506c640 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c894d0 0x55e9f3046080 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009530 tx=0x7f7b00010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c516c96ca1ac1984 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.583+0000 7f7afb7fe640 1 -- 172.21.15.39:0/3562600092 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00022070 con 0x55e9f2c894d0 2024-05-04T03:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.584+0000 7f7afb7fe640 1 -- 172.21.15.39:0/3562600092 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b0001d430 con 0x55e9f2c894d0 2024-05-04T03:15:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.584+0000 7f7afb7fe640 1 -- 172.21.15.39:0/3562600092 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004990 con 0x55e9f2c894d0 2024-05-04T03:15:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 -- 172.21.15.39:0/3562600092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c894d0 msgr2=0x55e9f3046080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c894d0 0x55e9f3046080 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009530 tx=0x7f7b00010800 comp rx=0 tx=0).stop 2024-05-04T03:15:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 -- 172.21.15.39:0/3562600092 shutdown_connections 2024-05-04T03:15:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2caa1b0 0x55e9f306db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2c894d0 0x55e9f3046080 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 --2- 172.21.15.39:0/3562600092 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c812f0 0x55e9f30501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 -- 172.21.15.39:0/3562600092 >> 172.21.15.39:0/3562600092 conn(0x55e9f2c9e300 msgr2=0x55e9f3045970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 -- 172.21.15.39:0/3562600092 shutdown_connections 2024-05-04T03:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.585+0000 7f7b1e987740 1 -- 172.21.15.39:0/3562600092 wait complete. 2024-05-04T03:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.586+0000 7f7b1e987740 1 -- start start 2024-05-04T03:15:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.586+0000 7f7b1e987740 1 Processor -- start 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f306b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f306bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1e987740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1e987740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101a6040 con 0x55e9f2caa1b0 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1e987740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101a5970 con 0x55e9f2c894d0 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1e987740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b100730f0 con 0x55e9f2c812f0 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1586d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f306b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1586d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f306b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:42518/0 (socket says 172.21.15.39:42518) 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1506c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f306bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1586d640 1 -- 172.21.15.39:0/2488375839 learned_addr learned my addr 172.21.15.39:0/2488375839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1e987740 10 client.0 _ll_register_callbacks cb 0x7f7b0c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1606e640 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1586d640 1 -- 172.21.15.39:0/2488375839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 msgr2=0x55e9f306bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1586d640 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f306bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1586d640 1 -- 172.21.15.39:0/2488375839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 msgr2=0x55e9f306ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.587+0000 7f7b1586d640 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7b1586d640 1 -- 172.21.15.39:0/2488375839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f306d020 con 0x55e9f2c812f0 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7b1606e640 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7b1586d640 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f306b640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0ca400 tx=0x7f7b0c0328d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bef276dd4510ae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7af97fa640 1 -- 172.21.15.39:0/2488375839 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c04b040 con 0x55e9f2c812f0 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7af97fa640 1 -- 172.21.15.39:0/2488375839 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7b0c0c9b70 con 0x55e9f2c812f0 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7b1e987740 1 -- 172.21.15.39:0/2488375839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0001d430 con 0x55e9f2c812f0 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7af97fa640 1 -- 172.21.15.39:0/2488375839 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b0c0ceab0 con 0x55e9f2c812f0 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7b1e987740 10 client.25471 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.588+0000 7f7b1e987740 1 -- 172.21.15.39:0/2488375839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e9f306e290 con 0x55e9f2c812f0 2024-05-04T03:15:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.589+0000 7f7af97fa640 1 -- 172.21.15.39:0/2488375839 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b0c0ca700 con 0x55e9f2c812f0 2024-05-04T03:15:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.589+0000 7f7af97fa640 1 -- 172.21.15.39:0/2488375839 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7b0c0c94f0 con 0x55e9f2c812f0 2024-05-04T03:15:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.589+0000 7f7b1e987740 10 client.25471 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.589+0000 7f7b1e987740 10 client.25471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.589+0000 7f7b1e987740 20 client.25471 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.589+0000 7f7b1e987740 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e9f2c3e1c0 0x55e9f30de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.589+0000 7f7b1e987740 4 client.25471 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:15:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.589+0000 7f7b1e987740 1 -- 172.21.15.39:0/2488375839 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7b0001d430 con 0x55e9f2c3e1c0 2024-05-04T03:15:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.590+0000 7f7b1506c640 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e9f2c3e1c0 0x55e9f30de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.590+0000 7f7b1506c640 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e9f2c3e1c0 0x55e9f30de7c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73045599f7104abd server_cookie=4e83e0bb2e1ffe30 in_seq=0 out_seq=0 2024-05-04T03:15:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.590+0000 7f7af97fa640 10 client.25471 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.599+0000 7f7af97fa640 1 -- 172.21.15.39:0/2488375839 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7b0001d430 con 0x55e9f2c3e1c0 2024-05-04T03:15:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.599+0000 7f7af97fa640 10 client.25471 handle_command_reply: tid=0 2024-05-04T03:15:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 client.25471 shutdown 2024-05-04T03:15:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 -- 172.21.15.39:0/2488375839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 msgr2=0x55e9f306b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f306b640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0ca400 tx=0x7f7b0c0328d0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 -- 172.21.15.39:0/2488375839 shutdown_connections 2024-05-04T03:15:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e9f2c3e1c0 0x55e9f30de7c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e9f2caa1b0 0x55e9f306ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e9f2c894d0 0x55e9f306bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 --2- 172.21.15.39:0/2488375839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9f2c812f0 0x55e9f306b640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7b1e987740 1 -- 172.21.15.39:0/2488375839 >> 172.21.15.39:0/2488375839 conn(0x55e9f2c9e300 msgr2=0x55e9f3045970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.600+0000 7f7af97fa640 0 client.25471 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.601+0000 7f7b1e987740 1 -- 172.21.15.39:0/2488375839 shutdown_connections 2024-05-04T03:15:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.601+0000 7f7b1e987740 1 -- 172.21.15.39:0/2488375839 wait complete. 2024-05-04T03:15:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.601+0000 7f7b1e987740 20 client.25471 trim_cache size 0 max 16384 2024-05-04T03:15:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:15:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:15:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:15:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 -- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7afc0b2970 msgr2=0x7f7afc0b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7afc0b2970 0x7f7afc0b4e80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b0801c000 tx=0x7f7b0801c590 comp rx=0 tx=0).stop 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 -- 172.21.15.39:0/441048888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 msgr2=0x7f7b1019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 0x7f7b1019c4a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009780 tx=0x7f7b0c032f50 comp rx=0 tx=0).stop 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 -- 172.21.15.39:0/441048888 shutdown_connections 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7afc0b2970 0x7f7afc0b4e80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b10108a90 0x7f7b101a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b10103d90 0x7f7b1019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 --2- 172.21.15.39:0/441048888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b10102b50 0x7f7b1019c4a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 -- 172.21.15.39:0/441048888 >> 172.21.15.39:0/441048888 conn(0x7f7b100fdd40 msgr2=0x7f7b10104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.603+0000 7f7b17ad1640 1 -- 172.21.15.39:0/441048888 shutdown_connections 2024-05-04T03:15:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:17.604+0000 7f7b17ad1640 1 -- 172.21.15.39:0/441048888 wait complete. 2024-05-04T03:15:18.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:18 smithi066 ceph-mon[33264]: pgmap v521: 97 pgs: 97 active+clean; 995 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-05-04T03:15:18.672 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:18 smithi039 ceph-mon[29715]: pgmap v521: 97 pgs: 97 active+clean; 995 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-05-04T03:15:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:18 smithi167 ceph-mon[33614]: pgmap v521: 97 pgs: 97 active+clean; 995 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.51k op/s 2024-05-04T03:15:20.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:20 smithi039 ceph-mon[29715]: pgmap v522: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:15:20.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:20 smithi066 ceph-mon[33264]: pgmap v522: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:15:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:20 smithi167 ceph-mon[33614]: pgmap v522: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.70k op/s 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 -- 172.21.15.39:0/4148243980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103100 msgr2=0x7fe574103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 --2- 172.21.15.39:0/4148243980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103100 0x7fe574103510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009f90 tx=0x7fe56c00b330 comp rx=0 tx=0).stop 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 -- 172.21.15.39:0/4148243980 shutdown_connections 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 --2- 172.21.15.39:0/4148243980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe574110cb0 0x7fe5741130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 --2- 172.21.15.39:0/4148243980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103100 0x7fe574103510 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 --2- 172.21.15.39:0/4148243980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe574103a50 0x7fe574110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 -- 172.21.15.39:0/4148243980 >> 172.21.15.39:0/4148243980 conn(0x7fe5740fedc0 msgr2=0x7fe5741011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 -- 172.21.15.39:0/4148243980 shutdown_connections 2024-05-04T03:15:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.735+0000 7fe57a494700 1 -- 172.21.15.39:0/4148243980 wait complete. 2024-05-04T03:15:20.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.736+0000 7fe57a494700 1 Processor -- start 2024-05-04T03:15:20.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.736+0000 7fe57a494700 1 -- start start 2024-05-04T03:15:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.737+0000 7fe57a494700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe574103100 0x7fe57419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.737+0000 7fe57a494700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103a50 0x7fe57419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.737+0000 7fe573fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe574103100 0x7fe57419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.737+0000 7fe573fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe574103100 0x7fe57419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54602/0 (socket says 172.21.15.39:54602) 2024-05-04T03:15:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.737+0000 7fe573fff700 1 -- 172.21.15.39:0/1685111101 learned_addr learned my addr 172.21.15.39:0/1685111101 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.738+0000 7fe57a494700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe574110cb0 0x7fe5741a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.738+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe574116560 con 0x7fe574103100 2024-05-04T03:15:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.738+0000 7fe5737fe700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103a50 0x7fe57419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.738+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741163e0 con 0x7fe574110cb0 2024-05-04T03:15:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.738+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741166e0 con 0x7fe574103a50 2024-05-04T03:15:20.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.738+0000 7fe578a31700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe574110cb0 0x7fe5741a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:20.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.739+0000 7fe5737fe700 1 -- 172.21.15.39:0/1685111101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe574110cb0 msgr2=0x7fe5741a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:20.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.739+0000 7fe5737fe700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe574110cb0 0x7fe5741a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.739+0000 7fe5737fe700 1 -- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe574103100 msgr2=0x7fe57419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.739+0000 7fe5737fe700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe574103100 0x7fe57419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.739+0000 7fe5737fe700 1 -- 172.21.15.39:0/1685111101 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5741a1e20 con 0x7fe574103a50 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.739+0000 7fe5737fe700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103a50 0x7fe57419d4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe568005fd0 tx=0x7fe56800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.739+0000 7fe5717fa700 1 -- 172.21.15.39:0/1685111101 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56800cc90 con 0x7fe574103a50 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.739+0000 7fe5717fa700 1 -- 172.21.15.39:0/1685111101 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe56800ce30 con 0x7fe574103a50 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.740+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5741a2110 con 0x7fe574103a50 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.740+0000 7fe5717fa700 1 -- 172.21.15.39:0/1685111101 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568018840 con 0x7fe574103a50 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.740+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5741a9a30 con 0x7fe574103a50 2024-05-04T03:15:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.741+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5740512c0 con 0x7fe574103a50 2024-05-04T03:15:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.743+0000 7fe5717fa700 1 -- 172.21.15.39:0/1685111101 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe5680189e0 con 0x7fe574103a50 2024-05-04T03:15:20.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.744+0000 7fe5717fa700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe55c0b29d0 0x7fe55c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:20.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.744+0000 7fe5717fa700 1 -- 172.21.15.39:0/1685111101 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe5680e24e0 con 0x7fe574103a50 2024-05-04T03:15:20.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.744+0000 7fe573fff700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe55c0b29d0 0x7fe55c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:20.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.745+0000 7fe573fff700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe55c0b29d0 0x7fe55c0b4ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009840 tx=0x7fe56c009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:20.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:20.749+0000 7fe5717fa700 1 -- 172.21.15.39:0/1685111101 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe568054df0 con 0x7fe574103a50 2024-05-04T03:15:21.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.239+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe57419e2b0 con 0x7fe574103a50 2024-05-04T03:15:21.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.240+0000 7fe5717fa700 1 -- 172.21.15.39:0/1685111101 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7fe5680aa1b0 con 0x7fe574103a50 2024-05-04T03:15:21.242 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:15:21.242 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:15:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.244+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe55c0b29d0 msgr2=0x7fe55c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.244+0000 7fe57a494700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe55c0b29d0 0x7fe55c0b4ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009840 tx=0x7fe56c009700 comp rx=0 tx=0).stop 2024-05-04T03:15:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.244+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103a50 msgr2=0x7fe57419d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.244+0000 7fe57a494700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103a50 0x7fe57419d4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe568005fd0 tx=0x7fe56800ee70 comp rx=0 tx=0).stop 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.244+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 shutdown_connections 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.245+0000 7fe57a494700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe55c0b29d0 0x7fe55c0b4ee0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.245+0000 7fe57a494700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe574103100 0x7fe57419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.245+0000 7fe57a494700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe574110cb0 0x7fe5741a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.245+0000 7fe57a494700 1 --2- 172.21.15.39:0/1685111101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe574103a50 0x7fe57419d4f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.245+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 >> 172.21.15.39:0/1685111101 conn(0x7fe5740fedc0 msgr2=0x7fe574101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.245+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 shutdown_connections 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:21.245+0000 7fe57a494700 1 -- 172.21.15.39:0/1685111101 wait complete. 2024-05-04T03:15:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:15:21.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 90.0 s, 1383.8 tps, lat 72.226 ms stddev 30.529 2024-05-04T03:15:21.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:21 smithi167 ceph-mon[33614]: pgmap v523: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 1.61k op/s 2024-05-04T03:15:21.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:21 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1685111101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:21.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:21 smithi039 ceph-mon[29715]: pgmap v523: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 1.61k op/s 2024-05-04T03:15:21.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:21 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1685111101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:22.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:21 smithi066 ceph-mon[33264]: pgmap v523: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 1.61k op/s 2024-05-04T03:15:22.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:21 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1685111101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:22.402 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-05-04T03:15:22.403 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-05-04T03:15:22.821 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:24.388 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:24 smithi039 ceph-mon[29715]: pgmap v524: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-05-04T03:15:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:24 smithi066 ceph-mon[33264]: pgmap v524: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-05-04T03:15:24.629 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:24 smithi167 ceph-mon[33614]: pgmap v524: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 1.73k op/s 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 -- 172.21.15.39:0/672213369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84108760 msgr2=0x7f9a841100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/672213369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84108760 0x7f9a841100c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00b0d0 tx=0x7f9a7c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 -- 172.21.15.39:0/672213369 shutdown_connections 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/672213369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a841034e0 0x7f9a841038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/672213369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84108760 0x7f9a841100c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/672213369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a84107e10 0x7f9a84108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 -- 172.21.15.39:0/672213369 >> 172.21.15.39:0/672213369 conn(0x7f9a840fef30 msgr2=0x7f9a84101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 -- 172.21.15.39:0/672213369 shutdown_connections 2024-05-04T03:15:24.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.908+0000 7f9a8ad86700 1 -- 172.21.15.39:0/672213369 wait complete. 2024-05-04T03:15:24.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.909+0000 7f9a8ad86700 1 Processor -- start 2024-05-04T03:15:24.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.909+0000 7f9a8ad86700 1 -- start start 2024-05-04T03:15:24.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.909+0000 7f9a8ad86700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a841034e0 0x7f9a841123f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.909+0000 7f9a8ad86700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84107e10 0x7f9a841108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.909+0000 7f9a8ad86700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a84108760 0x7f9a84110e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.909+0000 7f9a88b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a841034e0 0x7f9a841123f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.909+0000 7f9a8ad86700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841143f0 con 0x7f9a841034e0 2024-05-04T03:15:24.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a8ad86700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84114270 con 0x7f9a84108760 2024-05-04T03:15:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a8ad86700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84114570 con 0x7f9a84107e10 2024-05-04T03:15:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a83fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84107e10 0x7f9a841108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a88b22700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a841034e0 0x7f9a841123f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54632/0 (socket says 172.21.15.39:54632) 2024-05-04T03:15:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a89323700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a84108760 0x7f9a84110e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a88b22700 1 -- 172.21.15.39:0/3770644387 learned_addr learned my addr 172.21.15.39:0/3770644387 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a83fff700 1 -- 172.21.15.39:0/3770644387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a84108760 msgr2=0x7f9a84110e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a83fff700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a84108760 0x7f9a84110e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a83fff700 1 -- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a841034e0 msgr2=0x7f9a841123f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a83fff700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a841034e0 0x7f9a841123f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.910+0000 7f9a83fff700 1 -- 172.21.15.39:0/3770644387 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a84111670 con 0x7f9a84107e10 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.911+0000 7f9a83fff700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84107e10 0x7f9a841108a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a84076ab0 tx=0x7f9a7000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.911+0000 7f9a81ffb700 1 -- 172.21.15.39:0/3770644387 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70016070 con 0x7f9a84107e10 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.911+0000 7f9a81ffb700 1 -- 172.21.15.39:0/3770644387 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a7000fd20 con 0x7f9a84107e10 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.911+0000 7f9a81ffb700 1 -- 172.21.15.39:0/3770644387 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7000e3b0 con 0x7f9a84107e10 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.911+0000 7f9a8ad86700 1 -- 172.21.15.39:0/3770644387 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a841ad650 con 0x7f9a84107e10 2024-05-04T03:15:24.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.911+0000 7f9a8ad86700 1 -- 172.21.15.39:0/3770644387 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a841adb40 con 0x7f9a84107e10 2024-05-04T03:15:24.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.915+0000 7f9a81ffb700 1 -- 172.21.15.39:0/3770644387 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9a70019070 con 0x7f9a84107e10 2024-05-04T03:15:24.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.916+0000 7f9a81ffb700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9a740b2900 0x7f9a740b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.916+0000 7f9a81ffb700 1 -- 172.21.15.39:0/3770644387 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9a700e1b80 con 0x7f9a84107e10 2024-05-04T03:15:24.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.916+0000 7f9a88b22700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9a740b2900 0x7f9a740b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a9979ab80 1 Processor -- start 2024-05-04T03:15:24.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a9979ab80 1 -- start start 2024-05-04T03:15:24.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a88b22700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9a740b2900 0x7f9a740b4e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009680 tx=0x7f9a78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c75ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c767430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c769d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563f6c67f9f0 con 0x563f6c75cfb0 2024-05-04T03:15:24.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f6c61e950 con 0x563f6c767970 2024-05-04T03:15:24.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.917+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f6c67b9b0 con 0x563f6c75c660 2024-05-04T03:15:24.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c769d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c769d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:60202/0 (socket says 172.21.15.39:60202) 2024-05-04T03:15:24.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 -- 172.21.15.39:0/2657838562 learned_addr learned my addr 172.21.15.39:0/2657838562 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:24.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a88b22700 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c75ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a83fff700 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c767430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 -- 172.21.15.39:0/2657838562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 msgr2=0x563f6c75ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c75ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 -- 172.21.15.39:0/2657838562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 msgr2=0x563f6c767430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c767430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 -- 172.21.15.39:0/2657838562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6c5cca60 con 0x563f6c767970 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a89323700 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c769d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009c30 tx=0x7f9a7c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b9a56def538506 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.918+0000 7f9a6effd700 1 -- 172.21.15.39:0/2657838562 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00bca0 con 0x563f6c767970 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.919+0000 7f9a6effd700 1 -- 172.21.15.39:0/2657838562 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a7c020070 con 0x563f6c767970 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.919+0000 7f9a6effd700 1 -- 172.21.15.39:0/2657838562 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c005650 con 0x563f6c767970 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 -- 172.21.15.39:0/2657838562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 msgr2=0x563f6c769d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c769d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009c30 tx=0x7f9a7c0099e0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 -- 172.21.15.39:0/2657838562 shutdown_connections 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c769d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c75ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/2657838562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c767430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 -- 172.21.15.39:0/2657838562 >> 172.21.15.39:0/2657838562 conn(0x563f6c5dffd0 msgr2=0x563f6c758690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:24.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 -- 172.21.15.39:0/2657838562 shutdown_connections 2024-05-04T03:15:24.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.920+0000 7f9a9979ab80 1 -- 172.21.15.39:0/2657838562 wait complete. 2024-05-04T03:15:24.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.921+0000 7f9a9979ab80 1 -- start start 2024-05-04T03:15:24.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.921+0000 7f9a9979ab80 1 Processor -- start 2024-05-04T03:15:24.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.921+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75c660 0x563f6c764940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.921+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c764e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c767970 0x563f6c7653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563f6c76a740 con 0x563f6c75c660 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f6c5e5dc0 con 0x563f6c75cfb0 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f6c76a8c0 con 0x563f6c767970 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a83fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c764e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a9979ab80 10 client.0 _ll_register_callbacks cb 0x563f6c617ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a83fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c764e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:60206/0 (socket says 172.21.15.39:60206) 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a83fff700 1 -- 172.21.15.39:0/867410063 learned_addr learned my addr 172.21.15.39:0/867410063 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a89323700 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c767970 0x563f6c7653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a88b22700 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75c660 0x563f6c764940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a83fff700 1 -- 172.21.15.39:0/867410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c767970 msgr2=0x563f6c7653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a83fff700 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c767970 0x563f6c7653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a83fff700 1 -- 172.21.15.39:0/867410063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75c660 msgr2=0x563f6c764940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:24.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a83fff700 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75c660 0x563f6c764940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a83fff700 1 -- 172.21.15.39:0/867410063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6c61b820 con 0x563f6c75cfb0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.922+0000 7f9a89323700 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c767970 0x563f6c7653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a83fff700 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c764e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a700df630 tx=0x7f9a70010820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4292d09edf32b433 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a63fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a63fff700 1 -- 172.21.15.39:0/867410063 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70020040 con 0x563f6c75cfb0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a63fff700 1 -- 172.21.15.39:0/867410063 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9a700b1d30 con 0x563f6c75cfb0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a9979ab80 1 -- 172.21.15.39:0/867410063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a7c020070 con 0x563f6c75cfb0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a63fff700 1 -- 172.21.15.39:0/867410063 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9a700a9150 con 0x563f6c75cfb0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a9979ab80 10 client.25481 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a9979ab80 1 -- 172.21.15.39:0/867410063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f6c775370 con 0x563f6c75cfb0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.923+0000 7f9a63fff700 1 -- 172.21.15.39:0/867410063 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a700ad500 con 0x563f6c75cfb0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a63fff700 1 -- 172.21.15.39:0/867410063 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9a700a83b0 con 0x563f6c75cfb0 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a9979ab80 10 client.25481 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:24.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a9979ab80 10 client.25481 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:15:24.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a9979ab80 20 client.25481 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:24.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f6c777900 0x563f6c779d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a9979ab80 4 client.25481 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:15:24.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a9979ab80 1 -- 172.21.15.39:0/867410063 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a7c020070 con 0x563f6c777900 2024-05-04T03:15:24.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a88b22700 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f6c777900 0x563f6c779d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.924+0000 7f9a63fff700 10 client.25481 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:24.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.925+0000 7f9a88b22700 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f6c777900 0x563f6c779d20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf35bf1e666cfe5 server_cookie=2b370f53aac2d843 in_seq=0 out_seq=0 2024-05-04T03:15:24.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a63fff700 1 -- 172.21.15.39:0/867410063 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9a7c020070 con 0x563f6c777900 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a63fff700 10 client.25481 handle_command_reply: tid=0 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 client.25481 shutdown 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 -- 172.21.15.39:0/867410063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 msgr2=0x563f6c764e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c764e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a700df630 tx=0x7f9a70010820 comp rx=0 tx=0).stop 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 -- 172.21.15.39:0/867410063 shutdown_connections 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f6c777900 0x563f6c779d20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75c660 0x563f6c764940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c767970 0x563f6c7653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/867410063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c764e80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a9979ab80 1 -- 172.21.15.39:0/867410063 >> 172.21.15.39:0/867410063 conn(0x563f6c5dffd0 msgr2=0x563f6c75e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:24.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.926+0000 7f9a63fff700 0 client.25481 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:24.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.927+0000 7f9a9979ab80 1 -- 172.21.15.39:0/867410063 shutdown_connections 2024-05-04T03:15:24.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.927+0000 7f9a9979ab80 1 -- 172.21.15.39:0/867410063 wait complete. 2024-05-04T03:15:24.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.927+0000 7f9a9979ab80 20 client.25481 trim_cache size 0 max 16384 2024-05-04T03:15:24.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.963+0000 7f9a9979ab80 1 Processor -- start 2024-05-04T03:15:24.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.963+0000 7f9a9979ab80 1 -- start start 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.963+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c760380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.963+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c766670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.963+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c767970 0x563f6c766bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841b5020 con 0x563f6c767970 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841b6b60 con 0x563f6c75cfb0 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841b56f0 con 0x563f6c75c660 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c767970 0x563f6c766bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c767970 0x563f6c766bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54654/0 (socket says 172.21.15.39:54654) 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 -- 172.21.15.39:0/793976135 learned_addr learned my addr 172.21.15.39:0/793976135 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a83fff700 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c766670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 -- 172.21.15.39:0/793976135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 msgr2=0x563f6c760380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a88b22700 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c760380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c760380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 -- 172.21.15.39:0/793976135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 msgr2=0x563f6c766670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c766670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 -- 172.21.15.39:0/793976135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6c754340 con 0x563f6c767970 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a88b22700 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c760380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a83fff700 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c766670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.964+0000 7f9a89323700 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c767970 0x563f6c766bb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c005bd0 tx=0x7f9a7c00bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3dbf4a42bfdd70 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.965+0000 7f9a6e7fc700 1 -- 172.21.15.39:0/793976135 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c02d070 con 0x563f6c767970 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.965+0000 7f9a6e7fc700 1 -- 172.21.15.39:0/793976135 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a7c0094f0 con 0x563f6c767970 2024-05-04T03:15:24.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.965+0000 7f9a6e7fc700 1 -- 172.21.15.39:0/793976135 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c005730 con 0x563f6c767970 2024-05-04T03:15:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.966+0000 7f9a9979ab80 1 -- 172.21.15.39:0/793976135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c767970 msgr2=0x563f6c766bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.966+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c767970 0x563f6c766bb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c005bd0 tx=0x7f9a7c00bc30 comp rx=0 tx=0).stop 2024-05-04T03:15:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.967+0000 7f9a9979ab80 1 -- 172.21.15.39:0/793976135 shutdown_connections 2024-05-04T03:15:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.967+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c767970 0x563f6c766bb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.967+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c760380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.967+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/793976135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c75cfb0 0x563f6c766670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.967+0000 7f9a9979ab80 1 -- 172.21.15.39:0/793976135 >> 172.21.15.39:0/793976135 conn(0x563f6c5dffd0 msgr2=0x563f6c75ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.967+0000 7f9a9979ab80 1 -- 172.21.15.39:0/793976135 shutdown_connections 2024-05-04T03:15:24.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.967+0000 7f9a9979ab80 1 -- 172.21.15.39:0/793976135 wait complete. 2024-05-04T03:15:24.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.967+0000 7f9a9979ab80 1 -- start start 2024-05-04T03:15:24.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.968+0000 7f9a9979ab80 1 Processor -- start 2024-05-04T03:15:24.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.968+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c770100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c76e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a9979ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c76e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841b6490 con 0x563f6c75cfb0 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841b5dc0 con 0x563f6c767970 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a9979ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841b2fc0 con 0x563f6c75c660 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a9979ab80 10 client.0 _ll_register_callbacks cb 0x563f6c617ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a89323700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c76e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a89323700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c76e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:60228/0 (socket says 172.21.15.39:60228) 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a89323700 1 -- 172.21.15.39:0/3845351995 learned_addr learned my addr 172.21.15.39:0/3845351995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a88b22700 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c770100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a83fff700 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c76e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a89323700 1 -- 172.21.15.39:0/3845351995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 msgr2=0x563f6c770100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a89323700 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c770100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a89323700 1 -- 172.21.15.39:0/3845351995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 msgr2=0x563f6c76e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.969+0000 7f9a89323700 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c76e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.970+0000 7f9a89323700 1 -- 172.21.15.39:0/3845351995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6c76f170 con 0x563f6c767970 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.970+0000 7f9a83fff700 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c76e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.970+0000 7f9a89323700 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c76e9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0052c0 tx=0x7f9a7c003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c33d7645e34d1958 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.970+0000 7f9a63fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.970+0000 7f9a63fff700 1 -- 172.21.15.39:0/3845351995 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c02d030 con 0x563f6c767970 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.970+0000 7f9a63fff700 1 -- 172.21.15.39:0/3845351995 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9a7c00f040 con 0x563f6c767970 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.970+0000 7f9a9979ab80 1 -- 172.21.15.39:0/3845351995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a7c0094f0 con 0x563f6c767970 2024-05-04T03:15:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.970+0000 7f9a63fff700 1 -- 172.21.15.39:0/3845351995 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9a7c039050 con 0x563f6c767970 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a9979ab80 10 client.25484 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a9979ab80 1 -- 172.21.15.39:0/3845351995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f6c7e2e80 con 0x563f6c767970 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a63fff700 1 -- 172.21.15.39:0/3845351995 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a7c0094f0 con 0x563f6c767970 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a63fff700 1 -- 172.21.15.39:0/3845351995 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9a7c011070 con 0x563f6c767970 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a9979ab80 10 client.25484 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a9979ab80 10 client.25484 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a9979ab80 20 client.25484 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f6c7e3050 0x563f6c7e5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.971+0000 7f9a9979ab80 4 client.25484 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.972+0000 7f9a9979ab80 1 -- 172.21.15.39:0/3845351995 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x563f6c68d3d0 con 0x563f6c7e3050 2024-05-04T03:15:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.972+0000 7f9a88b22700 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f6c7e3050 0x563f6c7e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.972+0000 7f9a88b22700 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f6c7e3050 0x563f6c7e5470 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fd5054af56ac3 server_cookie=260793325e83ccc2 in_seq=0 out_seq=0 2024-05-04T03:15:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:24.972+0000 7f9a63fff700 10 client.25484 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:25.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.293+0000 7f9a63fff700 1 -- 172.21.15.39:0/3845351995 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563f6c68d3d0 con 0x563f6c7e3050 2024-05-04T03:15:25.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.293+0000 7f9a63fff700 10 client.25484 handle_command_reply: tid=0 2024-05-04T03:15:25.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.293+0000 7f9a9979ab80 1 client.25484 shutdown 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.293+0000 7f9a9979ab80 1 -- 172.21.15.39:0/3845351995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 msgr2=0x563f6c76e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.293+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c76e9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0052c0 tx=0x7f9a7c003ea0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.293+0000 7f9a9979ab80 1 -- 172.21.15.39:0/3845351995 shutdown_connections 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.293+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f6c7e3050 0x563f6c7e5470 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.294+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6c767970 0x563f6c76e9d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.294+0000 7f9a63fff700 0 client.25484 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.294+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6c75c660 0x563f6c770100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.294+0000 7f9a9979ab80 1 --2- 172.21.15.39:0/3845351995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6c75cfb0 0x563f6c76e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.294+0000 7f9a9979ab80 1 -- 172.21.15.39:0/3845351995 >> 172.21.15.39:0/3845351995 conn(0x563f6c5dffd0 msgr2=0x563f6c75ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.294+0000 7f9a9979ab80 1 -- 172.21.15.39:0/3845351995 shutdown_connections 2024-05-04T03:15:25.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.294+0000 7f9a9979ab80 1 -- 172.21.15.39:0/3845351995 wait complete. 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.294+0000 7f9a9979ab80 20 client.25484 trim_cache size 0 max 16384 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 10, 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout: "92682a91": { 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 10, 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:15:25.297 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 90.0, 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:15:25.298 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:15:25.299 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:15:25.299 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:15:25.299 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:15:25.299 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:15:25.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 -- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9a740b2900 msgr2=0x7f9a740b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:25.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9a740b2900 0x7f9a740b4e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009680 tx=0x7f9a78009260 comp rx=0 tx=0).stop 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 -- 172.21.15.39:0/3770644387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84107e10 msgr2=0x7f9a841108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84107e10 0x7f9a841108a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a84076ab0 tx=0x7f9a7000dc30 comp rx=0 tx=0).stop 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 -- 172.21.15.39:0/3770644387 shutdown_connections 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9a740b2900 0x7f9a740b4e10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a841034e0 0x7f9a841123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a84108760 0x7f9a84110e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 --2- 172.21.15.39:0/3770644387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a84107e10 0x7f9a841108a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 -- 172.21.15.39:0/3770644387 >> 172.21.15.39:0/3770644387 conn(0x7f9a840fef30 msgr2=0x7f9a84107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 -- 172.21.15.39:0/3770644387 shutdown_connections 2024-05-04T03:15:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:25.295+0000 7f9a8ad86700 1 -- 172.21.15.39:0/3770644387 wait complete. 2024-05-04T03:15:26.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:26 smithi039 ceph-mon[29715]: pgmap v525: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 13 MiB/s wr, 1.50k op/s 2024-05-04T03:15:26.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:26 smithi066 ceph-mon[33264]: pgmap v525: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 13 MiB/s wr, 1.50k op/s 2024-05-04T03:15:26.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:26 smithi167 ceph-mon[33614]: pgmap v525: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 13 MiB/s wr, 1.50k op/s 2024-05-04T03:15:26.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 95.0 s, 951.5 tps, lat 68.132 ms stddev 18.747 2024-05-04T03:15:27.797 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '92682a91', quiesce took 12.1 seconds. Will release after: 30.0 2024-05-04T03:15:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:28 smithi066 ceph-mon[33264]: pgmap v526: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-05-04T03:15:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:28 smithi167 ceph-mon[33614]: pgmap v526: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-05-04T03:15:28.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:28 smithi039 ceph-mon[29715]: pgmap v526: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-05-04T03:15:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:30 smithi066 ceph-mon[33264]: pgmap v527: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 1.43k op/s 2024-05-04T03:15:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:30 smithi167 ceph-mon[33614]: pgmap v527: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 1.43k op/s 2024-05-04T03:15:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:30 smithi039 ceph-mon[29715]: pgmap v527: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 1.43k op/s 2024-05-04T03:15:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4112432250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0103d90 msgr2=0x7fe4d01041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4112432250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0103d90 0x7fe4d01041a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009900 tx=0x7fe4c002a410 comp rx=0 tx=0).stop 2024-05-04T03:15:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4112432250 shutdown_connections 2024-05-04T03:15:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4112432250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4d0108a90 0x7fe4d010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4112432250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0103d90 0x7fe4d01041a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4112432250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0102b50 0x7fe4d0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4112432250 >> 172.21.15.39:0/4112432250 conn(0x7fe4d00fdd40 msgr2=0x7fe4d01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4112432250 shutdown_connections 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.723+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4112432250 wait complete. 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d7f9d640 1 Processor -- start 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d7f9d640 1 -- start start 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d7f9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4d0102b50 0x7fe4d019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d7f9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0103d90 0x7fe4d019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d7f9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0108a90 0x7fe4d01a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d7f9d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010e580 con 0x7fe4d0108a90 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d7f9d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010e400 con 0x7fe4d0103d90 2024-05-04T03:15:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d7f9d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d010e700 con 0x7fe4d0102b50 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.724+0000 7fe4d653a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0108a90 0x7fe4d01a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d653a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0108a90 0x7fe4d01a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56954/0 (socket says 172.21.15.39:56954) 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d653a640 1 -- 172.21.15.39:0/4149830917 learned_addr learned my addr 172.21.15.39:0/4149830917 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d5538640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0103d90 0x7fe4d019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4d0102b50 0x7fe4d019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d653a640 1 -- 172.21.15.39:0/4149830917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4d0102b50 msgr2=0x7fe4d019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d653a640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4d0102b50 0x7fe4d019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d653a640 1 -- 172.21.15.39:0/4149830917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0103d90 msgr2=0x7fe4d019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d653a640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0103d90 0x7fe4d019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d653a640 1 -- 172.21.15.39:0/4149830917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d01a1e40 con 0x7fe4d0108a90 2024-05-04T03:15:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d653a640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0108a90 0x7fe4d01a17d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00b970 tx=0x7fe4cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4c6ffd640 1 -- 172.21.15.39:0/4149830917 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc019070 con 0x7fe4d0108a90 2024-05-04T03:15:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4c6ffd640 1 -- 172.21.15.39:0/4149830917 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4cc014440 con 0x7fe4d0108a90 2024-05-04T03:15:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4149830917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d01a2130 con 0x7fe4d0108a90 2024-05-04T03:15:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4c6ffd640 1 -- 172.21.15.39:0/4149830917 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0075d0 con 0x7fe4d0108a90 2024-05-04T03:15:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.725+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4149830917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d01a2680 con 0x7fe4d0108a90 2024-05-04T03:15:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.728+0000 7fe4dee53740 1 Processor -- start 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.728+0000 7fe4c6ffd640 1 -- 172.21.15.39:0/4149830917 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe4cc01c070 con 0x7fe4d0108a90 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.728+0000 7fe4dee53740 1 -- start start 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13906f1b0 0x55d139410a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139410fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 0x55d139417ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4dee53740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d1393755d0 con 0x55d1390462f0 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4dee53740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d13932e890 con 0x55d13906f1b0 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4dee53740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d139371590 con 0x55d13904e4d0 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5d39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139410fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5d39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139410fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56960/0 (socket says 172.21.15.39:56960) 2024-05-04T03:15:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5d39640 1 -- 172.21.15.39:0/2495359414 learned_addr learned my addr 172.21.15.39:0/2495359414 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5538640 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 0x55d139417ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5538640 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13906f1b0 0x55d139410a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4c6ffd640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4bc0b2860 0x7fe4bc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5d39640 1 -- 172.21.15.39:0/2495359414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 msgr2=0x55d139417ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 0x55d139417ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5d39640 1 -- 172.21.15.39:0/2495359414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13906f1b0 msgr2=0x55d139410a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13906f1b0 0x55d139410a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d5d39640 1 -- 172.21.15.39:0/2495359414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d139087210 con 0x55d1390462f0 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4d653a640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4bc0b2860 0x7fe4bc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.729+0000 7fe4c6ffd640 1 -- 172.21.15.39:0/4149830917 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe4cc0e1750 con 0x7fe4d0108a90 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.730+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139410fe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800b750 tx=0x7fe4c800bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a4973ca8fb658d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.730+0000 7fe4bbfff640 1 -- 172.21.15.39:0/2495359414 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c800be60 con 0x55d1390462f0 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.730+0000 7fe4bbfff640 1 -- 172.21.15.39:0/2495359414 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c8015070 con 0x55d1390462f0 2024-05-04T03:15:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.730+0000 7fe4bbfff640 1 -- 172.21.15.39:0/2495359414 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80106a0 con 0x55d1390462f0 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 -- 172.21.15.39:0/2495359414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 msgr2=0x55d139410fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4d653a640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4bc0b2860 0x7fe4bc0b4d70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0df2b0 tx=0x7fe4cc0757b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139410fe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800b750 tx=0x7fe4c800bc10 comp rx=0 tx=0).stop 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 -- 172.21.15.39:0/2495359414 shutdown_connections 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 0x55d139417ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139410fe0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2495359414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13906f1b0 0x55d139410a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 -- 172.21.15.39:0/2495359414 >> 172.21.15.39:0/2495359414 conn(0x55d139063300 msgr2=0x55d13940c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 -- 172.21.15.39:0/2495359414 shutdown_connections 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 -- 172.21.15.39:0/2495359414 wait complete. 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 -- start start 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.731+0000 7fe4dee53740 1 Processor -- start 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d13941a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13904e4d0 0x55d13941ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13906f1b0 0x55d139421b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4dee53740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d139418930 con 0x55d1390462f0 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4dee53740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d139332370 con 0x55d13904e4d0 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4dee53740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d139418ab0 con 0x55d13906f1b0 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4dee53740 10 client.0 _ll_register_callbacks cb 0x55d139086a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d13941a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d13941a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56974/0 (socket says 172.21.15.39:56974) 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 -- 172.21.15.39:0/2752419030 learned_addr learned my addr 172.21.15.39:0/2752419030 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13906f1b0 0x55d139421b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13904e4d0 0x55d13941ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 -- 172.21.15.39:0/2752419030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13906f1b0 msgr2=0x55d139421b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13906f1b0 0x55d139421b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 -- 172.21.15.39:0/2752419030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13904e4d0 msgr2=0x55d13941ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13904e4d0 0x55d13941ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.732+0000 7fe4d5538640 1 -- 172.21.15.39:0/2752419030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d139153620 con 0x55d1390462f0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4d5538640 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d13941a6d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00057c0 tx=0x7fe4c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ff7a11bf52e8b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4b97fa640 1 -- 172.21.15.39:0/2752419030 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0035040 con 0x55d1390462f0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4b97fa640 1 -- 172.21.15.39:0/2752419030 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe4c0004310 con 0x55d1390462f0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 1 -- 172.21.15.39:0/2752419030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c8015070 con 0x55d1390462f0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4b97fa640 1 -- 172.21.15.39:0/2752419030 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe4c0031a70 con 0x55d1390462f0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 10 client.15846 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 1 -- 172.21.15.39:0/2752419030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d139424b10 con 0x55d1390462f0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4b97fa640 1 -- 172.21.15.39:0/2752419030 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c8015070 con 0x55d1390462f0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4b97fa640 1 -- 172.21.15.39:0/2752419030 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe4c0049020 con 0x55d1390462f0 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 10 client.15846 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 20 client.15846 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d13902dd80 0x55d139428190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 4 client.15846 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:15:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.733+0000 7fe4dee53740 1 -- 172.21.15.39:0/2752419030 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1393831e0 con 0x55d13902dd80 2024-05-04T03:15:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.734+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d13902dd80 0x55d139428190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.734+0000 7fe4b97fa640 10 client.15846 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.734+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d13902dd80 0x55d139428190 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af05aeec002a14f0 server_cookie=9834acfef3c5b89 in_seq=0 out_seq=0 2024-05-04T03:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4b97fa640 1 -- 172.21.15.39:0/2752419030 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1393831e0 con 0x55d13902dd80 2024-05-04T03:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4b97fa640 10 client.15846 handle_command_reply: tid=0 2024-05-04T03:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 client.15846 shutdown 2024-05-04T03:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 -- 172.21.15.39:0/2752419030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 msgr2=0x55d13941a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d13941a6d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00057c0 tx=0x7fe4c002a410 comp rx=0 tx=0).stop 2024-05-04T03:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 -- 172.21.15.39:0/2752419030 shutdown_connections 2024-05-04T03:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d13902dd80 0x55d139428190 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13906f1b0 0x55d139421b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4b97fa640 0 client.15846 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13904e4d0 0x55d13941ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 --2- 172.21.15.39:0/2752419030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d13941a6d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 -- 172.21.15.39:0/2752419030 >> 172.21.15.39:0/2752419030 conn(0x55d139063300 msgr2=0x55d13940f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 -- 172.21.15.39:0/2752419030 shutdown_connections 2024-05-04T03:15:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 1 -- 172.21.15.39:0/2752419030 wait complete. 2024-05-04T03:15:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.735+0000 7fe4dee53740 20 client.15846 trim_cache size 0 max 16384 2024-05-04T03:15:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4dee53740 1 Processor -- start 2024-05-04T03:15:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4dee53740 1 -- start start 2024-05-04T03:15:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13902dd80 0x55d13940c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139421b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 0x55d13941a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4dee53740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ab380 con 0x55d1390462f0 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4dee53740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01acec0 con 0x55d13902dd80 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4dee53740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01aba50 con 0x55d13904e4d0 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5d39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139421b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5d39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139421b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56984/0 (socket says 172.21.15.39:56984) 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5d39640 1 -- 172.21.15.39:0/1529169118 learned_addr learned my addr 172.21.15.39:0/1529169118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5538640 1 --2- 172.21.15.39:0/1529169118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13902dd80 0x55d13940c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5d39640 1 -- 172.21.15.39:0/1529169118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 msgr2=0x55d13941a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/1529169118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 0x55d13941a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5d39640 1 -- 172.21.15.39:0/1529169118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13902dd80 msgr2=0x55d13940c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/1529169118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13902dd80 0x55d13940c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.751+0000 7fe4d5d39640 1 -- 172.21.15.39:0/1529169118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d139418290 con 0x55d1390462f0 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.752+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/1529169118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139421b00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80039c0 tx=0x7fe4c8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135ba92f80128bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.752+0000 7fe4bb7fe640 1 -- 172.21.15.39:0/1529169118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c802d070 con 0x55d1390462f0 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.752+0000 7fe4bb7fe640 1 -- 172.21.15.39:0/1529169118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c8028e50 con 0x55d1390462f0 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.752+0000 7fe4bb7fe640 1 -- 172.21.15.39:0/1529169118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8004b70 con 0x55d1390462f0 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 -- 172.21.15.39:0/1529169118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 msgr2=0x55d139421b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1529169118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139421b00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80039c0 tx=0x7fe4c8013040 comp rx=0 tx=0).stop 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 -- 172.21.15.39:0/1529169118 shutdown_connections 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1529169118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13904e4d0 0x55d13941a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1529169118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1390462f0 0x55d139421b00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1529169118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d13902dd80 0x55d13940c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 -- 172.21.15.39:0/1529169118 >> 172.21.15.39:0/1529169118 conn(0x55d139063300 msgr2=0x55d13940bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 -- 172.21.15.39:0/1529169118 shutdown_connections 2024-05-04T03:15:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 -- 172.21.15.39:0/1529169118 wait complete. 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 -- start start 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 Processor -- start 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13902dd80 0x55d13941f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1390462f0 0x55d13941f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.753+0000 7fe4dee53740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d13904e4d0 0x55d139420900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4dee53740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ac7f0 con 0x55d13904e4d0 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4dee53740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01ac120 con 0x55d1390462f0 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4dee53740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01a9360 con 0x55d13902dd80 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4dee53740 10 client.0 _ll_register_callbacks cb 0x7fe4c0031650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d13904e4d0 0x55d139420900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d13904e4d0 0x55d139420900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56996/0 (socket says 172.21.15.39:56996) 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 -- 172.21.15.39:0/1003581259 learned_addr learned my addr 172.21.15.39:0/1003581259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13902dd80 0x55d13941f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 -- 172.21.15.39:0/1003581259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13902dd80 msgr2=0x55d13941f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13902dd80 0x55d13941f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1390462f0 0x55d13941f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 -- 172.21.15.39:0/1003581259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1390462f0 msgr2=0x55d13941f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1390462f0 0x55d13941f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 -- 172.21.15.39:0/1003581259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d139420ec0 con 0x55d13904e4d0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4d5538640 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d13904e4d0 0x55d139420900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe4c002a930 tx=0x7fe4c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8c97ef7c64e049 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4b97fa640 1 -- 172.21.15.39:0/1003581259 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0035070 con 0x55d13904e4d0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4b97fa640 1 -- 172.21.15.39:0/1003581259 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe4c0032bc0 con 0x55d13904e4d0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4dee53740 1 -- 172.21.15.39:0/1003581259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c8028e50 con 0x55d13904e4d0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4b97fa640 1 -- 172.21.15.39:0/1003581259 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe4c0032dc0 con 0x55d13904e4d0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4dee53740 10 client.15852 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.754+0000 7fe4dee53740 1 -- 172.21.15.39:0/1003581259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1394905a0 con 0x55d13904e4d0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4b97fa640 1 -- 172.21.15.39:0/1003581259 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4c8028e50 con 0x55d13904e4d0 2024-05-04T03:15:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4b97fa640 1 -- 172.21.15.39:0/1003581259 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe4c0049040 con 0x55d13904e4d0 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4dee53740 10 client.15852 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4dee53740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4dee53740 20 client.15852 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d13906f1b0 0x55d139492fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4dee53740 4 client.15852 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4dee53740 1 -- 172.21.15.39:0/1003581259 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d13937d050 con 0x55d13906f1b0 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.755+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d13906f1b0 0x55d139492fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.756+0000 7fe4b97fa640 10 client.15852 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.756+0000 7fe4d5d39640 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d13906f1b0 0x55d139492fa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe4f2fbc085518fc server_cookie=38ca1f64d87e7fbf in_seq=0 out_seq=0 2024-05-04T03:15:30.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4b97fa640 1 -- 172.21.15.39:0/1003581259 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d13937d050 con 0x55d13906f1b0 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4b97fa640 10 client.15852 handle_command_reply: tid=0 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 client.15852 shutdown 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 -- 172.21.15.39:0/1003581259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d13904e4d0 msgr2=0x55d139420900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d13904e4d0 0x55d139420900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe4c002a930 tx=0x7fe4c0000bc0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 -- 172.21.15.39:0/1003581259 shutdown_connections 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d13906f1b0 0x55d139492fa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d13904e4d0 0x55d139420900 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1390462f0 0x55d13941f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 --2- 172.21.15.39:0/1003581259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d13902dd80 0x55d13941f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4b97fa640 0 client.15852 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 -- 172.21.15.39:0/1003581259 >> 172.21.15.39:0/1003581259 conn(0x55d139063300 msgr2=0x55d13941a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 -- 172.21.15.39:0/1003581259 shutdown_connections 2024-05-04T03:15:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 1 -- 172.21.15.39:0/1003581259 wait complete. 2024-05-04T03:15:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.765+0000 7fe4dee53740 20 client.15852 trim_cache size 0 max 16384 2024-05-04T03:15:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:15:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:15:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:15:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4bc0b2860 msgr2=0x7fe4bc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4bc0b2860 0x7fe4bc0b4d70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0df2b0 tx=0x7fe4cc0757b0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0108a90 msgr2=0x7fe4d01a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0108a90 0x7fe4d01a17d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00b970 tx=0x7fe4cc00be30 comp rx=0 tx=0).stop 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4149830917 shutdown_connections 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4bc0b2860 0x7fe4bc0b4d70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4d0108a90 0x7fe4d01a17d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4d0103d90 0x7fe4d019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 --2- 172.21.15.39:0/4149830917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4d0102b50 0x7fe4d019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4149830917 >> 172.21.15.39:0/4149830917 conn(0x7fe4d00fdd40 msgr2=0x7fe4d0109f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4149830917 shutdown_connections 2024-05-04T03:15:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:30.766+0000 7fe4d7f9d640 1 -- 172.21.15.39:0/4149830917 wait complete. 2024-05-04T03:15:31.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 100.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:15:32.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:32 smithi066 ceph-mon[33264]: pgmap v528: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 MiB/s wr, 1.04k op/s 2024-05-04T03:15:32.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:32 smithi167 ceph-mon[33614]: pgmap v528: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 MiB/s wr, 1.04k op/s 2024-05-04T03:15:32.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:32 smithi039 ceph-mon[29715]: pgmap v528: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 MiB/s wr, 1.04k op/s 2024-05-04T03:15:34.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:34 smithi066 ceph-mon[33264]: pgmap v529: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 MiB/s wr, 877 op/s 2024-05-04T03:15:34.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:34 smithi167 ceph-mon[33614]: pgmap v529: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 MiB/s wr, 877 op/s 2024-05-04T03:15:34.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:34 smithi039 ceph-mon[29715]: pgmap v529: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 MiB/s wr, 877 op/s 2024-05-04T03:15:36.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:36 smithi066 ceph-mon[33264]: pgmap v530: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 526 op/s 2024-05-04T03:15:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:36 smithi167 ceph-mon[33614]: pgmap v530: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 526 op/s 2024-05-04T03:15:36.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:36 smithi039 ceph-mon[29715]: pgmap v530: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 526 op/s 2024-05-04T03:15:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 105.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:15:38.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:38 smithi066 ceph-mon[33264]: pgmap v531: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.0 MiB/s wr, 369 op/s 2024-05-04T03:15:38.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:38 smithi167 ceph-mon[33614]: pgmap v531: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.0 MiB/s wr, 369 op/s 2024-05-04T03:15:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:38 smithi039 ceph-mon[29715]: pgmap v531: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 3.0 MiB/s wr, 369 op/s 2024-05-04T03:15:40.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:40 smithi066 ceph-mon[33264]: pgmap v532: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-05-04T03:15:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:40 smithi167 ceph-mon[33614]: pgmap v532: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-05-04T03:15:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:40 smithi039 ceph-mon[29715]: pgmap v532: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.8 MiB/s wr, 218 op/s 2024-05-04T03:15:41.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 110.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:15:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:42 smithi066 ceph-mon[33264]: pgmap v533: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 221 KiB/s wr, 39 op/s 2024-05-04T03:15:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:42 smithi167 ceph-mon[33614]: pgmap v533: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 221 KiB/s wr, 39 op/s 2024-05-04T03:15:42.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:42 smithi039 ceph-mon[29715]: pgmap v533: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 221 KiB/s wr, 39 op/s 2024-05-04T03:15:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 -- 172.21.15.39:0/2536710071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103c70 msgr2=0x7f7410104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 --2- 172.21.15.39:0/2536710071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103c70 0x7f7410104970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7408009900 tx=0x7f740802a410 comp rx=0 tx=0).stop 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 -- 172.21.15.39:0/2536710071 shutdown_connections 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 --2- 172.21.15.39:0/2536710071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7410109bb0 0x7f741010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 --2- 172.21.15.39:0/2536710071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7410104eb0 0x7f74101052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 --2- 172.21.15.39:0/2536710071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410103c70 0x7f7410104970 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 -- 172.21.15.39:0/2536710071 >> 172.21.15.39:0/2536710071 conn(0x7f74100fee60 msgr2=0x7f7410101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 -- 172.21.15.39:0/2536710071 shutdown_connections 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.878+0000 7f74178f2640 1 -- 172.21.15.39:0/2536710071 wait complete. 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f74178f2640 1 Processor -- start 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f74178f2640 1 -- start start 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f74178f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7410103c70 0x7f741019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f74178f2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7410104eb0 0x7f741019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f74178f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410109bb0 0x7f74101a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f74178f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f741010f600 con 0x7f7410109bb0 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f74178f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f741010f480 con 0x7f7410103c70 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f74178f2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f741010f780 con 0x7f7410104eb0 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f7415e8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410109bb0 0x7f74101a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f7415e8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410109bb0 0x7f74101a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52148/0 (socket says 172.21.15.39:52148) 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f7415e8f640 1 -- 172.21.15.39:0/94480549 learned_addr learned my addr 172.21.15.39:0/94480549 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.879+0000 7f741568e640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7410103c70 0x7f741019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f7414e8d640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7410104eb0 0x7f741019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f7415e8f640 1 -- 172.21.15.39:0/94480549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7410104eb0 msgr2=0x7f741019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f7415e8f640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7410104eb0 0x7f741019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f7415e8f640 1 -- 172.21.15.39:0/94480549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7410103c70 msgr2=0x7f741019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f7415e8f640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7410103c70 0x7f741019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f7415e8f640 1 -- 172.21.15.39:0/94480549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74101a1dd0 con 0x7f7410109bb0 2024-05-04T03:15:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f7415e8f640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410109bb0 0x7f74101a1760 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f740c00eb10 tx=0x7f740c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f74067fc640 1 -- 172.21.15.39:0/94480549 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c004b20 con 0x7f7410109bb0 2024-05-04T03:15:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f74067fc640 1 -- 172.21.15.39:0/94480549 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f740c004cc0 con 0x7f7410109bb0 2024-05-04T03:15:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f74178f2640 1 -- 172.21.15.39:0/94480549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74101a20c0 con 0x7f7410109bb0 2024-05-04T03:15:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f74067fc640 1 -- 172.21.15.39:0/94480549 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c010690 con 0x7f7410109bb0 2024-05-04T03:15:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.880+0000 7f74178f2640 1 -- 172.21.15.39:0/94480549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74101a2610 con 0x7f7410109bb0 2024-05-04T03:15:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.883+0000 7f741e7a5740 1 Processor -- start 2024-05-04T03:15:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.883+0000 7f74067fc640 1 -- 172.21.15.39:0/94480549 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f740c005020 con 0x7f7410109bb0 2024-05-04T03:15:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.883+0000 7f741e7a5740 1 -- start start 2024-05-04T03:15:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb661b0 0x558bdff07a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb3d2f0 0x558bdff07fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff0eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f741e7a5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558bdfe6c5d0 con 0x558bdfb454d0 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f741e7a5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558bdfe25890 con 0x558bdfb661b0 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f741e7a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558bdfe68590 con 0x558bdfb3d2f0 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff0eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff0eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52158/0 (socket says 172.21.15.39:52158) 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 -- 172.21.15.39:0/2625549256 learned_addr learned my addr 172.21.15.39:0/2625549256 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb661b0 0x558bdff07a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f741568e640 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb3d2f0 0x558bdff07fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f74067fc640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f73fc0b2860 0x7f73fc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 -- 172.21.15.39:0/2625549256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb3d2f0 msgr2=0x558bdff07fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb3d2f0 0x558bdff07fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 -- 172.21.15.39:0/2625549256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb661b0 msgr2=0x558bdff07a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb661b0 0x558bdff07a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 -- 172.21.15.39:0/2625549256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bdfb7e210 con 0x558bdfb454d0 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f741568e640 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb3d2f0 0x558bdff07fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7415e8f640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f73fc0b2860 0x7f73fc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f7414e8d640 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff0eee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x558bdff08c30 tx=0x7f74000097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53f12988ac326de server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.884+0000 7f74067fc640 1 -- 172.21.15.39:0/94480549 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f740c0e2810 con 0x7f7410109bb0 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.885+0000 7f73fb7fe640 1 -- 172.21.15.39:0/2625549256 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000107a0 con 0x558bdfb454d0 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.885+0000 7f73fb7fe640 1 -- 172.21.15.39:0/2625549256 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7400010dd0 con 0x558bdfb454d0 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.885+0000 7f73fb7fe640 1 -- 172.21.15.39:0/2625549256 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740001fd10 con 0x558bdfb454d0 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 -- 172.21.15.39:0/2625549256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 msgr2=0x558bdff0eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff0eee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x558bdff08c30 tx=0x7f74000097b0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f7415e8f640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f73fc0b2860 0x7f73fc0b4d70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f740c0a9b40 tx=0x7f740c074660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 -- 172.21.15.39:0/2625549256 shutdown_connections 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff0eee0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb3d2f0 0x558bdff07fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2625549256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb661b0 0x558bdff07a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 -- 172.21.15.39:0/2625549256 >> 172.21.15.39:0/2625549256 conn(0x558bdfb5a300 msgr2=0x558bdff033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 -- 172.21.15.39:0/2625549256 shutdown_connections 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 -- 172.21.15.39:0/2625549256 wait complete. 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 -- start start 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.886+0000 7f741e7a5740 1 Processor -- start 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb3d2f0 0x558bdff11790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb454d0 0x558bdff11cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb661b0 0x558bdff18bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f741e7a5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558bdff0f930 con 0x558bdfb3d2f0 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f741e7a5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558bdfe29370 con 0x558bdfb454d0 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f741e7a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558bdff0fab0 con 0x558bdfb661b0 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f741e7a5740 10 client.0 _ll_register_callbacks cb 0x558bdfb7da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb3d2f0 0x558bdff11790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb3d2f0 0x558bdff11790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52166/0 (socket says 172.21.15.39:52166) 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 -- 172.21.15.39:0/2592337534 learned_addr learned my addr 172.21.15.39:0/2592337534 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb661b0 0x558bdff18bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f741568e640 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb454d0 0x558bdff11cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 -- 172.21.15.39:0/2592337534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb661b0 msgr2=0x558bdff18bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb661b0 0x558bdff18bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 -- 172.21.15.39:0/2592337534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb454d0 msgr2=0x558bdff11cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb454d0 0x558bdff11cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 -- 172.21.15.39:0/2592337534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bdfc4a620 con 0x558bdfb3d2f0 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f7414e8d640 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb3d2f0 0x558bdff11790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f740001f800 tx=0x7f7400005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39072a4ba6b7dbc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.887+0000 7f73f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f73f8ff9640 1 -- 172.21.15.39:0/2592337534 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400010bb0 con 0x558bdfb3d2f0 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f73f8ff9640 1 -- 172.21.15.39:0/2592337534 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f740001fa90 con 0x558bdfb3d2f0 2024-05-04T03:15:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 1 -- 172.21.15.39:0/2592337534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7400010dd0 con 0x558bdfb3d2f0 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f73f8ff9640 1 -- 172.21.15.39:0/2592337534 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7400034050 con 0x558bdfb3d2f0 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 10 client.15864 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 1 -- 172.21.15.39:0/2592337534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558bdff1baf0 con 0x558bdfb3d2f0 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f73f8ff9640 1 -- 172.21.15.39:0/2592337534 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7400010dd0 con 0x558bdfb3d2f0 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f73f8ff9640 1 -- 172.21.15.39:0/2592337534 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f74000046c0 con 0x558bdfb3d2f0 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 10 client.15864 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 20 client.15864 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558bdfb24d80 0x558bdff1eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 4 client.15864 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.888+0000 7f741e7a5740 1 -- 172.21.15.39:0/2592337534 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bdfe7a1e0 con 0x558bdfb24d80 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.889+0000 7f741568e640 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558bdfb24d80 0x558bdff1eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.889+0000 7f73f8ff9640 10 client.15864 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.889+0000 7f741568e640 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558bdfb24d80 0x558bdff1eee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476d3dbe8c2da46e server_cookie=cd403aa53296c5b8 in_seq=0 out_seq=0 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f73f8ff9640 1 -- 172.21.15.39:0/2592337534 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558bdfe7a1e0 con 0x558bdfb24d80 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f73f8ff9640 10 client.15864 handle_command_reply: tid=0 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 client.15864 shutdown 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 -- 172.21.15.39:0/2592337534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb3d2f0 msgr2=0x558bdff11790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb3d2f0 0x558bdff11790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f740001f800 tx=0x7f7400005fd0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 -- 172.21.15.39:0/2592337534 shutdown_connections 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558bdfb24d80 0x558bdff1eee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb661b0 0x558bdff18bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f73f8ff9640 0 client.15864 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb454d0 0x558bdff11cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 --2- 172.21.15.39:0/2592337534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb3d2f0 0x558bdff11790 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 -- 172.21.15.39:0/2592337534 >> 172.21.15.39:0/2592337534 conn(0x558bdfb5a300 msgr2=0x558bdff06300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 -- 172.21.15.39:0/2592337534 shutdown_connections 2024-05-04T03:15:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 1 -- 172.21.15.39:0/2592337534 wait complete. 2024-05-04T03:15:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.890+0000 7f741e7a5740 20 client.15864 trim_cache size 0 max 16384 2024-05-04T03:15:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741e7a5740 1 Processor -- start 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741e7a5740 1 -- start start 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb24d80 0x558bdff0c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 0x558bdff11a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff18b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741e7a5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ab2d0 con 0x558bdfb454d0 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741e7a5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ace10 con 0x558bdfb3d2f0 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741e7a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ab9a0 con 0x558bdfb24d80 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff18b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff18b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52178/0 (socket says 172.21.15.39:52178) 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 -- 172.21.15.39:0/750614131 learned_addr learned my addr 172.21.15.39:0/750614131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f741568e640 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 0x558bdff11a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb24d80 0x558bdff0c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 -- 172.21.15.39:0/750614131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb24d80 msgr2=0x558bdff0c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb24d80 0x558bdff0c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 -- 172.21.15.39:0/750614131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 msgr2=0x558bdff11a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 0x558bdff11a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.906+0000 7f7414e8d640 1 -- 172.21.15.39:0/750614131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bdff12830 con 0x558bdfb454d0 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.907+0000 7f7414e8d640 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff18b50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f74000095b0 tx=0x7f740001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd72a98eafeca3fa server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.907+0000 7f73faffd640 1 -- 172.21.15.39:0/750614131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400009e80 con 0x558bdfb454d0 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.907+0000 7f73faffd640 1 -- 172.21.15.39:0/750614131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7400004dd0 con 0x558bdfb454d0 2024-05-04T03:15:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.907+0000 7f73faffd640 1 -- 172.21.15.39:0/750614131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740001fa90 con 0x558bdfb454d0 2024-05-04T03:15:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 -- 172.21.15.39:0/750614131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 msgr2=0x558bdff18b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff18b50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f74000095b0 tx=0x7f740001b040 comp rx=0 tx=0).stop 2024-05-04T03:15:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 -- 172.21.15.39:0/750614131 shutdown_connections 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb454d0 0x558bdff18b50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 0x558bdff11a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 --2- 172.21.15.39:0/750614131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb24d80 0x558bdff0c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 -- 172.21.15.39:0/750614131 >> 172.21.15.39:0/750614131 conn(0x558bdfb5a300 msgr2=0x558bdff11360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 -- 172.21.15.39:0/750614131 shutdown_connections 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 -- 172.21.15.39:0/750614131 wait complete. 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 -- start start 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 Processor -- start 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.908+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb24d80 0x558bdff18840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 0x558bdff18d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f741e7a5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb454d0 0x558bdff85630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f741e7a5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ac740 con 0x558bdfb24d80 2024-05-04T03:15:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f741e7a5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ac070 con 0x558bdfb3d2f0 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f741e7a5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74101a92b0 con 0x558bdfb454d0 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f741e7a5740 10 client.0 _ll_register_callbacks cb 0x7f7400009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb24d80 0x558bdff18840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb24d80 0x558bdff18840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52186/0 (socket says 172.21.15.39:52186) 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 -- 172.21.15.39:0/3168678031 learned_addr learned my addr 172.21.15.39:0/3168678031 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f741568e640 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 0x558bdff18d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 -- 172.21.15.39:0/3168678031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb454d0 msgr2=0x558bdff85630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb454d0 0x558bdff85630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 -- 172.21.15.39:0/3168678031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 msgr2=0x558bdff18d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 0x558bdff18d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 -- 172.21.15.39:0/3168678031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bdff85c20 con 0x558bdfb24d80 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f7414e8d640 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb24d80 0x558bdff18840 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7400009f80 tx=0x7f7400009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff174920a768285 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f73f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f73f8ff9640 1 -- 172.21.15.39:0/3168678031 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740003c020 con 0x558bdfb24d80 2024-05-04T03:15:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f73f8ff9640 1 -- 172.21.15.39:0/3168678031 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7400005180 con 0x558bdfb24d80 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f741e7a5740 1 -- 172.21.15.39:0/3168678031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7400004dd0 con 0x558bdfb24d80 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.909+0000 7f73f8ff9640 1 -- 172.21.15.39:0/3168678031 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f74000109b0 con 0x558bdfb24d80 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741e7a5740 10 client.15870 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741e7a5740 1 -- 172.21.15.39:0/3168678031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558bdff86eb0 con 0x558bdfb24d80 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f73f8ff9640 1 -- 172.21.15.39:0/3168678031 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7400004dd0 con 0x558bdfb24d80 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f73f8ff9640 1 -- 172.21.15.39:0/3168678031 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f74000211e0 con 0x558bdfb24d80 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741e7a5740 10 client.15870 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741e7a5740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741e7a5740 20 client.15870 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741e7a5740 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558bdfb661b0 0x558bdff89890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741e7a5740 4 client.15870 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741e7a5740 1 -- 172.21.15.39:0/3168678031 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558bdfe74050 con 0x558bdfb661b0 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.910+0000 7f741568e640 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558bdfb661b0 0x558bdff89890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.911+0000 7f73f8ff9640 10 client.15870 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.911+0000 7f741568e640 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558bdfb661b0 0x558bdff89890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3ebdfe395d3c47 server_cookie=2d78a517615b713c in_seq=0 out_seq=0 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f73f8ff9640 1 -- 172.21.15.39:0/3168678031 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558bdfe74050 con 0x558bdfb661b0 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f73f8ff9640 10 client.15870 handle_command_reply: tid=0 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 client.15870 shutdown 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 -- 172.21.15.39:0/3168678031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb24d80 msgr2=0x558bdff18840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb24d80 0x558bdff18840 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7400009f80 tx=0x7f7400009fb0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 -- 172.21.15.39:0/3168678031 shutdown_connections 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558bdfb661b0 0x558bdff89890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558bdfb454d0 0x558bdff85630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f73f8ff9640 0 client.15870 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558bdfb3d2f0 0x558bdff18d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 --2- 172.21.15.39:0/3168678031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558bdfb24d80 0x558bdff18840 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 -- 172.21.15.39:0/3168678031 >> 172.21.15.39:0/3168678031 conn(0x558bdfb5a300 msgr2=0x558bdff236d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 -- 172.21.15.39:0/3168678031 shutdown_connections 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 1 -- 172.21.15.39:0/3168678031 wait complete. 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.920+0000 7f741e7a5740 20 client.15870 trim_cache size 0 max 16384 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:15:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:15:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 -- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f73fc0b2860 msgr2=0x7f73fc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f73fc0b2860 0x7f73fc0b4d70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f740c0a9b40 tx=0x7f740c074660 comp rx=0 tx=0).stop 2024-05-04T03:15:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 -- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410109bb0 msgr2=0x7f74101a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410109bb0 0x7f74101a1760 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f740c00eb10 tx=0x7f740c00c2d0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 -- 172.21.15.39:0/94480549 shutdown_connections 2024-05-04T03:15:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f73fc0b2860 0x7f73fc0b4d70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7410109bb0 0x7f74101a1760 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7410104eb0 0x7f741019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 --2- 172.21.15.39:0/94480549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7410103c70 0x7f741019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 -- 172.21.15.39:0/94480549 >> 172.21.15.39:0/94480549 conn(0x7f74100fee60 msgr2=0x7f7410105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 -- 172.21.15.39:0/94480549 shutdown_connections 2024-05-04T03:15:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:43.921+0000 7f74178f2640 1 -- 172.21.15.39:0/94480549 wait complete. 2024-05-04T03:15:44.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:44 smithi066 ceph-mon[33264]: pgmap v534: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 219 KiB/s wr, 38 op/s 2024-05-04T03:15:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:44 smithi167 ceph-mon[33614]: pgmap v534: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 219 KiB/s wr, 38 op/s 2024-05-04T03:15:44.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:44 smithi039 ceph-mon[29715]: pgmap v534: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 219 KiB/s wr, 38 op/s 2024-05-04T03:15:46.494 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:46 smithi039 ceph-mon[29715]: pgmap v535: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 216 KiB/s wr, 36 op/s 2024-05-04T03:15:46.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:46 smithi066 ceph-mon[33264]: pgmap v535: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 216 KiB/s wr, 36 op/s 2024-05-04T03:15:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:46 smithi167 ceph-mon[33614]: pgmap v535: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 216 KiB/s wr, 36 op/s 2024-05-04T03:15:46.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 115.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:15:48.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:48 smithi066 ceph-mon[33264]: pgmap v536: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 223 KiB/s wr, 39 op/s 2024-05-04T03:15:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:48 smithi167 ceph-mon[33614]: pgmap v536: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 223 KiB/s wr, 39 op/s 2024-05-04T03:15:48.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:48 smithi039 ceph-mon[29715]: pgmap v536: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 223 KiB/s wr, 39 op/s 2024-05-04T03:15:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:50 smithi066 ceph-mon[33264]: pgmap v537: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 229 KiB/s wr, 40 op/s 2024-05-04T03:15:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:50 smithi167 ceph-mon[33614]: pgmap v537: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 229 KiB/s wr, 40 op/s 2024-05-04T03:15:50.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:50 smithi039 ceph-mon[29715]: pgmap v537: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 229 KiB/s wr, 40 op/s 2024-05-04T03:15:51.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:51 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:51.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:51 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:51.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:51 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:51.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 120.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:15:52.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:52 smithi066 ceph-mon[33264]: pgmap v538: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 215 KiB/s wr, 38 op/s 2024-05-04T03:15:52.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:52.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:52.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:15:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:52 smithi167 ceph-mon[33614]: pgmap v538: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 215 KiB/s wr, 38 op/s 2024-05-04T03:15:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:15:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:52 smithi039 ceph-mon[29715]: pgmap v538: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 215 KiB/s wr, 38 op/s 2024-05-04T03:15:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:15:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:15:53.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:53 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-04T03:15:53.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:53 smithi066 ceph-mon[33264]: Unable to set osd_memory_target on smithi039 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-04T03:15:53.684 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:53 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-04T03:15:53.947 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:53 smithi167 ceph-mon[33614]: Unable to set osd_memory_target on smithi039 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-04T03:15:53.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:53 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-04T03:15:53.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:53 smithi039 ceph-mon[29715]: Unable to set osd_memory_target on smithi039 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-04T03:15:54.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:54 smithi066 ceph-mon[33264]: pgmap v539: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 197 KiB/s wr, 35 op/s 2024-05-04T03:15:54.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:54 smithi167 ceph-mon[33614]: pgmap v539: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 197 KiB/s wr, 35 op/s 2024-05-04T03:15:54.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:54 smithi039 ceph-mon[29715]: pgmap v539: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 197 KiB/s wr, 35 op/s 2024-05-04T03:15:56.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:56 smithi066 ceph-mon[33264]: pgmap v540: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 192 KiB/s wr, 33 op/s 2024-05-04T03:15:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:56 smithi167 ceph-mon[33614]: pgmap v540: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 192 KiB/s wr, 33 op/s 2024-05-04T03:15:56.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:56 smithi039 ceph-mon[29715]: pgmap v540: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 192 KiB/s wr, 33 op/s 2024-05-04T03:15:56.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 125.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:15:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.032+0000 7fd226af9640 1 -- 172.21.15.39:0/89996617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd220104380 msgr2=0x7fd2201067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.032+0000 7fd226af9640 1 --2- 172.21.15.39:0/89996617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd220104380 0x7fd2201067a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd214009900 tx=0x7fd21402a410 comp rx=0 tx=0).stop 2024-05-04T03:15:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 -- 172.21.15.39:0/89996617 shutdown_connections 2024-05-04T03:15:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 --2- 172.21.15.39:0/89996617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2201098e0 0x7fd22010bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 --2- 172.21.15.39:0/89996617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd220106d70 0x7fd220109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 --2- 172.21.15.39:0/89996617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd220104380 0x7fd2201067a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 -- 172.21.15.39:0/89996617 >> 172.21.15.39:0/89996617 conn(0x7fd2200fdd40 msgr2=0x7fd2201001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 -- 172.21.15.39:0/89996617 shutdown_connections 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 -- 172.21.15.39:0/89996617 wait complete. 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 Processor -- start 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.033+0000 7fd226af9640 1 -- start start 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd226af9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd220104380 0x7fd220103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd226af9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd220106d70 0x7fd220102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd226af9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2201098e0 0x7fd2201027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd226af9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd22010f5f0 con 0x7fd2201098e0 2024-05-04T03:15:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd226af9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd22010f470 con 0x7fd220106d70 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd226af9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd22010f770 con 0x7fd220104380 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2201098e0 0x7fd2201027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2201098e0 0x7fd2201027d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38486/0 (socket says 172.21.15.39:38486) 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 -- 172.21.15.39:0/491118316 learned_addr learned my addr 172.21.15.39:0/491118316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd21ffff640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd220106d70 0x7fd220102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd224895640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd220104380 0x7fd220103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 -- 172.21.15.39:0/491118316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd220104380 msgr2=0x7fd220103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd220104380 0x7fd220103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 -- 172.21.15.39:0/491118316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd220106d70 msgr2=0x7fd220102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd220106d70 0x7fd220102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 -- 172.21.15.39:0/491118316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd220102f00 con 0x7fd2201098e0 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd225096640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2201098e0 0x7fd2201027d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd21800eb10 tx=0x7fd21800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.034+0000 7fd21dffb640 1 -- 172.21.15.39:0/491118316 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218004b20 con 0x7fd2201098e0 2024-05-04T03:15:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.035+0000 7fd21dffb640 1 -- 172.21.15.39:0/491118316 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd218004cc0 con 0x7fd2201098e0 2024-05-04T03:15:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.035+0000 7fd226af9640 1 -- 172.21.15.39:0/491118316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2201a4370 con 0x7fd2201098e0 2024-05-04T03:15:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.035+0000 7fd21dffb640 1 -- 172.21.15.39:0/491118316 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218010690 con 0x7fd2201098e0 2024-05-04T03:15:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.035+0000 7fd226af9640 1 -- 172.21.15.39:0/491118316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2201a4830 con 0x7fd2201098e0 2024-05-04T03:15:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.037+0000 7fd21dffb640 1 -- 172.21.15.39:0/491118316 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd218005020 con 0x7fd2201098e0 2024-05-04T03:15:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.037+0000 7fd22d9af740 1 Processor -- start 2024-05-04T03:15:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.037+0000 7fd22d9af740 1 -- start start 2024-05-04T03:15:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e251b0 0x55f4821c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e044d0 0x55f4821cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd22d9af740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f48212b5d0 con 0x55f481e044d0 2024-05-04T03:15:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd22d9af740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f4820e4890 con 0x55f481dfc2f0 2024-05-04T03:15:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd22d9af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f482127590 con 0x55f481e251b0 2024-05-04T03:15:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e044d0 0x55f4821cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd224895640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e044d0 0x55f4821cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38492/0 (socket says 172.21.15.39:38492) 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 -- 172.21.15.39:0/228601814 learned_addr learned my addr 172.21.15.39:0/228601814 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e251b0 0x55f4821c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21dffb640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd2100b27e0 0x7fd2100b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 -- 172.21.15.39:0/228601814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e251b0 msgr2=0x55f4821c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e251b0 0x55f4821c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 -- 172.21.15.39:0/228601814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 msgr2=0x55f4821c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 -- 172.21.15.39:0/228601814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f481e3d210 con 0x55f481e044d0 2024-05-04T03:15:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd224895640 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:15:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd225096640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd2100b27e0 0x7fd2100b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21dffb640 1 -- 172.21.15.39:0/491118316 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd2180e2760 con 0x7fd2201098e0 2024-05-04T03:15:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.038+0000 7fd21ffff640 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e044d0 0x55f4821cdee0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd20c005bc0 tx=0x7fd20c009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d79b5c24d337ecc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.039+0000 7fd20affd640 1 -- 172.21.15.39:0/228601814 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c004030 con 0x55f481e044d0 2024-05-04T03:15:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.039+0000 7fd20affd640 1 -- 172.21.15.39:0/228601814 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd20c005020 con 0x55f481e044d0 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.039+0000 7fd20affd640 1 -- 172.21.15.39:0/228601814 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c004c60 con 0x55f481e044d0 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 -- 172.21.15.39:0/228601814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e044d0 msgr2=0x55f4821cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e044d0 0x55f4821cdee0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd20c005bc0 tx=0x7fd20c009a00 comp rx=0 tx=0).stop 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd225096640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd2100b27e0 0x7fd2100b4cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd2180e02c0 tx=0x7fd218099040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 -- 172.21.15.39:0/228601814 shutdown_connections 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e044d0 0x55f4821cdee0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 --2- 172.21.15.39:0/228601814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e251b0 0x55f4821c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 -- 172.21.15.39:0/228601814 >> 172.21.15.39:0/228601814 conn(0x55f481e19300 msgr2=0x55f4821c23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 -- 172.21.15.39:0/228601814 shutdown_connections 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 -- 172.21.15.39:0/228601814 wait complete. 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 -- start start 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.040+0000 7fd22d9af740 1 Processor -- start 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e251b0 0x55f4821d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd22d9af740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f4821ce930 con 0x55f481e251b0 2024-05-04T03:15:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd22d9af740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f4820e8370 con 0x55f481dfc2f0 2024-05-04T03:15:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd22d9af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f4821ceab0 con 0x55f481e044d0 2024-05-04T03:15:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd22d9af740 10 client.0 _ll_register_callbacks cb 0x55f481e3ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e251b0 0x55f4821d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e251b0 0x55f4821d7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38494/0 (socket says 172.21.15.39:38494) 2024-05-04T03:15:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd21ffff640 1 -- 172.21.15.39:0/814805361 learned_addr learned my addr 172.21.15.39:0/814805361 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd224895640 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.041+0000 7fd21ffff640 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd224895640 1 -- 172.21.15.39:0/814805361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 msgr2=0x55f4821d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd224895640 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd224895640 1 -- 172.21.15.39:0/814805361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e251b0 msgr2=0x55f4821d7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd224895640 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e251b0 0x55f4821d7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd224895640 1 -- 172.21.15.39:0/814805361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f481f09620 con 0x55f481e044d0 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd21ffff640 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e251b0 0x55f4821d7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd224895640 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821d0be0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd214009230 tx=0x7fd214031870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c1f3ef3fe77c75 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd1fffff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd1fffff640 1 -- 172.21.15.39:0/814805361 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214035040 con 0x55f481e044d0 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd1fffff640 1 -- 172.21.15.39:0/814805361 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd214004240 con 0x55f481e044d0 2024-05-04T03:15:57.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.042+0000 7fd22d9af740 1 -- 172.21.15.39:0/814805361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20c005020 con 0x55f481e044d0 2024-05-04T03:15:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.043+0000 7fd1fffff640 1 -- 172.21.15.39:0/814805361 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd214004440 con 0x55f481e044d0 2024-05-04T03:15:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.043+0000 7fd22d9af740 10 client.25531 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.043+0000 7fd22d9af740 1 -- 172.21.15.39:0/814805361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4821daa70 con 0x55f481e044d0 2024-05-04T03:15:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.043+0000 7fd1fffff640 1 -- 172.21.15.39:0/814805361 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd20c005020 con 0x55f481e044d0 2024-05-04T03:15:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.044+0000 7fd1fffff640 1 -- 172.21.15.39:0/814805361 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd21403c5d0 con 0x55f481e044d0 2024-05-04T03:15:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.044+0000 7fd22d9af740 10 client.25531 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.044+0000 7fd22d9af740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.044+0000 7fd22d9af740 20 client.25531 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.044+0000 7fd22d9af740 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f481de3d80 0x55f4821de620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.044+0000 7fd22d9af740 4 client.25531 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:15:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.044+0000 7fd22d9af740 1 -- 172.21.15.39:0/814805361 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4821391e0 con 0x55f481de3d80 2024-05-04T03:15:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.044+0000 7fd21ffff640 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f481de3d80 0x55f4821de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.045+0000 7fd1fffff640 10 client.25531 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.045+0000 7fd21ffff640 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f481de3d80 0x55f4821de620 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc574e0c9e310bbc server_cookie=57fd04db68a11acd in_seq=0 out_seq=0 2024-05-04T03:15:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.045+0000 7fd1fffff640 1 -- 172.21.15.39:0/814805361 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4821391e0 con 0x55f481de3d80 2024-05-04T03:15:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.045+0000 7fd1fffff640 10 client.25531 handle_command_reply: tid=0 2024-05-04T03:15:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.045+0000 7fd22d9af740 1 client.25531 shutdown 2024-05-04T03:15:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 -- 172.21.15.39:0/814805361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 msgr2=0x55f4821d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821d0be0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd214009230 tx=0x7fd214031870 comp rx=0 tx=0).stop 2024-05-04T03:15:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 -- 172.21.15.39:0/814805361 shutdown_connections 2024-05-04T03:15:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f481de3d80 0x55f4821de620 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481e251b0 0x55f4821d7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd1fffff640 0 client.25531 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821d0be0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 --2- 172.21.15.39:0/814805361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 -- 172.21.15.39:0/814805361 >> 172.21.15.39:0/814805361 conn(0x55f481e19300 msgr2=0x55f4821c5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 -- 172.21.15.39:0/814805361 shutdown_connections 2024-05-04T03:15:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 1 -- 172.21.15.39:0/814805361 wait complete. 2024-05-04T03:15:57.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.046+0000 7fd22d9af740 20 client.25531 trim_cache size 0 max 16384 2024-05-04T03:15:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.061+0000 7fd22d9af740 1 Processor -- start 2024-05-04T03:15:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.061+0000 7fd22d9af740 1 -- start start 2024-05-04T03:15:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481de3d80 0x55f4821bcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481dfc2f0 0x55f4821e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd22d9af740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd2201ad560 con 0x55f481dfc2f0 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd22d9af740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd2201af0a0 con 0x55f481de3d80 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd22d9af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2201adc30 con 0x55f481e044d0 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481dfc2f0 0x55f4821e3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481dfc2f0 0x55f4821e3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38508/0 (socket says 172.21.15.39:38508) 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 -- 172.21.15.39:0/124187684 learned_addr learned my addr 172.21.15.39:0/124187684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd21ffff640 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481de3d80 0x55f4821bcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd21ffff640 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 -- 172.21.15.39:0/124187684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 msgr2=0x55f4821e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 -- 172.21.15.39:0/124187684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481de3d80 msgr2=0x55f4821bcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481de3d80 0x55f4821bcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 -- 172.21.15.39:0/124187684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4821cded0 con 0x55f481dfc2f0 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd21ffff640 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd224895640 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481dfc2f0 0x55f4821e3e20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd21402a930 tx=0x7fd21402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5959e1fad2a1afc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.062+0000 7fd20a7fc640 1 -- 172.21.15.39:0/124187684 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214055020 con 0x55f481dfc2f0 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd20a7fc640 1 -- 172.21.15.39:0/124187684 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd214051450 con 0x55f481dfc2f0 2024-05-04T03:15:57.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd20a7fc640 1 -- 172.21.15.39:0/124187684 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214035070 con 0x55f481dfc2f0 2024-05-04T03:15:57.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd22d9af740 1 -- 172.21.15.39:0/124187684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481dfc2f0 msgr2=0x55f4821e3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd22d9af740 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481dfc2f0 0x55f4821e3e20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd21402a930 tx=0x7fd21402a410 comp rx=0 tx=0).stop 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd22d9af740 1 -- 172.21.15.39:0/124187684 shutdown_connections 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd22d9af740 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f4821e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd22d9af740 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481dfc2f0 0x55f4821e3e20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd22d9af740 1 --2- 172.21.15.39:0/124187684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481de3d80 0x55f4821bcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.063+0000 7fd22d9af740 1 -- 172.21.15.39:0/124187684 >> 172.21.15.39:0/124187684 conn(0x55f481e19300 msgr2=0x55f4821caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 -- 172.21.15.39:0/124187684 shutdown_connections 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 -- 172.21.15.39:0/124187684 wait complete. 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 -- start start 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 Processor -- start 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481de3d80 0x55f4821d74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f482244ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd2201ae9d0 con 0x55f481de3d80 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd2201ae300 con 0x55f481dfc2f0 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2201ab540 con 0x55f481e044d0 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.064+0000 7fd22d9af740 10 client.0 _ll_register_callbacks cb 0x7fd214009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481de3d80 0x55f4821d74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481de3d80 0x55f4821d74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38518/0 (socket says 172.21.15.39:38518) 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 -- 172.21.15.39:0/2584385680 learned_addr learned my addr 172.21.15.39:0/2584385680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd224895640 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 -- 172.21.15.39:0/2584385680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 msgr2=0x55f482244ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f482244ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 -- 172.21.15.39:0/2584385680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 msgr2=0x55f4821d79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821d79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 -- 172.21.15.39:0/2584385680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4822450b0 con 0x55f481de3d80 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd21ffff640 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481de3d80 0x55f4821d74b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55f4821ded90 tx=0x7fd20c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02867b0d8c0f22c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd1fffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd1fffff640 1 -- 172.21.15.39:0/2584385680 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c035060 con 0x55f481de3d80 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd1fffff640 1 -- 172.21.15.39:0/2584385680 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd20c004f00 con 0x55f481de3d80 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd22d9af740 1 -- 172.21.15.39:0/2584385680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd214051450 con 0x55f481de3d80 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd1fffff640 1 -- 172.21.15.39:0/2584385680 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd20c0303d0 con 0x55f481de3d80 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd22d9af740 10 client.15888 fetch_fsmap learned FSMap version 32 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.065+0000 7fd22d9af740 1 -- 172.21.15.39:0/2584385680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f482246370 con 0x55f481de3d80 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd1fffff640 1 -- 172.21.15.39:0/2584385680 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd214051450 con 0x55f481de3d80 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd1fffff640 1 -- 172.21.15.39:0/2584385680 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd20c004350 con 0x55f481de3d80 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd22d9af740 10 client.15888 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd22d9af740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd22d9af740 20 client.15888 populate_metadata read hostname 'smithi039' 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd22d9af740 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f481e251b0 0x55f482248c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd22d9af740 4 client.15888 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:15:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd22d9af740 1 -- 172.21.15.39:0/2584385680 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f482133050 con 0x55f481e251b0 2024-05-04T03:15:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd224895640 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f481e251b0 0x55f482248c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd1fffff640 10 client.15888 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.066+0000 7fd224895640 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f481e251b0 0x55f482248c20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f28c3283da7395b server_cookie=a19077f12125b194 in_seq=0 out_seq=0 2024-05-04T03:15:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd1fffff640 1 -- 172.21.15.39:0/2584385680 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f482133050 con 0x55f481e251b0 2024-05-04T03:15:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd1fffff640 10 client.15888 handle_command_reply: tid=0 2024-05-04T03:15:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 client.15888 shutdown 2024-05-04T03:15:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 -- 172.21.15.39:0/2584385680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481de3d80 msgr2=0x55f4821d74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481de3d80 0x55f4821d74b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55f4821ded90 tx=0x7fd20c0096c0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 -- 172.21.15.39:0/2584385680 shutdown_connections 2024-05-04T03:15:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f481e251b0 0x55f482248c20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f481e044d0 0x55f482244ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f481dfc2f0 0x55f4821d79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 --2- 172.21.15.39:0/2584385680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f481de3d80 0x55f4821d74b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd1fffff640 0 client.15888 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:15:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 -- 172.21.15.39:0/2584385680 >> 172.21.15.39:0/2584385680 conn(0x55f481e19300 msgr2=0x55f4821c51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 -- 172.21.15.39:0/2584385680 shutdown_connections 2024-05-04T03:15:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 1 -- 172.21.15.39:0/2584385680 wait complete. 2024-05-04T03:15:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.075+0000 7fd22d9af740 20 client.15888 trim_cache size 0 max 16384 2024-05-04T03:15:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:15:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:15:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:15:57.081 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 -- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd2100b27e0 msgr2=0x7fd2100b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd2100b27e0 0x7fd2100b4cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd2180e02c0 tx=0x7fd218099040 comp rx=0 tx=0).stop 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 -- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2201098e0 msgr2=0x7fd2201027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2201098e0 0x7fd2201027d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd21800eb10 tx=0x7fd21800c2d0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 -- 172.21.15.39:0/491118316 shutdown_connections 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd2100b27e0 0x7fd2100b4cf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2201098e0 0x7fd2201027d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd220106d70 0x7fd220102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 --2- 172.21.15.39:0/491118316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd220104380 0x7fd220103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 -- 172.21.15.39:0/491118316 >> 172.21.15.39:0/491118316 conn(0x7fd2200fdd40 msgr2=0x7fd2201063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 -- 172.21.15.39:0/491118316 shutdown_connections 2024-05-04T03:15:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:15:57.076+0000 7fd226af9640 1 -- 172.21.15.39:0/491118316 wait complete. 2024-05-04T03:15:57.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:58.059 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:58.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:15:58 smithi039 ceph-mon[29715]: pgmap v541: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 210 KiB/s wr, 37 op/s 2024-05-04T03:15:58.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:15:58 smithi066 ceph-mon[33264]: pgmap v541: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 210 KiB/s wr, 37 op/s 2024-05-04T03:15:58.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:15:58 smithi167 ceph-mon[33614]: pgmap v541: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 210 KiB/s wr, 37 op/s 2024-05-04T03:15:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.824+0000 7f4ab57a2700 1 -- 172.21.15.39:0/2260055819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 msgr2=0x7f4ab0108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.824+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/2260055819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 0x7f4ab0108050 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40099c0 tx=0x7f4aa4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.824+0000 7f4ab57a2700 1 -- 172.21.15.39:0/2260055819 shutdown_connections 2024-05-04T03:15:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.824+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/2260055819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0108590 0x7f4ab010fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.824+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/2260055819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab0103310 0x7f4ab0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.824+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/2260055819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 0x7f4ab0108050 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.824+0000 7f4ab57a2700 1 -- 172.21.15.39:0/2260055819 >> 172.21.15.39:0/2260055819 conn(0x7f4ab0078c40 msgr2=0x7f4ab0079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.825+0000 7f4ab57a2700 1 -- 172.21.15.39:0/2260055819 shutdown_connections 2024-05-04T03:15:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.825+0000 7f4ab57a2700 1 -- 172.21.15.39:0/2260055819 wait complete. 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.825+0000 7f4ab57a2700 1 Processor -- start 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.825+0000 7f4ab57a2700 1 -- start start 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4ab57a2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab0103310 0x7f4ab01108d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4ab57a2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 0x7f4ab0110e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4ab57a2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0108590 0x7f4ab0111350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4ab57a2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01141c0 con 0x7f4ab0107c40 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4ab57a2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0114040 con 0x7f4ab0103310 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4ab57a2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0114340 con 0x7f4ab0108590 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 0x7f4ab0110e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 0x7f4ab0110e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38534/0 (socket says 172.21.15.39:38534) 2024-05-04T03:15:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 -- 172.21.15.39:0/3732759261 learned_addr learned my addr 172.21.15.39:0/3732759261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaffff700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab0103310 0x7f4ab01108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4ab4fa1700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0108590 0x7f4ab0111350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 -- 172.21.15.39:0/3732759261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0108590 msgr2=0x7f4ab0111350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0108590 0x7f4ab0111350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 -- 172.21.15.39:0/3732759261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab0103310 msgr2=0x7f4ab01108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab0103310 0x7f4ab01108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 -- 172.21.15.39:0/3732759261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab01a8ef0 con 0x7f4ab0107c40 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.826+0000 7f4aaf7fe700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 0x7f4ab0110e10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400b580 tx=0x7f4aa40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.827+0000 7f4aad7fa700 1 -- 172.21.15.39:0/3732759261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4027070 con 0x7f4ab0107c40 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.827+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab01a9180 con 0x7f4ab0107c40 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.827+0000 7f4aad7fa700 1 -- 172.21.15.39:0/3732759261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4aa400fc80 con 0x7f4ab0107c40 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.827+0000 7f4aad7fa700 1 -- 172.21.15.39:0/3732759261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4022da0 con 0x7f4ab0107c40 2024-05-04T03:15:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.827+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01a9630 con 0x7f4ab0107c40 2024-05-04T03:15:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.828+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab00512c0 con 0x7f4ab0107c40 2024-05-04T03:15:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.829+0000 7f4aad7fa700 1 -- 172.21.15.39:0/3732759261 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4aa402a070 con 0x7f4ab0107c40 2024-05-04T03:15:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.830+0000 7f4aad7fa700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a980b2880 0x7f4a980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.830+0000 7f4aaffff700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a980b2880 0x7f4a980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.830+0000 7f4aad7fa700 1 -- 172.21.15.39:0/3732759261 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4aa40e31b0 con 0x7f4ab0107c40 2024-05-04T03:15:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.830+0000 7f4aaffff700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a980b2880 0x7f4a980b4d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0005950 tx=0x7f4aa000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:15:59.834+0000 7f4aad7fa700 1 -- 172.21.15.39:0/3732759261 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4aa405fdf0 con 0x7f4ab0107c40 2024-05-04T03:16:00.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.094+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ab002e5f0 con 0x7f4ab0107c40 2024-05-04T03:16:00.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.095+0000 7f4aad7fa700 1 -- 172.21.15.39:0/3732759261 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f4aa4013070 con 0x7f4ab0107c40 2024-05-04T03:16:00.097 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:16:00.097 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:16:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.097+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a980b2880 msgr2=0x7f4a980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.097+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a980b2880 0x7f4a980b4d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0005950 tx=0x7f4aa000b410 comp rx=0 tx=0).stop 2024-05-04T03:16:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.097+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 msgr2=0x7f4ab0110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.097+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 0x7f4ab0110e10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400b580 tx=0x7f4aa40057c0 comp rx=0 tx=0).stop 2024-05-04T03:16:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.098+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 shutdown_connections 2024-05-04T03:16:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.098+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a980b2880 0x7f4a980b4d90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.098+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab0108590 0x7f4ab0111350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:00.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.098+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ab0103310 0x7f4ab01108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:00.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.098+0000 7f4ab57a2700 1 --2- 172.21.15.39:0/3732759261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ab0107c40 0x7f4ab0110e10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:00.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.098+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 >> 172.21.15.39:0/3732759261 conn(0x7f4ab0078c40 msgr2=0x7f4ab0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:00.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.098+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 shutdown_connections 2024-05-04T03:16:00.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:00.098+0000 7f4ab57a2700 1 -- 172.21.15.39:0/3732759261 wait complete. 2024-05-04T03:16:00.100 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:16:00.401 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:00 smithi039 ceph-mon[29715]: pgmap v542: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 257 KiB/s wr, 43 op/s 2024-05-04T03:16:00.401 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:00 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3732759261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:00.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:00 smithi066 ceph-mon[33264]: pgmap v542: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 257 KiB/s wr, 43 op/s 2024-05-04T03:16:00.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:00 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3732759261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:00 smithi167 ceph-mon[33614]: pgmap v542: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 257 KiB/s wr, 43 op/s 2024-05-04T03:16:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:00 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3732759261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:00.893 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 92682a91 --release --await' 2024-05-04T03:16:00.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id 92682a91 --release --await 2024-05-04T03:16:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 130.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:16:02.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:02 smithi066 ceph-mon[33264]: pgmap v543: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 238 KiB/s wr, 41 op/s 2024-05-04T03:16:02.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:02 smithi039 ceph-mon[29715]: pgmap v543: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 238 KiB/s wr, 41 op/s 2024-05-04T03:16:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:02 smithi167 ceph-mon[33614]: pgmap v543: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 238 KiB/s wr, 41 op/s 2024-05-04T03:16:03.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.020+0000 7f55718fd700 1 -- 172.21.15.39:0/3146837172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 msgr2=0x7f556c104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.020+0000 7f55718fd700 1 --2- 172.21.15.39:0/3146837172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 0x7f556c104a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f555c0099c0 tx=0x7f555c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 -- 172.21.15.39:0/3146837172 shutdown_connections 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 --2- 172.21.15.39:0/3146837172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 0x7f556c104a60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 --2- 172.21.15.39:0/3146837172 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c109880 0x7f556c1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 --2- 172.21.15.39:0/3146837172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f556c108f30 0x7f556c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 -- 172.21.15.39:0/3146837172 >> 172.21.15.39:0/3146837172 conn(0x7f556c079f00 msgr2=0x7f556c07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 -- 172.21.15.39:0/3146837172 shutdown_connections 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 -- 172.21.15.39:0/3146837172 wait complete. 2024-05-04T03:16:03.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 Processor -- start 2024-05-04T03:16:03.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.021+0000 7f55718fd700 1 -- start start 2024-05-04T03:16:03.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f55718fd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 0x7f556c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f55718fd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f556c108f30 0x7f556c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f55718fd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c109880 0x7f556c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f55718fd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f556c114380 con 0x7f556c104650 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f55718fd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f556c114200 con 0x7f556c109880 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f55718fd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f556c114500 con 0x7f556c108f30 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 0x7f556c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 0x7f556c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38550/0 (socket says 172.21.15.39:38550) 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556affd700 1 -- 172.21.15.39:0/3673776003 learned_addr learned my addr 172.21.15.39:0/3673776003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556a7fc700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f556c108f30 0x7f556c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556b7fe700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c109880 0x7f556c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556affd700 1 -- 172.21.15.39:0/3673776003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f556c108f30 msgr2=0x7f556c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556affd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f556c108f30 0x7f556c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556affd700 1 -- 172.21.15.39:0/3673776003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c109880 msgr2=0x7f556c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556affd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c109880 0x7f556c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556affd700 1 -- 172.21.15.39:0/3673776003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c19fce0 con 0x7f556c104650 2024-05-04T03:16:03.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.022+0000 7f556b7fe700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c109880 0x7f556c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:03.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.023+0000 7f556affd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 0x7f556c19ae10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f555c000c00 tx=0x7f555c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:03.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.023+0000 7f55708fb700 1 -- 172.21.15.39:0/3673776003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00f8a0 con 0x7f556c104650 2024-05-04T03:16:03.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.023+0000 7f55718fd700 1 -- 172.21.15.39:0/3673776003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c19ff70 con 0x7f556c104650 2024-05-04T03:16:03.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.023+0000 7f55708fb700 1 -- 172.21.15.39:0/3673776003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f555c004830 con 0x7f556c104650 2024-05-04T03:16:03.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.023+0000 7f55708fb700 1 -- 172.21.15.39:0/3673776003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c005410 con 0x7f556c104650 2024-05-04T03:16:03.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.023+0000 7f55718fd700 1 -- 172.21.15.39:0/3673776003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c1a0440 con 0x7f556c104650 2024-05-04T03:16:03.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.026+0000 7f55708fb700 1 -- 172.21.15.39:0/3673776003 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f555c029060 con 0x7f556c104650 2024-05-04T03:16:03.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.027+0000 7f55708fb700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55580b2880 0x7f55580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.027+0000 7f5580311b80 1 Processor -- start 2024-05-04T03:16:03.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.027+0000 7f556a7fc700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55580b2880 0x7f55580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.027+0000 7f5580311b80 1 -- start start 2024-05-04T03:16:03.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.027+0000 7f55708fb700 1 -- 172.21.15.39:0/3673776003 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f555c0e2cf0 con 0x7f556c104650 2024-05-04T03:16:03.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.027+0000 7f5580311b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d96593880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.027+0000 7f5580311b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.027+0000 7f5580311b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 0x563d965a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f5580311b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563d964b6960 con 0x563d96593dc0 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f5580311b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563d964558c0 con 0x563d96593470 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f5580311b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d964b2920 con 0x563d965a74c0 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55580b2880 0x7f55580b4d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f556c07a350 tx=0x7f555400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38554/0 (socket says 172.21.15.39:38554) 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 -- 172.21.15.39:0/2099533372 learned_addr learned my addr 172.21.15.39:0/2099533372 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556affd700 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d96593880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556b7fe700 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 0x563d965a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 -- 172.21.15.39:0/2099533372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 msgr2=0x563d965a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:03.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 0x563d965a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 -- 172.21.15.39:0/2099533372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 msgr2=0x563d96593880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:03.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d96593880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 -- 172.21.15.39:0/2099533372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d964039d0 con 0x563d96593dc0 2024-05-04T03:16:03.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556affd700 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d96593880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:03.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f556a7fc700 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a6f80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f555400fdb0 tx=0x7f5554012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc56ad140ba2b648 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:03.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.028+0000 7f5561ffb700 1 -- 172.21.15.39:0/2099533372 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554016070 con 0x563d96593dc0 2024-05-04T03:16:03.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5561ffb700 1 -- 172.21.15.39:0/2099533372 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f555400fe90 con 0x563d96593dc0 2024-05-04T03:16:03.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5561ffb700 1 -- 172.21.15.39:0/2099533372 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555401cda0 con 0x563d96593dc0 2024-05-04T03:16:03.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5580311b80 1 -- 172.21.15.39:0/2099533372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 msgr2=0x563d965a6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5580311b80 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a6f80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f555400fdb0 tx=0x7f5554012000 comp rx=0 tx=0).stop 2024-05-04T03:16:03.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5580311b80 1 -- 172.21.15.39:0/2099533372 shutdown_connections 2024-05-04T03:16:03.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5580311b80 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 0x563d965a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5580311b80 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d96593880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5580311b80 1 --2- 172.21.15.39:0/2099533372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a6f80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5580311b80 1 -- 172.21.15.39:0/2099533372 >> 172.21.15.39:0/2099533372 conn(0x563d96416f40 msgr2=0x563d9658f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.029+0000 7f5580311b80 1 -- 172.21.15.39:0/2099533372 shutdown_connections 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 -- 172.21.15.39:0/2099533372 wait complete. 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 -- start start 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 Processor -- start 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d965a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 0x563d965a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563d965a9ee0 con 0x563d96593dc0 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563d9641cd30 con 0x563d96593470 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.030+0000 7f5580311b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d965aa060 con 0x563d965a74c0 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f5580311b80 10 client.0 _ll_register_callbacks cb 0x563d9644ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:03.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38566/0 (socket says 172.21.15.39:38566) 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 -- 172.21.15.39:0/1277015305 learned_addr learned my addr 172.21.15.39:0/1277015305 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556affd700 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d965a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556b7fe700 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 0x563d965a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 -- 172.21.15.39:0/1277015305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 msgr2=0x563d965a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 0x563d965a4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 -- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 msgr2=0x563d965a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d965a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 -- 172.21.15.39:0/1277015305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d96452790 con 0x563d96593dc0 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556affd700 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d965a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f556a7fc700 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a4980 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5554005950 tx=0x7f555401dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df3480ec633f8b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f554affd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:03.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f554affd700 1 -- 172.21.15.39:0/1277015305 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554016040 con 0x563d96593dc0 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f554affd700 1 -- 172.21.15.39:0/1277015305 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f5554010ad0 con 0x563d96593dc0 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.031+0000 7f5580311b80 1 -- 172.21.15.39:0/1277015305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f555400fe90 con 0x563d96593dc0 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f554affd700 1 -- 172.21.15.39:0/1277015305 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5554010cd0 con 0x563d96593dc0 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f5580311b80 10 client.15906 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f5580311b80 1 -- 172.21.15.39:0/1277015305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563d965b4a80 con 0x563d96593dc0 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f554affd700 1 -- 172.21.15.39:0/1277015305 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f555400fe90 con 0x563d96593dc0 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f554affd700 1 -- 172.21.15.39:0/1277015305 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5554004af0 con 0x563d96593dc0 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f5580311b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f5580311b80 10 client.15906 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f5580311b80 20 client.15906 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f5580311b80 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563d965b7000 0x563d965b9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f5580311b80 4 client.15906 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f5580311b80 1 -- 172.21.15.39:0/1277015305 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d964c4340 con 0x563d965b7000 2024-05-04T03:16:03.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.032+0000 7f556affd700 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563d965b7000 0x563d965b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.033+0000 7f554affd700 10 client.15906 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.033+0000 7f556affd700 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563d965b7000 0x563d965b9420 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7e059c034ee02f server_cookie=1e457b5c4e365ef1 in_seq=0 out_seq=0 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f554affd700 1 -- 172.21.15.39:0/1277015305 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d964c4340 con 0x563d965b7000 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f554affd700 10 client.15906 handle_command_reply: tid=0 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 client.15906 shutdown 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 -- 172.21.15.39:0/1277015305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 msgr2=0x563d965a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a4980 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5554005950 tx=0x7f555401dae0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 -- 172.21.15.39:0/1277015305 shutdown_connections 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563d965b7000 0x563d965b9420 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d965a74c0 0x563d965a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d96593470 0x563d965a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f554affd700 0 client.15906 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:03.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 --2- 172.21.15.39:0/1277015305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593dc0 0x563d965a4980 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 -- 172.21.15.39:0/1277015305 >> 172.21.15.39:0/1277015305 conn(0x563d96416f40 msgr2=0x563d96595230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 -- 172.21.15.39:0/1277015305 shutdown_connections 2024-05-04T03:16:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 1 -- 172.21.15.39:0/1277015305 wait complete. 2024-05-04T03:16:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.034+0000 7f5580311b80 20 client.15906 trim_cache size 0 max 16384 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.058+0000 7f5580311b80 1 Processor -- start 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.058+0000 7f5580311b80 1 -- start start 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.058+0000 7f5580311b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d965bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.058+0000 7f5580311b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 0x563d965bc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.058+0000 7f5580311b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 0x563d96598ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.058+0000 7f5580311b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f556c1ac8f0 con 0x563d96593470 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.058+0000 7f5580311b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f556c1ae430 con 0x563d965a74c0 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.058+0000 7f5580311b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f556c1acfc0 con 0x563d96593dc0 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d965bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d965bc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38572/0 (socket says 172.21.15.39:38572) 2024-05-04T03:16:03.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 -- 172.21.15.39:0/3588133968 learned_addr learned my addr 172.21.15.39:0/3588133968 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556a7fc700 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 0x563d965bc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556b7fe700 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 0x563d96598ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 -- 172.21.15.39:0/3588133968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 msgr2=0x563d965bc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 0x563d965bc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 -- 172.21.15.39:0/3588133968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 msgr2=0x563d96598ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 0x563d96598ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 -- 172.21.15.39:0/3588133968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9658b800 con 0x563d96593470 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f556affd700 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d965bc450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f555c0aa0b0 tx=0x7f555c092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517737ed8e160d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f55617fa700 1 -- 172.21.15.39:0/3588133968 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c0e0910 con 0x563d96593470 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f55617fa700 1 -- 172.21.15.39:0/3588133968 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f555c004ce0 con 0x563d96593470 2024-05-04T03:16:03.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.059+0000 7f55617fa700 1 -- 172.21.15.39:0/3588133968 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c0b2cf0 con 0x563d96593470 2024-05-04T03:16:03.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 -- 172.21.15.39:0/3588133968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 msgr2=0x563d965bc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d965bc450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f555c0aa0b0 tx=0x7f555c092040 comp rx=0 tx=0).stop 2024-05-04T03:16:03.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 -- 172.21.15.39:0/3588133968 shutdown_connections 2024-05-04T03:16:03.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d965bc450 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 0x563d96598ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 --2- 172.21.15.39:0/3588133968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 0x563d965bc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 -- 172.21.15.39:0/3588133968 >> 172.21.15.39:0/3588133968 conn(0x563d96416f40 msgr2=0x563d965b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 -- 172.21.15.39:0/3588133968 shutdown_connections 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.060+0000 7f5580311b80 1 -- 172.21.15.39:0/3588133968 wait complete. 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 1 -- start start 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 1 Processor -- start 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d9661afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 0x563d9661b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 0x563d966153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f556c1add60 con 0x563d96593470 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f556c1ad690 con 0x563d965a74c0 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f556c1aa8b0 con 0x563d96593dc0 2024-05-04T03:16:03.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f5580311b80 10 client.0 _ll_register_callbacks cb 0x563d9644ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f556affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d9661afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f556affd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d9661afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38574/0 (socket says 172.21.15.39:38574) 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.061+0000 7f556affd700 1 -- 172.21.15.39:0/3656664463 learned_addr learned my addr 172.21.15.39:0/3656664463 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f556b7fe700 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 0x563d966153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f556a7fc700 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 0x563d9661b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f556affd700 1 -- 172.21.15.39:0/3656664463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 msgr2=0x563d9661b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f556affd700 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 0x563d9661b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f556affd700 1 -- 172.21.15.39:0/3656664463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 msgr2=0x563d966153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f556affd700 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 0x563d966153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f556affd700 1 -- 172.21.15.39:0/3656664463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d96615a30 con 0x563d96593470 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f556affd700 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d9661afc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f555c0e0850 tx=0x7f555c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acc4a59091aceee server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f554affd700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f554affd700 1 -- 172.21.15.39:0/3656664463 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c0aa610 con 0x563d96593470 2024-05-04T03:16:03.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f554affd700 1 -- 172.21.15.39:0/3656664463 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f555c0aa7b0 con 0x563d96593470 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f5580311b80 1 -- 172.21.15.39:0/3656664463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f555c004ce0 con 0x563d96593470 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f554affd700 1 -- 172.21.15.39:0/3656664463 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f555c0b2cf0 con 0x563d96593470 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f5580311b80 10 client.15912 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.062+0000 7f5580311b80 1 -- 172.21.15.39:0/3656664463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563d966226d0 con 0x563d96593470 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f554affd700 1 -- 172.21.15.39:0/3656664463 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f555c004ce0 con 0x563d96593470 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f554affd700 1 -- 172.21.15.39:0/3656664463 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f555c00fab0 con 0x563d96593470 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f5580311b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f5580311b80 10 client.15912 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f5580311b80 20 client.15912 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f5580311b80 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563d96622ba0 0x563d96624fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f5580311b80 4 client.15912 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "92682a91", "release": true, "await": true}] 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f5580311b80 1 -- 172.21.15.39:0/3656664463 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "92682a91", "release": true, "await": true}) -- 0x563d964be3e0 con 0x563d96622ba0 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.063+0000 7f556a7fc700 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563d96622ba0 0x563d96624fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:03.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.064+0000 7f554affd700 10 client.15912 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:03.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.064+0000 7f556a7fc700 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563d96622ba0 0x563d96624fc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc34c1d4067782f server_cookie=fe229a8a2b19625a in_seq=0 out_seq=0 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.069+0000 7f554affd700 1 -- 172.21.15.39:0/3656664463 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563d964be3e0 con 0x563d96622ba0 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.069+0000 7f554affd700 10 client.15912 handle_command_reply: tid=0 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.069+0000 7f5580311b80 1 client.15912 shutdown 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 -- 172.21.15.39:0/3656664463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 msgr2=0x563d9661afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d9661afc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f555c0e0850 tx=0x7f555c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 -- 172.21.15.39:0/3656664463 shutdown_connections 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563d96622ba0 0x563d96624fc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d96593470 0x563d9661afc0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563d965a74c0 0x563d966153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 --2- 172.21.15.39:0/3656664463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d96593dc0 0x563d9661b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 -- 172.21.15.39:0/3656664463 >> 172.21.15.39:0/3656664463 conn(0x563d96416f40 msgr2=0x563d965b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f554affd700 0 client.15912 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:03.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 -- 172.21.15.39:0/3656664463 shutdown_connections 2024-05-04T03:16:03.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 1 -- 172.21.15.39:0/3656664463 wait complete. 2024-05-04T03:16:03.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.070+0000 7f5580311b80 20 client.15912 trim_cache size 0 max 16384 2024-05-04T03:16:03.073 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:16:03.073 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:16:03.073 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:16:03.073 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 12, 2024-05-04T03:16:03.073 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "92682a91": { 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 12, 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 90.0, 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:16:03.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:16:03.075 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:16:03.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:03.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:03.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:03.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:03.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:03.075 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:16:03.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 -- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55580b2880 msgr2=0x7f55580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55580b2880 0x7f55580b4d90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f556c07a350 tx=0x7f555400a300 comp rx=0 tx=0).stop 2024-05-04T03:16:03.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 -- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 msgr2=0x7f556c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:03.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 0x7f556c19ae10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f555c000c00 tx=0x7f555c0047b0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 -- 172.21.15.39:0/3673776003 shutdown_connections 2024-05-04T03:16:03.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55580b2880 0x7f55580b4d90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f556c104650 0x7f556c19ae10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f556c109880 0x7f556c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 --2- 172.21.15.39:0/3673776003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f556c108f30 0x7f556c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:03.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 -- 172.21.15.39:0/3673776003 >> 172.21.15.39:0/3673776003 conn(0x7f556c079f00 msgr2=0x7f556c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:03.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 -- 172.21.15.39:0/3673776003 shutdown_connections 2024-05-04T03:16:03.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:03.071+0000 7f55718fd700 1 -- 172.21.15.39:0/3673776003 wait complete. 2024-05-04T03:16:03.764 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '92682a91', seconds elapsed: 48.1 2024-05-04T03:16:03.764 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-04T03:16:03.765 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:16:04.172 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:04.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:04 smithi066 ceph-mon[33264]: pgmap v544: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 269 KiB/s wr, 45 op/s 2024-05-04T03:16:04.598 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:04 smithi039 ceph-mon[29715]: pgmap v544: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 269 KiB/s wr, 45 op/s 2024-05-04T03:16:04.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:04 smithi167 ceph-mon[33614]: pgmap v544: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 269 KiB/s wr, 45 op/s 2024-05-04T03:16:06.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.054+0000 7efe143a9700 1 -- 172.21.15.39:0/3923638979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 msgr2=0x7efe0c111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:06.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.054+0000 7efe143a9700 1 --2- 172.21.15.39:0/3923638979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 0x7efe0c111120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efe04009f90 tx=0x7efe0400b330 comp rx=0 tx=0).stop 2024-05-04T03:16:06.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.055+0000 7efe143a9700 1 -- 172.21.15.39:0/3923638979 shutdown_connections 2024-05-04T03:16:06.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.055+0000 7efe143a9700 1 --2- 172.21.15.39:0/3923638979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c10a3e0 0x7efe0c10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.055+0000 7efe143a9700 1 --2- 172.21.15.39:0/3923638979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 0x7efe0c111120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.055+0000 7efe143a9700 1 --2- 172.21.15.39:0/3923638979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe0c1027e0 0x7efe0c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.055+0000 7efe143a9700 1 -- 172.21.15.39:0/3923638979 >> 172.21.15.39:0/3923638979 conn(0x7efe0c0fe6f0 msgr2=0x7efe0c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:06.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.056+0000 7efe143a9700 1 -- 172.21.15.39:0/3923638979 shutdown_connections 2024-05-04T03:16:06.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.056+0000 7efe143a9700 1 -- 172.21.15.39:0/3923638979 wait complete. 2024-05-04T03:16:06.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.057+0000 7efe143a9700 1 Processor -- start 2024-05-04T03:16:06.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.057+0000 7efe143a9700 1 -- start start 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe143a9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c1027e0 0x7efe0c10b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe143a9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 0x7efe0c10b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe143a9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe0c10a3e0 0x7efe0c10bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe11944700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 0x7efe0c10b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe11944700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 0x7efe0c10b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:39152/0 (socket says 172.21.15.39:39152) 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe143a9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c114380 con 0x7efe0c1027e0 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe143a9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c114200 con 0x7efe0c10a3e0 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe143a9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c114500 con 0x7efe0c103130 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe12946700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe0c10a3e0 0x7efe0c10bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe12946700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe0c10a3e0 0x7efe0c10bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:59432/0 (socket says 172.21.15.39:59432) 2024-05-04T03:16:06.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe12946700 1 -- 172.21.15.39:0/3420141872 learned_addr learned my addr 172.21.15.39:0/3420141872 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:06.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe11944700 1 -- 172.21.15.39:0/3420141872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe0c10a3e0 msgr2=0x7efe0c10bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:06.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe11944700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe0c10a3e0 0x7efe0c10bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe11944700 1 -- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c1027e0 msgr2=0x7efe0c10b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:06.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe11944700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c1027e0 0x7efe0c10b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe11944700 1 -- 172.21.15.39:0/3420141872 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c1ad530 con 0x7efe0c103130 2024-05-04T03:16:06.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.058+0000 7efe12946700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe0c10a3e0 0x7efe0c10bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:06.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.059+0000 7efe11944700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 0x7efe0c10b6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00e9c0 tx=0x7efdfc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:06.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.059+0000 7efe037fe700 1 -- 172.21.15.39:0/3420141872 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00cbd0 con 0x7efe0c103130 2024-05-04T03:16:06.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.059+0000 7efe037fe700 1 -- 172.21.15.39:0/3420141872 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdfc00cd70 con 0x7efe0c103130 2024-05-04T03:16:06.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.059+0000 7efe037fe700 1 -- 172.21.15.39:0/3420141872 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc018680 con 0x7efe0c103130 2024-05-04T03:16:06.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.059+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c1ad7c0 con 0x7efe0c103130 2024-05-04T03:16:06.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.059+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c1adcb0 con 0x7efe0c103130 2024-05-04T03:16:06.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.061+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0c105b00 con 0x7efe0c103130 2024-05-04T03:16:06.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.063+0000 7efe037fe700 1 -- 172.21.15.39:0/3420141872 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efdfc010430 con 0x7efe0c103130 2024-05-04T03:16:06.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.065+0000 7efe037fe700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdf80b2900 0x7efdf80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:06.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.066+0000 7efe12145700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdf80b2900 0x7efdf80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:06.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.066+0000 7efe037fe700 1 -- 172.21.15.39:0/3420141872 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efdfc0e2340 con 0x7efe0c103130 2024-05-04T03:16:06.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.066+0000 7efe12145700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdf80b2900 0x7efdf80b4e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe08005fd0 tx=0x7efe080095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:06.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.071+0000 7efe037fe700 1 -- 172.21.15.39:0/3420141872 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efdfc0aa070 con 0x7efe0c103130 2024-05-04T03:16:06.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.515+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe0c0512c0 con 0x7efe0c103130 2024-05-04T03:16:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.516+0000 7efe037fe700 1 -- 172.21.15.39:0/3420141872 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7efdfc0b20f0 con 0x7efe0c103130 2024-05-04T03:16:06.518 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:16:06.518 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.520+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdf80b2900 msgr2=0x7efdf80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdf80b2900 0x7efdf80b4e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe08005fd0 tx=0x7efe080095b0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 msgr2=0x7efe0c10b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 0x7efe0c10b6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00e9c0 tx=0x7efdfc00ecd0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 shutdown_connections 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efdf80b2900 0x7efdf80b4e10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe0c1027e0 0x7efe0c10b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe0c10a3e0 0x7efe0c10bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 --2- 172.21.15.39:0/3420141872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe0c103130 0x7efe0c10b6e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.521+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 >> 172.21.15.39:0/3420141872 conn(0x7efe0c0fe6f0 msgr2=0x7efe0c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.522+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 shutdown_connections 2024-05-04T03:16:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:06.522+0000 7efe143a9700 1 -- 172.21.15.39:0/3420141872 wait complete. 2024-05-04T03:16:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:16:06.542 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:06 smithi039 ceph-mon[29715]: pgmap v545: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 401 KiB/s wr, 59 op/s 2024-05-04T03:16:06.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:06 smithi066 ceph-mon[33264]: pgmap v545: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 401 KiB/s wr, 59 op/s 2024-05-04T03:16:06.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:06 smithi167 ceph-mon[33614]: pgmap v545: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 401 KiB/s wr, 59 op/s 2024-05-04T03:16:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 135.0 s, 871.2 tps, lat 948.323 ms stddev 5709.872 2024-05-04T03:16:07.370 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-05-04T03:16:07.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-05-04T03:16:07.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:07 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3420141872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:07.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:07 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3420141872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:07.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:07 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3420141872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:08.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:08 smithi039 ceph-mon[29715]: pgmap v546: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-04T03:16:08.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:08 smithi066 ceph-mon[33264]: pgmap v546: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-04T03:16:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:08 smithi167 ceph-mon[33614]: pgmap v546: 97 pgs: 97 active+clean; 995 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-05-04T03:16:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.970+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1316037887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d8110990 msgr2=0x7fb9d8110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.970+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1316037887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d8110990 0x7fb9d8110da0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009990 tx=0x7fb9c0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.971+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1316037887 shutdown_connections 2024-05-04T03:16:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.971+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1316037887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d8110990 0x7fb9d8110da0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.971+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1316037887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9d810b2b0 0x7fb9d8117830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.971+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1316037887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9d810a960 0x7fb9d810ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.971+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1316037887 >> 172.21.15.39:0/1316037887 conn(0x7fb9d8076d30 msgr2=0x7fb9d8077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:09.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.971+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1316037887 shutdown_connections 2024-05-04T03:16:09.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.971+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1316037887 wait complete. 2024-05-04T03:16:09.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9dcbfe700 1 Processor -- start 2024-05-04T03:16:09.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9dcbfe700 1 -- start start 2024-05-04T03:16:09.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9dcbfe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d810a960 0x7fb9d81a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9dcbfe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9d810b2b0 0x7fb9d81a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9dcbfe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9d8110990 0x7fb9d81a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9dcbfe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d811a9a0 con 0x7fb9d810a960 2024-05-04T03:16:09.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9dcbfe700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d811a820 con 0x7fb9d810b2b0 2024-05-04T03:16:09.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9dcbfe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d811ab20 con 0x7fb9d8110990 2024-05-04T03:16:09.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9d659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d810a960 0x7fb9d81a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9d5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9d810b2b0 0x7fb9d81a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9d659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d810a960 0x7fb9d81a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56912/0 (socket says 172.21.15.39:56912) 2024-05-04T03:16:09.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9d5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9d810b2b0 0x7fb9d81a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:59444/0 (socket says 172.21.15.39:59444) 2024-05-04T03:16:09.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9d659c700 1 -- 172.21.15.39:0/1872391879 learned_addr learned my addr 172.21.15.39:0/1872391879 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:09.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.973+0000 7fb9d6d9d700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9d8110990 0x7fb9d81a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.974+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/1872391879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9d8110990 msgr2=0x7fb9d81a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:09.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.974+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9d8110990 0x7fb9d81a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.974+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d810a960 msgr2=0x7fb9d81a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:09.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.974+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d810a960 0x7fb9d81a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.974+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/1872391879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d81a6280 con 0x7fb9d810b2b0 2024-05-04T03:16:09.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.974+0000 7fb9d659c700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d810a960 0x7fb9d81a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:09.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.974+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9d810b2b0 0x7fb9d81a1950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800eee0 tx=0x7fb9c800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:09.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.974+0000 7fb9cf7fe700 1 -- 172.21.15.39:0/1872391879 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800c9b0 con 0x7fb9d810b2b0 2024-05-04T03:16:09.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.975+0000 7fb9cf7fe700 1 -- 172.21.15.39:0/1872391879 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9c8019070 con 0x7fb9d810b2b0 2024-05-04T03:16:09.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.975+0000 7fb9cf7fe700 1 -- 172.21.15.39:0/1872391879 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8014480 con 0x7fb9d810b2b0 2024-05-04T03:16:09.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.975+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1872391879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d81a6510 con 0x7fb9d810b2b0 2024-05-04T03:16:09.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.975+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1872391879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d81ade90 con 0x7fb9d810b2b0 2024-05-04T03:16:09.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.978+0000 7fb9cf7fe700 1 -- 172.21.15.39:0/1872391879 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb9c8012050 con 0x7fb9d810b2b0 2024-05-04T03:16:09.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.979+0000 7fb9cf7fe700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9c40b2900 0x7fb9c40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.980+0000 7fb9cf7fe700 1 -- 172.21.15.39:0/1872391879 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb9c80e2180 con 0x7fb9d810b2b0 2024-05-04T03:16:09.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.980+0000 7fb9d659c700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9c40b2900 0x7fb9c40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.980+0000 7fb9eb612b80 1 Processor -- start 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9eb612b80 1 -- start start 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 0x562a2559cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9d659c700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9c40b2900 0x7fb9c40b4e10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009750 tx=0x7fb9c000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9eb612b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562a254bfad0 con 0x562a2559cb60 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9eb612b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562a25424f30 con 0x562a255a7cd0 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9eb612b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562a254bba90 con 0x562a2559d4b0 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9d5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9d5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:39178/0 (socket says 172.21.15.39:39178) 2024-05-04T03:16:09.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.981+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/2456930777 learned_addr learned my addr 172.21.15.39:0/2456930777 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d6d9d700 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d659c700 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 0x562a2559cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/2456930777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 msgr2=0x562a255aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/2456930777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 msgr2=0x562a2559cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 0x562a2559cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/2456930777 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2540ca60 con 0x562a2559d4b0 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d6d9d700 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:09.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.982+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a7790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80a9740 tx=0x7fb9c80a8f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5652507692f944e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:09.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.983+0000 7fb9ccff9700 1 -- 172.21.15.39:0/2456930777 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80cc0e0 con 0x562a2559d4b0 2024-05-04T03:16:09.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.983+0000 7fb9ccff9700 1 -- 172.21.15.39:0/2456930777 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9c80a9eb0 con 0x562a2559d4b0 2024-05-04T03:16:09.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.983+0000 7fb9ccff9700 1 -- 172.21.15.39:0/2456930777 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80a9c70 con 0x562a2559d4b0 2024-05-04T03:16:09.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.984+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2456930777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 msgr2=0x562a255a7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:09.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.984+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a7790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80a9740 tx=0x7fb9c80a8f20 comp rx=0 tx=0).stop 2024-05-04T03:16:09.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.984+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2456930777 shutdown_connections 2024-05-04T03:16:09.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.984+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 0x562a2559cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.984+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.984+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2456930777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a7790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.984+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2456930777 >> 172.21.15.39:0/2456930777 conn(0x562a2541ffd0 msgr2=0x562a25598a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:09.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.985+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2456930777 shutdown_connections 2024-05-04T03:16:09.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.985+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2456930777 wait complete. 2024-05-04T03:16:09.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.986+0000 7fb9eb612b80 1 -- start start 2024-05-04T03:16:09.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.986+0000 7fb9eb612b80 1 Processor -- start 2024-05-04T03:16:09.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559cb60 0x562a255a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559d4b0 0x562a255a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559cb60 0x562a255a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559cb60 0x562a255a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:39182/0 (socket says 172.21.15.39:39182) 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255a5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559d4b0 0x562a255a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d5d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559d4b0 0x562a255a52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56932/0 (socket says 172.21.15.39:56932) 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d659c700 1 -- 172.21.15.39:0/2837117431 learned_addr learned my addr 172.21.15.39:0/2837117431 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562a255aaaf0 con 0x562a2559d4b0 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562a25479060 con 0x562a255a7cd0 2024-05-04T03:16:09.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562a255aac70 con 0x562a2559cb60 2024-05-04T03:16:09.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d6d9d700 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9eb612b80 10 client.0 _ll_register_callbacks cb 0x562a25457ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d659c700 1 -- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 msgr2=0x562a255a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d659c700 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d659c700 1 -- 172.21.15.39:0/2837117431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559d4b0 msgr2=0x562a255a52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d659c700 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559d4b0 0x562a255a52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d659c700 1 -- 172.21.15.39:0/2837117431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a254768f0 con 0x562a2559cb60 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.987+0000 7fb9d6d9d700 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.988+0000 7fb9d659c700 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559cb60 0x562a255a4da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0005910 tx=0x7fb9c0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da865defecf5201 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.988+0000 7fb9bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.988+0000 7fb9bdffb700 1 -- 172.21.15.39:0/2837117431 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0034060 con 0x562a2559cb60 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.988+0000 7fb9bdffb700 1 -- 172.21.15.39:0/2837117431 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb9c000f870 con 0x562a2559cb60 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.988+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c80a9eb0 con 0x562a2559cb60 2024-05-04T03:16:09.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.988+0000 7fb9bdffb700 1 -- 172.21.15.39:0/2837117431 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9c0003e70 con 0x562a2559cb60 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.988+0000 7fb9eb612b80 10 client.25573 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.988+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a255b55b0 con 0x562a2559cb60 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9bdffb700 1 -- 172.21.15.39:0/2837117431 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9c002f430 con 0x562a2559cb60 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9bdffb700 1 -- 172.21.15.39:0/2837117431 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c002e070 con 0x562a2559cb60 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9eb612b80 10 client.25573 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9eb612b80 10 client.25573 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9eb612b80 20 client.25573 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562a255b7b40 0x562a255b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9eb612b80 4 client.25573 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:16:09.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb9c80a9eb0 con 0x562a255b7b40 2024-05-04T03:16:09.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.989+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562a255b7b40 0x562a255b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.990+0000 7fb9bdffb700 10 client.25573 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.990+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562a255b7b40 0x562a255b9f60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c8f1c241cc53f69 server_cookie=e824649cba04eafe in_seq=0 out_seq=0 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.991+0000 7fb9bdffb700 1 -- 172.21.15.39:0/2837117431 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb9c80a9eb0 con 0x562a255b7b40 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.991+0000 7fb9bdffb700 10 client.25573 handle_command_reply: tid=0 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.991+0000 7fb9eb612b80 1 client.25573 shutdown 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559cb60 msgr2=0x562a255a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559cb60 0x562a255a4da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0005910 tx=0x7fb9c0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 shutdown_connections 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562a255b7b40 0x562a255b9f60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a255a7cd0 0x562a255a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559cb60 0x562a255a4da0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/2837117431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559d4b0 0x562a255a52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:09.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 >> 172.21.15.39:0/2837117431 conn(0x562a2541ffd0 msgr2=0x562a2559e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:09.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9bdffb700 0 client.25573 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:09.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 shutdown_connections 2024-05-04T03:16:09.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.992+0000 7fb9eb612b80 1 -- 172.21.15.39:0/2837117431 wait complete. 2024-05-04T03:16:09.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:09.993+0000 7fb9eb612b80 20 client.25573 trim_cache size 0 max 16384 2024-05-04T03:16:10.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.030+0000 7fb9eb612b80 1 Processor -- start 2024-05-04T03:16:10.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9eb612b80 1 -- start start 2024-05-04T03:16:10.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559cb60 0x562a2559f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a255a7cd0 0x562a255b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9d659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559cb60 0x562a2559f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9eb612b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b5280 con 0x562a255a7cd0 2024-05-04T03:16:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9eb612b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b6dc0 con 0x562a2559cb60 2024-05-04T03:16:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9eb612b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b5950 con 0x562a2559d4b0 2024-05-04T03:16:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9d659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559cb60 0x562a2559f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:59474/0 (socket says 172.21.15.39:59474) 2024-05-04T03:16:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9d659c700 1 -- 172.21.15.39:0/95969560 learned_addr learned my addr 172.21.15.39:0/95969560 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.031+0000 7fb9d6d9d700 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a255a7cd0 0x562a255b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9d659c700 1 -- 172.21.15.39:0/95969560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 msgr2=0x562a255a6b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9d659c700 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9d659c700 1 -- 172.21.15.39:0/95969560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a255a7cd0 msgr2=0x562a255b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9d659c700 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a255a7cd0 0x562a255b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9d659c700 1 -- 172.21.15.39:0/95969560 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2559d8c0 con 0x562a2559cb60 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9d6d9d700 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a255a7cd0 0x562a255b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9d659c700 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559cb60 0x562a2559f000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0000c00 tx=0x7fb9c00092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43107a8ed17ba414 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9bffff700 1 -- 172.21.15.39:0/95969560 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0034070 con 0x562a2559cb60 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.032+0000 7fb9bffff700 1 -- 172.21.15.39:0/95969560 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9c000f670 con 0x562a2559cb60 2024-05-04T03:16:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.033+0000 7fb9bffff700 1 -- 172.21.15.39:0/95969560 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c003e080 con 0x562a2559cb60 2024-05-04T03:16:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.034+0000 7fb9eb612b80 1 -- 172.21.15.39:0/95969560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559cb60 msgr2=0x562a2559f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.034+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559cb60 0x562a2559f000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0000c00 tx=0x7fb9c00092d0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.034+0000 7fb9eb612b80 1 -- 172.21.15.39:0/95969560 shutdown_connections 2024-05-04T03:16:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.034+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a255a7cd0 0x562a255b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.034+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559cb60 0x562a2559f000 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.034+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/95969560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2559d4b0 0x562a255a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.034+0000 7fb9eb612b80 1 -- 172.21.15.39:0/95969560 >> 172.21.15.39:0/95969560 conn(0x562a2541ffd0 msgr2=0x562a255a6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.035+0000 7fb9eb612b80 1 -- 172.21.15.39:0/95969560 shutdown_connections 2024-05-04T03:16:10.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.035+0000 7fb9eb612b80 1 -- 172.21.15.39:0/95969560 wait complete. 2024-05-04T03:16:10.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.036+0000 7fb9eb612b80 1 -- start start 2024-05-04T03:16:10.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.036+0000 7fb9eb612b80 1 Processor -- start 2024-05-04T03:16:10.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 0x562a255b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559d4b0 0x562a255aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9eb612b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a255a7cd0 0x562a255ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9d5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559d4b0 0x562a255aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9d5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559d4b0 0x562a255aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:59488/0 (socket says 172.21.15.39:59488) 2024-05-04T03:16:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9d659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 0x562a255b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/3803373161 learned_addr learned my addr 172.21.15.39:0/3803373161 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:10.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b66f0 con 0x562a2559cb60 2024-05-04T03:16:10.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b6020 con 0x562a2559d4b0 2024-05-04T03:16:10.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b3220 con 0x562a255a7cd0 2024-05-04T03:16:10.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.037+0000 7fb9eb612b80 10 client.0 _ll_register_callbacks cb 0x562a25457ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.038+0000 7fb9d6d9d700 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a255a7cd0 0x562a255ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.038+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/3803373161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a255a7cd0 msgr2=0x562a255ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.038+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a255a7cd0 0x562a255ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.038+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/3803373161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 msgr2=0x562a255b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.038+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 0x562a255b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.038+0000 7fb9d5d9b700 1 -- 172.21.15.39:0/3803373161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a255adb50 con 0x562a2559d4b0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.038+0000 7fb9d5d9b700 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559d4b0 0x562a255aced0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80a94b0 tx=0x7fb9c8007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7541c1a88b57efe server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.038+0000 7fb9bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.039+0000 7fb9bdffb700 1 -- 172.21.15.39:0/3803373161 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80a9eb0 con 0x562a2559d4b0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.039+0000 7fb9bdffb700 1 -- 172.21.15.39:0/3803373161 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb9c80a9c20 con 0x562a2559d4b0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.039+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c000f670 con 0x562a2559d4b0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9bdffb700 1 -- 172.21.15.39:0/3803373161 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb9c80cc0b0 con 0x562a2559d4b0 2024-05-04T03:16:10.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9eb612b80 10 client.25574 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:10.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a25634130 con 0x562a2559d4b0 2024-05-04T03:16:10.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9bdffb700 1 -- 172.21.15.39:0/3803373161 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9c000f670 con 0x562a2559d4b0 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9bdffb700 1 -- 172.21.15.39:0/3803373161 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c80a8a10 con 0x562a2559d4b0 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9eb612b80 10 client.25574 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9eb612b80 10 client.25574 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9eb612b80 20 client.25574 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562a256344c0 0x562a256368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9eb612b80 4 client.25574 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.040+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x562a254cd4b0 con 0x562a256344c0 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.041+0000 7fb9d659c700 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562a256344c0 0x562a256368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.041+0000 7fb9bdffb700 10 client.25574 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.042+0000 7fb9d659c700 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562a256344c0 0x562a256368e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69510016e4777fec server_cookie=d9ac8e4838b3aa3a in_seq=0 out_seq=0 2024-05-04T03:16:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 -- 172.21.15.39:0/250308181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474103c70 msgr2=0x7f2474104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 --2- 172.21.15.39:0/250308181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474103c70 0x7f2474104970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f246c009900 tx=0x7f246c02a410 comp rx=0 tx=0).stop 2024-05-04T03:16:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 -- 172.21.15.39:0/250308181 shutdown_connections 2024-05-04T03:16:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 --2- 172.21.15.39:0/250308181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474109bb0 0x7f247410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 --2- 172.21.15.39:0/250308181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474104eb0 0x7f24741052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 --2- 172.21.15.39:0/250308181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474103c70 0x7f2474104970 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 -- 172.21.15.39:0/250308181 >> 172.21.15.39:0/250308181 conn(0x7f24740fee40 msgr2=0x7f2474101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 -- 172.21.15.39:0/250308181 shutdown_connections 2024-05-04T03:16:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.197+0000 7f2478a49640 1 -- 172.21.15.39:0/250308181 wait complete. 2024-05-04T03:16:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2478a49640 1 Processor -- start 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2478a49640 1 -- start start 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2478a49640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474103c70 0x7f247419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2478a49640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474104eb0 0x7f247419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2478a49640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474109bb0 0x7f24741a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2478a49640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f247410f600 con 0x7f2474109bb0 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2478a49640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f247410f480 con 0x7f2474103c70 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2478a49640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f247410f780 con 0x7f2474104eb0 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2472d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474109bb0 0x7f24741a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2472d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474109bb0 0x7f24741a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56966/0 (socket says 172.21.15.39:56966) 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2472d9d640 1 -- 172.21.15.39:0/2569473796 learned_addr learned my addr 172.21.15.39:0/2569473796 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2471d9b640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474104eb0 0x7f247419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f247259c640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474103c70 0x7f247419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2472d9d640 1 -- 172.21.15.39:0/2569473796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474104eb0 msgr2=0x7f247419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2472d9d640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474104eb0 0x7f247419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2472d9d640 1 -- 172.21.15.39:0/2569473796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474103c70 msgr2=0x7f247419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2472d9d640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474103c70 0x7f247419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f2472d9d640 1 -- 172.21.15.39:0/2569473796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24741a1df0 con 0x7f2474109bb0 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.198+0000 7f247259c640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474103c70 0x7f247419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.199+0000 7f2472d9d640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474109bb0 0x7f24741a1780 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f246400eb10 tx=0x7f246400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.199+0000 7f246b7fe640 1 -- 172.21.15.39:0/2569473796 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464004b20 con 0x7f2474109bb0 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.199+0000 7f2478a49640 1 -- 172.21.15.39:0/2569473796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24741a20e0 con 0x7f2474109bb0 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.199+0000 7f246b7fe640 1 -- 172.21.15.39:0/2569473796 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2464004cc0 con 0x7f2474109bb0 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.199+0000 7f246b7fe640 1 -- 172.21.15.39:0/2569473796 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464010690 con 0x7f2474109bb0 2024-05-04T03:16:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.199+0000 7f2478a49640 1 -- 172.21.15.39:0/2569473796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741a25e0 con 0x7f2474109bb0 2024-05-04T03:16:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.201+0000 7f247f9d1740 1 Processor -- start 2024-05-04T03:16:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.201+0000 7f246b7fe640 1 -- 172.21.15.39:0/2569473796 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2464005020 con 0x7f2474109bb0 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f247f9d1740 1 -- start start 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7e62f0 0x5596fabb0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7ee4d0 0x5596fabb7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f247f9d1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5596fab155d0 con 0x5596fa80f1b0 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f247f9d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5596faace890 con 0x5596fa7e62f0 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f247f9d1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5596fab11590 con 0x5596fa7ee4d0 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabb0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56968/0 (socket says 172.21.15.39:56968) 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 -- 172.21.15.39:0/2581392704 learned_addr learned my addr 172.21.15.39:0/2581392704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7ee4d0 0x5596fabb7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f247259c640 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7e62f0 0x5596fabb0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f246b7fe640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24600b27e0 0x7f24600b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 -- 172.21.15.39:0/2581392704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7ee4d0 msgr2=0x5596fabb7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7ee4d0 0x5596fabb7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 -- 172.21.15.39:0/2581392704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7e62f0 msgr2=0x5596fabb0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7e62f0 0x5596fabb0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.202+0000 7f2471d9b640 1 -- 172.21.15.39:0/2581392704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596fa827210 con 0x5596fa80f1b0 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.203+0000 7f247259c640 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7e62f0 0x5596fabb0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.203+0000 7f2472d9d640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24600b27e0 0x7f24600b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.203+0000 7f246b7fe640 1 -- 172.21.15.39:0/2569473796 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f24640e2860 con 0x7f2474109bb0 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.203+0000 7f2471d9b640 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabb0a60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f245c013820 tx=0x7f245c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398e33826777698f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.203+0000 7f2468ff9640 1 -- 172.21.15.39:0/2581392704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c013da0 con 0x5596fa80f1b0 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.203+0000 7f2468ff9640 1 -- 172.21.15.39:0/2581392704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f245c013170 con 0x5596fa80f1b0 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.203+0000 7f2468ff9640 1 -- 172.21.15.39:0/2581392704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c004c80 con 0x5596fa80f1b0 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 -- 172.21.15.39:0/2581392704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 msgr2=0x5596fabb0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabb0a60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f245c013820 tx=0x7f245c0097a0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f2472d9d640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24600b27e0 0x7f24600b4cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f24640a9e20 tx=0x7f24640d94a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 -- 172.21.15.39:0/2581392704 shutdown_connections 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7ee4d0 0x5596fabb7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7e62f0 0x5596fabb0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 --2- 172.21.15.39:0/2581392704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabb0a60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 -- 172.21.15.39:0/2581392704 >> 172.21.15.39:0/2581392704 conn(0x5596fa803300 msgr2=0x5596fabac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 -- 172.21.15.39:0/2581392704 shutdown_connections 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 -- 172.21.15.39:0/2581392704 wait complete. 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 -- start start 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.204+0000 7f247f9d1740 1 Processor -- start 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247f9d1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5596fabb88d0 con 0x5596fa80f1b0 2024-05-04T03:16:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247f9d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5596faad2370 con 0x5596fa7ee4d0 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247f9d1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5596fabb8a50 con 0x5596fa7e62f0 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247f9d1740 10 client.0 _ll_register_callbacks cb 0x5596fa826a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f2471d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f2471d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56970/0 (socket says 172.21.15.39:56970) 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f2471d9b640 1 -- 172.21.15.39:0/836243111 learned_addr learned my addr 172.21.15.39:0/836243111 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247259c640 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f2471d9b640 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247259c640 1 -- 172.21.15.39:0/836243111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 msgr2=0x5596fabba5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247259c640 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabba5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247259c640 1 -- 172.21.15.39:0/836243111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 msgr2=0x5596fabc1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247259c640 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247259c640 1 -- 172.21.15.39:0/836243111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596fa8f3620 con 0x5596fa7ee4d0 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f2471d9b640 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f247259c640 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbaaf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f246c02a930 tx=0x7f246c031830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676bbe77079da5ba server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.205+0000 7f2459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f2459ffb640 1 -- 172.21.15.39:0/836243111 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c035030 con 0x5596fa7ee4d0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 1 -- 172.21.15.39:0/836243111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f245c013170 con 0x5596fa7ee4d0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f2459ffb640 1 -- 172.21.15.39:0/836243111 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f246c004240 con 0x5596fa7ee4d0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f2459ffb640 1 -- 172.21.15.39:0/836243111 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f246c004440 con 0x5596fa7ee4d0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 10 client.25586 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 1 -- 172.21.15.39:0/836243111 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5596fabc49f0 con 0x5596fa7ee4d0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f2459ffb640 1 -- 172.21.15.39:0/836243111 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f245c013170 con 0x5596fa7ee4d0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f2459ffb640 1 -- 172.21.15.39:0/836243111 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f246c03c560 con 0x5596fa7ee4d0 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 10 client.25586 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 20 client.25586 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5596fa7cdd80 0x5596fabc8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 4 client.25586 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:16:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.206+0000 7f247f9d1740 1 -- 172.21.15.39:0/836243111 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596fab231e0 con 0x5596fa7cdd80 2024-05-04T03:16:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.207+0000 7f2471d9b640 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5596fa7cdd80 0x5596fabc8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.207+0000 7f2459ffb640 10 client.25586 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.207+0000 7f2471d9b640 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5596fa7cdd80 0x5596fabc8080 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e442d62a312cd8 server_cookie=b8272f71489d9565 in_seq=0 out_seq=0 2024-05-04T03:16:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f2459ffb640 1 -- 172.21.15.39:0/836243111 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596fab231e0 con 0x5596fa7cdd80 2024-05-04T03:16:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f2459ffb640 10 client.25586 handle_command_reply: tid=0 2024-05-04T03:16:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 client.25586 shutdown 2024-05-04T03:16:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 -- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 msgr2=0x5596fabbaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbaaf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f246c02a930 tx=0x7f246c031830 comp rx=0 tx=0).stop 2024-05-04T03:16:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 -- 172.21.15.39:0/836243111 shutdown_connections 2024-05-04T03:16:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5596fa7cdd80 0x5596fabc8080 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbaaf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 --2- 172.21.15.39:0/836243111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f2459ffb640 0 client.25586 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 -- 172.21.15.39:0/836243111 >> 172.21.15.39:0/836243111 conn(0x5596fa803300 msgr2=0x5596fabaf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 -- 172.21.15.39:0/836243111 shutdown_connections 2024-05-04T03:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 1 -- 172.21.15.39:0/836243111 wait complete. 2024-05-04T03:16:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.208+0000 7f247f9d1740 20 client.25586 trim_cache size 0 max 16384 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247f9d1740 1 Processor -- start 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247f9d1740 1 -- start start 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabb41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa7ee4d0 0x5596fabc1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa80f1b0 0x5596fabaf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247f9d1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24741ab260 con 0x5596fa7ee4d0 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247f9d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24741acda0 con 0x5596fa80f1b0 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247f9d1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24741ab930 con 0x5596fa7e62f0 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.223+0000 7f247259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa7ee4d0 0x5596fabc1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f247259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa7ee4d0 0x5596fabc1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56986/0 (socket says 172.21.15.39:56986) 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f247259c640 1 -- 172.21.15.39:0/3122712379 learned_addr learned my addr 172.21.15.39:0/3122712379 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f2471d9b640 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabb41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f2471d9b640 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa80f1b0 0x5596fabaf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f247259c640 1 -- 172.21.15.39:0/3122712379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 msgr2=0x5596fabb41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f247259c640 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabb41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f247259c640 1 -- 172.21.15.39:0/3122712379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa80f1b0 msgr2=0x5596fabaf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f247259c640 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa80f1b0 0x5596fabaf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f247259c640 1 -- 172.21.15.39:0/3122712379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596fabb5490 con 0x5596fa7ee4d0 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f2471d9b640 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa80f1b0 0x5596fabaf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f247259c640 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa7ee4d0 0x5596fabc1970 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f246c03cac0 tx=0x7f246c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07e34d7f9883e9b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f245bfff640 1 -- 172.21.15.39:0/3122712379 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c000e60 con 0x5596fa7ee4d0 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f245bfff640 1 -- 172.21.15.39:0/3122712379 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f246c040070 con 0x5596fa7ee4d0 2024-05-04T03:16:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.224+0000 7f245bfff640 1 -- 172.21.15.39:0/3122712379 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c053070 con 0x5596fa7ee4d0 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 -- 172.21.15.39:0/3122712379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa7ee4d0 msgr2=0x5596fabc1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa7ee4d0 0x5596fabc1970 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f246c03cac0 tx=0x7f246c000bc0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 -- 172.21.15.39:0/3122712379 shutdown_connections 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa80f1b0 0x5596fabaf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa7ee4d0 0x5596fabc1970 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 --2- 172.21.15.39:0/3122712379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabb41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 -- 172.21.15.39:0/3122712379 >> 172.21.15.39:0/3122712379 conn(0x5596fa803300 msgr2=0x5596fabb3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 -- 172.21.15.39:0/3122712379 shutdown_connections 2024-05-04T03:16:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 -- 172.21.15.39:0/3122712379 wait complete. 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 -- start start 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.225+0000 7f247f9d1740 1 Processor -- start 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabbef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247f9d1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247f9d1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24741ac6d0 con 0x5596fa80f1b0 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247f9d1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f24741ac000 con 0x5596fa7ee4d0 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247f9d1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24741a9240 con 0x5596fa7e62f0 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247f9d1740 10 client.0 _ll_register_callbacks cb 0x7f245c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56988/0 (socket says 172.21.15.39:56988) 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 -- 172.21.15.39:0/702395261 learned_addr learned my addr 172.21.15.39:0/702395261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabbef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247259c640 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 -- 172.21.15.39:0/702395261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 msgr2=0x5596fabbef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabbef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 -- 172.21.15.39:0/702395261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 msgr2=0x5596fabbf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 -- 172.21.15.39:0/702395261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596fabc08e0 con 0x5596fa80f1b0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2471d9b640 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc0320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5596fabc8870 tx=0x7f246c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f397c11711b333d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2459ffb640 1 -- 172.21.15.39:0/702395261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c004090 con 0x5596fa80f1b0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f2459ffb640 1 -- 172.21.15.39:0/702395261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f245c02d040 con 0x5596fa80f1b0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.226+0000 7f247f9d1740 1 -- 172.21.15.39:0/702395261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f246c040070 con 0x5596fa80f1b0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f2459ffb640 1 -- 172.21.15.39:0/702395261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f245c004230 con 0x5596fa80f1b0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247f9d1740 10 client.15951 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247f9d1740 1 -- 172.21.15.39:0/702395261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5596fabc1b70 con 0x5596fa80f1b0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f2459ffb640 1 -- 172.21.15.39:0/702395261 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f246c040070 con 0x5596fa80f1b0 2024-05-04T03:16:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f2459ffb640 1 -- 172.21.15.39:0/702395261 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f245c02f2c0 con 0x5596fa80f1b0 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247f9d1740 10 client.15951 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247f9d1740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247f9d1740 20 client.15951 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247f9d1740 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5596fa7a31c0 0x5596fac329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247f9d1740 4 client.15951 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247f9d1740 1 -- 172.21.15.39:0/702395261 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596fab1d050 con 0x5596fa7a31c0 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.227+0000 7f247259c640 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5596fa7a31c0 0x5596fac329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.228+0000 7f247259c640 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5596fa7a31c0 0x5596fac329a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f61a31a08b22431 server_cookie=b53e50dfc768c7da in_seq=0 out_seq=0 2024-05-04T03:16:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.228+0000 7f2459ffb640 10 client.15951 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f2459ffb640 1 -- 172.21.15.39:0/702395261 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596fab1d050 con 0x5596fa7a31c0 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f2459ffb640 10 client.15951 handle_command_reply: tid=0 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 client.15951 shutdown 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 -- 172.21.15.39:0/702395261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 msgr2=0x5596fabc0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc0320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5596fabc8870 tx=0x7f246c005e30 comp rx=0 tx=0).stop 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 -- 172.21.15.39:0/702395261 shutdown_connections 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5596fa7a31c0 0x5596fac329a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5596fa80f1b0 0x5596fabc0320 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5596fa7ee4d0 0x5596fabbf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f2459ffb640 0 client.15951 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 --2- 172.21.15.39:0/702395261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596fa7e62f0 0x5596fabbef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.234+0000 7f247f9d1740 1 -- 172.21.15.39:0/702395261 >> 172.21.15.39:0/702395261 conn(0x5596fa803300 msgr2=0x5596fabaf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f247f9d1740 1 -- 172.21.15.39:0/702395261 shutdown_connections 2024-05-04T03:16:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f247f9d1740 1 -- 172.21.15.39:0/702395261 wait complete. 2024-05-04T03:16:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f247f9d1740 20 client.15951 trim_cache size 0 max 16384 2024-05-04T03:16:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:16:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:16:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:16:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:16:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 -- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24600b27e0 msgr2=0x7f24600b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24600b27e0 0x7f24600b4cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f24640a9e20 tx=0x7f24640d94a0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 -- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474109bb0 msgr2=0x7f24741a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474109bb0 0x7f24741a1780 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f246400eb10 tx=0x7f246400c2d0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 -- 172.21.15.39:0/2569473796 shutdown_connections 2024-05-04T03:16:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f24600b27e0 0x7f24600b4cf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2474109bb0 0x7f24741a1780 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2474104eb0 0x7f247419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 --2- 172.21.15.39:0/2569473796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2474103c70 0x7f247419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 -- 172.21.15.39:0/2569473796 >> 172.21.15.39:0/2569473796 conn(0x7f24740fee40 msgr2=0x7f2474105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 -- 172.21.15.39:0/2569473796 shutdown_connections 2024-05-04T03:16:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:10.235+0000 7f2478a49640 1 -- 172.21.15.39:0/2569473796 wait complete. 2024-05-04T03:16:10.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9bdffb700 1 -- 172.21.15.39:0/3803373161 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562a254cd4b0 con 0x562a256344c0 2024-05-04T03:16:10.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9bdffb700 10 client.25574 handle_command_reply: tid=0 2024-05-04T03:16:10.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 client.25574 shutdown 2024-05-04T03:16:10.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559d4b0 msgr2=0x562a255aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559d4b0 0x562a255aced0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80a94b0 tx=0x7fb9c8007980 comp rx=0 tx=0).stop 2024-05-04T03:16:10.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 shutdown_connections 2024-05-04T03:16:10.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562a256344c0 0x562a256368e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a2559cb60 0x562a255b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a255a7cd0 0x562a255ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9bdffb700 0 client.25574 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 --2- 172.21.15.39:0/3803373161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562a2559d4b0 0x562a255aced0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 >> 172.21.15.39:0/3803373161 conn(0x562a2541ffd0 msgr2=0x562a25598630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 shutdown_connections 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 1 -- 172.21.15.39:0/3803373161 wait complete. 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.393+0000 7fb9eb612b80 20 client.25574 trim_cache size 0 max 16384 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:16:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 14, 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "d399bd5f": { 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 14, 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 90.0, 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:16:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:10.398 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9c40b2900 msgr2=0x7fb9c40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9c40b2900 0x7fb9c40b4e10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009750 tx=0x7fb9c000b260 comp rx=0 tx=0).stop 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1872391879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9d810b2b0 msgr2=0x7fb9d81a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9d810b2b0 0x7fb9d81a1950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800eee0 tx=0x7fb9c800c3a0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1872391879 shutdown_connections 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb9c40b2900 0x7fb9c40b4e10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9d810a960 0x7fb9d81a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9d8110990 0x7fb9d81a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 --2- 172.21.15.39:0/1872391879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9d810b2b0 0x7fb9d81a1950 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:10.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1872391879 >> 172.21.15.39:0/1872391879 conn(0x7fb9d8076d30 msgr2=0x7fb9d8109260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:10.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1872391879 shutdown_connections 2024-05-04T03:16:10.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:10.394+0000 7fb9dcbfe700 1 -- 172.21.15.39:0/1872391879 wait complete. 2024-05-04T03:16:10.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:10 smithi066 ceph-mon[33264]: pgmap v547: 97 pgs: 97 active+clean; 1000 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 443 op/s 2024-05-04T03:16:10.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:10 smithi167 ceph-mon[33614]: pgmap v547: 97 pgs: 97 active+clean; 1000 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 443 op/s 2024-05-04T03:16:10.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:10 smithi039 ceph-mon[29715]: pgmap v547: 97 pgs: 97 active+clean; 1000 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 443 op/s 2024-05-04T03:16:11.344 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd399bd5f', quiesce took 7.6 seconds. Will release after: 30.0 2024-05-04T03:16:11.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 140.0 s, 871.0 tps, lat 87.333 ms stddev 97.362 2024-05-04T03:16:12.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:12 smithi066 ceph-mon[33264]: pgmap v548: 97 pgs: 97 active+clean; 1008 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-05-04T03:16:12.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:12 smithi167 ceph-mon[33614]: pgmap v548: 97 pgs: 97 active+clean; 1008 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-05-04T03:16:12.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:12 smithi039 ceph-mon[29715]: pgmap v548: 97 pgs: 97 active+clean; 1008 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-05-04T03:16:14.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:14 smithi066 ceph-mon[33264]: pgmap v549: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.0 MiB/s wr, 762 op/s 2024-05-04T03:16:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:14 smithi167 ceph-mon[33614]: pgmap v549: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.0 MiB/s wr, 762 op/s 2024-05-04T03:16:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:14 smithi039 ceph-mon[29715]: pgmap v549: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.0 MiB/s wr, 762 op/s 2024-05-04T03:16:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:16 smithi167 ceph-mon[33614]: pgmap v550: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-05-04T03:16:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:16 smithi039 ceph-mon[29715]: pgmap v550: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-05-04T03:16:16.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 145.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:16:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:16 smithi066 ceph-mon[33264]: pgmap v550: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-05-04T03:16:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:18 smithi167 ceph-mon[33614]: pgmap v551: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 772 op/s 2024-05-04T03:16:18.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:18 smithi039 ceph-mon[29715]: pgmap v551: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 772 op/s 2024-05-04T03:16:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:18 smithi066 ceph-mon[33264]: pgmap v551: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 772 op/s 2024-05-04T03:16:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:20 smithi167 ceph-mon[33614]: pgmap v552: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.0 MiB/s wr, 662 op/s 2024-05-04T03:16:20.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:20 smithi039 ceph-mon[29715]: pgmap v552: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.0 MiB/s wr, 662 op/s 2024-05-04T03:16:20.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:20 smithi066 ceph-mon[33264]: pgmap v552: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.0 MiB/s wr, 662 op/s 2024-05-04T03:16:21.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 150.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:16:22.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:22 smithi167 ceph-mon[33614]: pgmap v553: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.3 MiB/s wr, 384 op/s 2024-05-04T03:16:22.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:22 smithi039 ceph-mon[29715]: pgmap v553: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.3 MiB/s wr, 384 op/s 2024-05-04T03:16:22.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:22 smithi066 ceph-mon[33264]: pgmap v553: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.3 MiB/s wr, 384 op/s 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 -- 172.21.15.39:0/3184877118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103c90 msgr2=0x7f6788104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 --2- 172.21.15.39:0/3184877118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103c90 0x7f6788104160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f677c009900 tx=0x7f677c02a410 comp rx=0 tx=0).stop 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 -- 172.21.15.39:0/3184877118 shutdown_connections 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 --2- 172.21.15.39:0/3184877118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67881047f0 0x7f678810af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 --2- 172.21.15.39:0/3184877118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103c90 0x7f6788104160 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 --2- 172.21.15.39:0/3184877118 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6788103130 0x7f6788103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 -- 172.21.15.39:0/3184877118 >> 172.21.15.39:0/3184877118 conn(0x7f67880fd4b0 msgr2=0x7f67880ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 -- 172.21.15.39:0/3184877118 shutdown_connections 2024-05-04T03:16:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.349+0000 7f678d134640 1 -- 172.21.15.39:0/3184877118 wait complete. 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f678d134640 1 Processor -- start 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f678d134640 1 -- start start 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f678d134640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103130 0x7f678819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f678d134640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6788103c90 0x7f678819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f678d134640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f678819f560 0x7f67881a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f678d134640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f678810f870 con 0x7f6788103130 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f678d134640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f678810f6f0 con 0x7f678819f560 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f678d134640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f678810f9f0 con 0x7f6788103c90 2024-05-04T03:16:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.350+0000 7f6786d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103130 0x7f678819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f6786d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103130 0x7f678819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59838/0 (socket says 172.21.15.39:59838) 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f6786d9d640 1 -- 172.21.15.39:0/1735797520 learned_addr learned my addr 172.21.15.39:0/1735797520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f678759e640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f678819f560 0x7f67881a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f678659c640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6788103c90 0x7f678819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f6786d9d640 1 -- 172.21.15.39:0/1735797520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6788103c90 msgr2=0x7f678819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f6786d9d640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6788103c90 0x7f678819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f6786d9d640 1 -- 172.21.15.39:0/1735797520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f678819f560 msgr2=0x7f67881a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f6786d9d640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f678819f560 0x7f67881a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f6786d9d640 1 -- 172.21.15.39:0/1735797520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67881a1ff0 con 0x7f6788103130 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f678759e640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f678819f560 0x7f67881a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f6786d9d640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103130 0x7f678819a0a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f677800d770 tx=0x7f677800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f676ffff640 1 -- 172.21.15.39:0/1735797520 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778017070 con 0x7f6788103130 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f678d134640 1 -- 172.21.15.39:0/1735797520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881a22e0 con 0x7f6788103130 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f676ffff640 1 -- 172.21.15.39:0/1735797520 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6778009c30 con 0x7f6788103130 2024-05-04T03:16:23.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f676ffff640 1 -- 172.21.15.39:0/1735797520 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677800f620 con 0x7f6788103130 2024-05-04T03:16:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.351+0000 7f678d134640 1 -- 172.21.15.39:0/1735797520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a2880 con 0x7f6788103130 2024-05-04T03:16:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f6793fe5740 1 Processor -- start 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f6793fe5740 1 -- start start 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f676ffff640 1 -- 172.21.15.39:0/1735797520 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f677801a070 con 0x7f6788103130 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 0x564d07450e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070822f0 0x564d074513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d0708a4d0 0x564d074582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f6793fe5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564d073b15d0 con 0x564d070822f0 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f6793fe5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564d0736a890 con 0x564d0708a4d0 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.354+0000 7f6793fe5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564d073ad590 con 0x564d070ab1b0 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070822f0 0x564d074513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070822f0 0x564d074513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59852/0 (socket says 172.21.15.39:59852) 2024-05-04T03:16:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 -- 172.21.15.39:0/3643898816 learned_addr learned my addr 172.21.15.39:0/3643898816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f678759e640 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 0x564d07450e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f678659c640 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d0708a4d0 0x564d074582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 -- 172.21.15.39:0/3643898816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 msgr2=0x564d07450e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 0x564d07450e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 -- 172.21.15.39:0/3643898816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d0708a4d0 msgr2=0x564d074582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d0708a4d0 0x564d074582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f676ffff640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f67700b27e0 0x7f67700b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 -- 172.21.15.39:0/3643898816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d070c3210 con 0x564d070822f0 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f678659c640 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d0708a4d0 0x564d074582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f678759e640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f67700b27e0 0x7f67700b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f676ffff640 1 -- 172.21.15.39:0/1735797520 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f67780e1db0 con 0x7f6788103130 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.355+0000 7f6786d9d640 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070822f0 0x564d074513f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f67780b8f60 tx=0x7f67780a5af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac7c89fb96e76a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.356+0000 7f676d7fa640 1 -- 172.21.15.39:0/3643898816 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778021070 con 0x564d070822f0 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.356+0000 7f676d7fa640 1 -- 172.21.15.39:0/3643898816 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67780b8fc0 con 0x564d070822f0 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.356+0000 7f676d7fa640 1 -- 172.21.15.39:0/3643898816 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67780a8d30 con 0x564d070822f0 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- 172.21.15.39:0/3643898816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070822f0 msgr2=0x564d074513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f678759e640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f67700b27e0 0x7f67700b4cf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x564d07452040 tx=0x7f6780009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070822f0 0x564d074513f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f67780b8f60 tx=0x7f67780a5af0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- 172.21.15.39:0/3643898816 shutdown_connections 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d0708a4d0 0x564d074582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070822f0 0x564d074513f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3643898816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 0x564d07450e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- 172.21.15.39:0/3643898816 >> 172.21.15.39:0/3643898816 conn(0x564d0709f300 msgr2=0x564d074483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- 172.21.15.39:0/3643898816 shutdown_connections 2024-05-04T03:16:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- 172.21.15.39:0/3643898816 wait complete. 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- start start 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 Processor -- start 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d0708a4d0 0x564d0745af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 0x564d07461ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564d07458d40 con 0x564d0708a4d0 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564d0736e370 con 0x564d070822f0 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564d07458ec0 con 0x564d070ab1b0 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.357+0000 7f6793fe5740 10 client.0 _ll_register_callbacks cb 0x564d070c2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d0708a4d0 0x564d0745af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d0708a4d0 0x564d0745af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59856/0 (socket says 172.21.15.39:59856) 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 -- 172.21.15.39:0/3528238614 learned_addr learned my addr 172.21.15.39:0/3528238614 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678759e640 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 0x564d07461ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f6786d9d640 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 -- 172.21.15.39:0/3528238614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 msgr2=0x564d07461ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 0x564d07461ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 -- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 msgr2=0x564d0745aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 -- 172.21.15.39:0/3528238614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0718f620 con 0x564d0708a4d0 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f6786d9d640 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f678659c640 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d0708a4d0 0x564d0745af60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f677c005ab0 tx=0x7f677c031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd10cde6c045893 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f676aff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f676aff5640 1 -- 172.21.15.39:0/3528238614 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c035030 con 0x564d0708a4d0 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f676aff5640 1 -- 172.21.15.39:0/3528238614 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f677c004240 con 0x564d0708a4d0 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f6793fe5740 1 -- 172.21.15.39:0/3528238614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564d073bf1e0 con 0x564d0708a4d0 2024-05-04T03:16:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f676aff5640 1 -- 172.21.15.39:0/3528238614 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f677c004440 con 0x564d0708a4d0 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f6793fe5740 10 client.15963 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.358+0000 7f6793fe5740 1 -- 172.21.15.39:0/3528238614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d07464e60 con 0x564d0708a4d0 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f676aff5640 1 -- 172.21.15.39:0/3528238614 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x564d073bf1e0 con 0x564d0708a4d0 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f676aff5640 1 -- 172.21.15.39:0/3528238614 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f677c03c640 con 0x564d0708a4d0 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f6793fe5740 10 client.15963 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f6793fe5740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f6793fe5740 20 client.15963 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d07069d80 0x564d074685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f6793fe5740 4 client.15963 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:16:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f6793fe5740 1 -- 172.21.15.39:0/3528238614 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d073b9050 con 0x564d07069d80 2024-05-04T03:16:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.359+0000 7f6786d9d640 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d07069d80 0x564d074685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.360+0000 7f676aff5640 10 client.15963 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.360+0000 7f6786d9d640 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d07069d80 0x564d074685c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b07fc20e66e087 server_cookie=e0c0f70a36353f91 in_seq=0 out_seq=0 2024-05-04T03:16:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f676aff5640 1 -- 172.21.15.39:0/3528238614 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d073b9050 con 0x564d07069d80 2024-05-04T03:16:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f676aff5640 10 client.15963 handle_command_reply: tid=0 2024-05-04T03:16:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 client.15963 shutdown 2024-05-04T03:16:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 -- 172.21.15.39:0/3528238614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d0708a4d0 msgr2=0x564d0745af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d0708a4d0 0x564d0745af60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f677c005ab0 tx=0x7f677c031730 comp rx=0 tx=0).stop 2024-05-04T03:16:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 -- 172.21.15.39:0/3528238614 shutdown_connections 2024-05-04T03:16:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d07069d80 0x564d074685c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d070ab1b0 0x564d07461ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d0708a4d0 0x564d0745af60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 --2- 172.21.15.39:0/3528238614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f676aff5640 0 client.15963 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 -- 172.21.15.39:0/3528238614 >> 172.21.15.39:0/3528238614 conn(0x564d0709f300 msgr2=0x564d0744b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 -- 172.21.15.39:0/3528238614 shutdown_connections 2024-05-04T03:16:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 1 -- 172.21.15.39:0/3528238614 wait complete. 2024-05-04T03:16:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.361+0000 7f6793fe5740 20 client.15963 trim_cache size 0 max 16384 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.376+0000 7f6793fe5740 1 Processor -- start 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.376+0000 7f6793fe5740 1 -- start start 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0744b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 0x564d07461de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07462320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f6793fe5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67881a9160 con 0x564d070ab1b0 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f6793fe5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67881aaca0 con 0x564d070822f0 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f6793fe5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67881a9830 con 0x564d0708a4d0 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07462320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07462320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59866/0 (socket says 172.21.15.39:59866) 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 -- 172.21.15.39:0/2113228080 learned_addr learned my addr 172.21.15.39:0/2113228080 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f6786d9d640 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0744b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678659c640 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 0x564d07461de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 -- 172.21.15.39:0/2113228080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 msgr2=0x564d07461de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 0x564d07461de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 -- 172.21.15.39:0/2113228080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 msgr2=0x564d0744b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0744b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 -- 172.21.15.39:0/2113228080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d07448d00 con 0x564d070ab1b0 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f678759e640 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07462320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6780010700 tx=0x7f6780015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a2d96278530266 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f676cff9640 1 -- 172.21.15.39:0/2113228080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67800207c0 con 0x564d070ab1b0 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f676cff9640 1 -- 172.21.15.39:0/2113228080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6780020df0 con 0x564d070ab1b0 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.377+0000 7f676cff9640 1 -- 172.21.15.39:0/2113228080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780028e30 con 0x564d070ab1b0 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 -- 172.21.15.39:0/2113228080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 msgr2=0x564d07462320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07462320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6780010700 tx=0x7f6780015000 comp rx=0 tx=0).stop 2024-05-04T03:16:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 -- 172.21.15.39:0/2113228080 shutdown_connections 2024-05-04T03:16:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07462320 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 0x564d07461de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 --2- 172.21.15.39:0/2113228080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0744b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 -- 172.21.15.39:0/2113228080 >> 172.21.15.39:0/2113228080 conn(0x564d0709f300 msgr2=0x564d0744b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 -- 172.21.15.39:0/2113228080 shutdown_connections 2024-05-04T03:16:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 -- 172.21.15.39:0/2113228080 wait complete. 2024-05-04T03:16:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.378+0000 7f6793fe5740 1 -- start start 2024-05-04T03:16:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6793fe5740 1 Processor -- start 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 0x564d07460380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6793fe5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07461240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6793fe5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67881aa5d0 con 0x564d070ab1b0 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6793fe5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67881a9f00 con 0x564d070822f0 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6793fe5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67881a7140 con 0x564d0708a4d0 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6793fe5740 10 client.0 _ll_register_callbacks cb 0x7f677c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07461240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07461240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59872/0 (socket says 172.21.15.39:59872) 2024-05-04T03:16:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678759e640 1 -- 172.21.15.39:0/736550215 learned_addr learned my addr 172.21.15.39:0/736550215 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f6786d9d640 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678659c640 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 0x564d07460380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678759e640 1 -- 172.21.15.39:0/736550215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 msgr2=0x564d07460380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678759e640 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 0x564d07460380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678759e640 1 -- 172.21.15.39:0/736550215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 msgr2=0x564d0745fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678759e640 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.379+0000 7f678759e640 1 -- 172.21.15.39:0/736550215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d07461800 con 0x564d070ab1b0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f678759e640 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07461240 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6780010540 tx=0x7f67800285d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71df5196314f1468 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f676aff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f676aff5640 1 -- 172.21.15.39:0/736550215 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780037020 con 0x564d070ab1b0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f676aff5640 1 -- 172.21.15.39:0/736550215 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6780028d80 con 0x564d070ab1b0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 1 -- 172.21.15.39:0/736550215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6780020df0 con 0x564d070ab1b0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f676aff5640 1 -- 172.21.15.39:0/736550215 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6780004a40 con 0x564d070ab1b0 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 10 client.15969 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 1 -- 172.21.15.39:0/736550215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d074d0970 con 0x564d070ab1b0 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f676aff5640 1 -- 172.21.15.39:0/736550215 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6780020df0 con 0x564d070ab1b0 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f676aff5640 1 -- 172.21.15.39:0/736550215 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6780027070 con 0x564d070ab1b0 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 10 client.15969 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 20 client.15969 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d0703f1c0 0x564d074d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 4 client.15969 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.380+0000 7f6793fe5740 1 -- 172.21.15.39:0/736550215 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d073345f0 con 0x564d0703f1c0 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.381+0000 7f6786d9d640 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d0703f1c0 0x564d074d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.381+0000 7f676aff5640 10 client.15969 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.381+0000 7f6786d9d640 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d0703f1c0 0x564d074d31f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0affdb121b3aa97 server_cookie=ff7792ca08b1e638 in_seq=0 out_seq=0 2024-05-04T03:16:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f676aff5640 1 -- 172.21.15.39:0/736550215 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d073345f0 con 0x564d0703f1c0 2024-05-04T03:16:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f676aff5640 10 client.15969 handle_command_reply: tid=0 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 client.15969 shutdown 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 -- 172.21.15.39:0/736550215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 msgr2=0x564d07461240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07461240 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6780010540 tx=0x7f67800285d0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 -- 172.21.15.39:0/736550215 shutdown_connections 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d0703f1c0 0x564d074d31f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d070ab1b0 0x564d07461240 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d0708a4d0 0x564d07460380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 --2- 172.21.15.39:0/736550215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d070822f0 0x564d0745fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 -- 172.21.15.39:0/736550215 >> 172.21.15.39:0/736550215 conn(0x564d0709f300 msgr2=0x564d07447a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f676aff5640 0 client.15969 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 -- 172.21.15.39:0/736550215 shutdown_connections 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 1 -- 172.21.15.39:0/736550215 wait complete. 2024-05-04T03:16:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.390+0000 7f6793fe5740 20 client.15969 trim_cache size 0 max 16384 2024-05-04T03:16:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:16:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:16:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:16:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 -- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f67700b27e0 msgr2=0x7f67700b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f67700b27e0 0x7f67700b4cf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x564d07452040 tx=0x7f6780009700 comp rx=0 tx=0).stop 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 -- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103130 msgr2=0x7f678819a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103130 0x7f678819a0a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f677800d770 tx=0x7f677800dc30 comp rx=0 tx=0).stop 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 -- 172.21.15.39:0/1735797520 shutdown_connections 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f67700b27e0 0x7f67700b4cf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f678819f560 0x7f67881a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6788103c90 0x7f678819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 --2- 172.21.15.39:0/1735797520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6788103130 0x7f678819a0a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 -- 172.21.15.39:0/1735797520 >> 172.21.15.39:0/1735797520 conn(0x7f67880fd4b0 msgr2=0x7f67880ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 -- 172.21.15.39:0/1735797520 shutdown_connections 2024-05-04T03:16:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:23.391+0000 7f678d134640 1 -- 172.21.15.39:0/1735797520 wait complete. 2024-05-04T03:16:24.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:24 smithi167 ceph-mon[33614]: pgmap v554: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2024-05-04T03:16:24.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:24 smithi039 ceph-mon[29715]: pgmap v554: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2024-05-04T03:16:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:24 smithi066 ceph-mon[33264]: pgmap v554: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 2.6 MiB/s wr, 257 op/s 2024-05-04T03:16:26.620 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:26 smithi066 ceph-mon[33264]: pgmap v555: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 421 KiB/s wr, 61 op/s 2024-05-04T03:16:26.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:26 smithi167 ceph-mon[33614]: pgmap v555: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 421 KiB/s wr, 61 op/s 2024-05-04T03:16:26.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:26 smithi039 ceph-mon[29715]: pgmap v555: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 421 KiB/s wr, 61 op/s 2024-05-04T03:16:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 155.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:16:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:28 smithi167 ceph-mon[33614]: pgmap v556: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 173 KiB/s wr, 34 op/s 2024-05-04T03:16:28.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:28 smithi039 ceph-mon[29715]: pgmap v556: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 173 KiB/s wr, 34 op/s 2024-05-04T03:16:28.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:28 smithi066 ceph-mon[33264]: pgmap v556: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 173 KiB/s wr, 34 op/s 2024-05-04T03:16:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:30 smithi167 ceph-mon[33614]: pgmap v557: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 210 KiB/s wr, 38 op/s 2024-05-04T03:16:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:30 smithi039 ceph-mon[29715]: pgmap v557: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 210 KiB/s wr, 38 op/s 2024-05-04T03:16:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:30 smithi066 ceph-mon[33264]: pgmap v557: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 210 KiB/s wr, 38 op/s 2024-05-04T03:16:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 160.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:16:32.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:32 smithi167 ceph-mon[33614]: pgmap v558: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 221 KiB/s wr, 40 op/s 2024-05-04T03:16:32.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:32 smithi039 ceph-mon[29715]: pgmap v558: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 221 KiB/s wr, 40 op/s 2024-05-04T03:16:32.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:32 smithi066 ceph-mon[33264]: pgmap v558: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 221 KiB/s wr, 40 op/s 2024-05-04T03:16:34.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:34 smithi167 ceph-mon[33614]: pgmap v559: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 234 KiB/s wr, 41 op/s 2024-05-04T03:16:34.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:34 smithi039 ceph-mon[29715]: pgmap v559: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 234 KiB/s wr, 41 op/s 2024-05-04T03:16:34.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:34 smithi066 ceph-mon[33264]: pgmap v559: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 234 KiB/s wr, 41 op/s 2024-05-04T03:16:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 -- 172.21.15.39:0/1632788639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8103c70 msgr2=0x7faba8104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 --2- 172.21.15.39:0/1632788639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8103c70 0x7faba8104970 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fab98009900 tx=0x7fab9802a410 comp rx=0 tx=0).stop 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 -- 172.21.15.39:0/1632788639 shutdown_connections 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 --2- 172.21.15.39:0/1632788639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8109bb0 0x7faba810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 --2- 172.21.15.39:0/1632788639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8104eb0 0x7faba81052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 --2- 172.21.15.39:0/1632788639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8103c70 0x7faba8104970 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 -- 172.21.15.39:0/1632788639 >> 172.21.15.39:0/1632788639 conn(0x7faba80fee60 msgr2=0x7faba8101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 -- 172.21.15.39:0/1632788639 shutdown_connections 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.504+0000 7fabadb14640 1 -- 172.21.15.39:0/1632788639 wait complete. 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.505+0000 7fabadb14640 1 Processor -- start 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.505+0000 7fabadb14640 1 -- start start 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7fabadb14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103c70 0x7faba819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7fabadb14640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8104eb0 0x7faba819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7fabadb14640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8109bb0 0x7faba81a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7fabadb14640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faba810f600 con 0x7faba8109bb0 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7fabadb14640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faba810f480 con 0x7faba8103c70 2024-05-04T03:16:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7fabadb14640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faba810f780 con 0x7faba8104eb0 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8109bb0 0x7faba81a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8109bb0 0x7faba81a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35382/0 (socket says 172.21.15.39:35382) 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 -- 172.21.15.39:0/674105780 learned_addr learned my addr 172.21.15.39:0/674105780 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba6ffd640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8104eb0 0x7faba819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba77fe640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103c70 0x7faba819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 -- 172.21.15.39:0/674105780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8104eb0 msgr2=0x7faba819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8104eb0 0x7faba819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 -- 172.21.15.39:0/674105780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103c70 msgr2=0x7faba819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103c70 0x7faba819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 -- 172.21.15.39:0/674105780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba81a1e40 con 0x7faba8109bb0 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.506+0000 7faba7fff640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8109bb0 0x7faba81a17d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faba0009fd0 tx=0x7faba000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.507+0000 7faba4ff9640 1 -- 172.21.15.39:0/674105780 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0017070 con 0x7faba8109bb0 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.507+0000 7faba4ff9640 1 -- 172.21.15.39:0/674105780 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faba0003e00 con 0x7faba8109bb0 2024-05-04T03:16:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.507+0000 7fabadb14640 1 -- 172.21.15.39:0/674105780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba81a2130 con 0x7faba8109bb0 2024-05-04T03:16:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.507+0000 7faba4ff9640 1 -- 172.21.15.39:0/674105780 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0010460 con 0x7faba8109bb0 2024-05-04T03:16:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.507+0000 7fabadb14640 1 -- 172.21.15.39:0/674105780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba81a2630 con 0x7faba8109bb0 2024-05-04T03:16:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.509+0000 7fabb49c8740 1 Processor -- start 2024-05-04T03:16:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.509+0000 7faba4ff9640 1 -- 172.21.15.39:0/674105780 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7faba0003970 con 0x7faba8109bb0 2024-05-04T03:16:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.509+0000 7fabb49c8740 1 -- start start 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea19e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 0x562c8ea1a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea212f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7fabb49c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562c8e97a5d0 con 0x562c8e6534d0 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7fabb49c8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562c8e933890 con 0x562c8e64b2f0 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7fabb49c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c8e976590 con 0x562c8e6741b0 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea212f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35392/0 (socket says 172.21.15.39:35392) 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 -- 172.21.15.39:0/724315137 learned_addr learned my addr 172.21.15.39:0/724315137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea19e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba77fe640 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 0x562c8ea1a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba4ff9640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fab900b27e0 0x7fab900b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 -- 172.21.15.39:0/724315137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 msgr2=0x562c8ea19e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea19e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 -- 172.21.15.39:0/724315137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 msgr2=0x562c8ea1a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 0x562c8ea1a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 -- 172.21.15.39:0/724315137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8e68c210 con 0x562c8e6534d0 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba7fff640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fab900b27e0 0x7fab900b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba6ffd640 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea212f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0127f0 tx=0x7fab9c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b51c13e9b91325 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.510+0000 7faba4ff9640 1 -- 172.21.15.39:0/674105780 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7faba00e2240 con 0x7faba8109bb0 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.511+0000 7fab8dffb640 1 -- 172.21.15.39:0/724315137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c012e30 con 0x562c8e6534d0 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.511+0000 7fab8dffb640 1 -- 172.21.15.39:0/724315137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab9c010c00 con 0x562c8e6534d0 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.511+0000 7fab8dffb640 1 -- 172.21.15.39:0/724315137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c01e970 con 0x562c8e6534d0 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 -- 172.21.15.39:0/724315137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 msgr2=0x562c8ea212f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7faba7fff640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fab900b27e0 0x7fab900b4cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faba00e00c0 tx=0x7faba0073210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea212f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0127f0 tx=0x7fab9c00b860 comp rx=0 tx=0).stop 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 -- 172.21.15.39:0/724315137 shutdown_connections 2024-05-04T03:16:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea212f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 0x562c8ea1a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 --2- 172.21.15.39:0/724315137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea19e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 -- 172.21.15.39:0/724315137 >> 172.21.15.39:0/724315137 conn(0x562c8e668300 msgr2=0x562c8ea113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 -- 172.21.15.39:0/724315137 shutdown_connections 2024-05-04T03:16:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 -- 172.21.15.39:0/724315137 wait complete. 2024-05-04T03:16:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 -- start start 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.512+0000 7fabb49c8740 1 Processor -- start 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea23a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6534d0 0x562c8ea23f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6741b0 0x562c8ea2aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7fabb49c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562c8ea21d40 con 0x562c8e64b2f0 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7fabb49c8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562c8e937370 con 0x562c8e6741b0 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7fabb49c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c8ea21ec0 con 0x562c8e6534d0 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7fabb49c8740 10 client.0 _ll_register_callbacks cb 0x562c8e68ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea23a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea23a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35394/0 (socket says 172.21.15.39:35394) 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 -- 172.21.15.39:0/422643397 learned_addr learned my addr 172.21.15.39:0/422643397 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba77fe640 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6534d0 0x562c8ea23f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 -- 172.21.15.39:0/422643397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6534d0 msgr2=0x562c8ea23f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6534d0 0x562c8ea23f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 -- 172.21.15.39:0/422643397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6741b0 msgr2=0x562c8ea2aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6741b0 0x562c8ea2aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 -- 172.21.15.39:0/422643397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8e758620 con 0x562c8e64b2f0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7faba6ffd640 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea23a00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fab9c015040 tx=0x7fab9c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a1b0392beccd6f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.513+0000 7fab86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fab86ffd640 1 -- 172.21.15.39:0/422643397 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c02c060 con 0x562c8e64b2f0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 1 -- 172.21.15.39:0/422643397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9c010c00 con 0x562c8e64b2f0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fab86ffd640 1 -- 172.21.15.39:0/422643397 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fab9c01e6f0 con 0x562c8e64b2f0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fab86ffd640 1 -- 172.21.15.39:0/422643397 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab9c012e30 con 0x562c8e64b2f0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 10 client.15981 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 1 -- 172.21.15.39:0/422643397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c8ea2ddd0 con 0x562c8e64b2f0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fab86ffd640 1 -- 172.21.15.39:0/422643397 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab9c010c00 con 0x562c8e64b2f0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fab86ffd640 1 -- 172.21.15.39:0/422643397 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fab9c01f920 con 0x562c8e64b2f0 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 10 client.15981 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 20 client.15981 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562c8e632d80 0x562c8ea31250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 4 client.15981 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.514+0000 7fabb49c8740 1 -- 172.21.15.39:0/422643397 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c8e9881e0 con 0x562c8e632d80 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.515+0000 7faba77fe640 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562c8e632d80 0x562c8ea31250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.515+0000 7faba77fe640 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562c8e632d80 0x562c8ea31250 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffeef9e444f80236 server_cookie=d6b776547298dea9 in_seq=0 out_seq=0 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.515+0000 7fab86ffd640 10 client.15981 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fab86ffd640 1 -- 172.21.15.39:0/422643397 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c8e9881e0 con 0x562c8e632d80 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fab86ffd640 10 client.15981 handle_command_reply: tid=0 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 client.15981 shutdown 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 -- 172.21.15.39:0/422643397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 msgr2=0x562c8ea23a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea23a00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fab9c015040 tx=0x7fab9c00b480 comp rx=0 tx=0).stop 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 -- 172.21.15.39:0/422643397 shutdown_connections 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562c8e632d80 0x562c8ea31250 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6741b0 0x562c8ea2aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6534d0 0x562c8ea23f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fab86ffd640 0 client.15981 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 --2- 172.21.15.39:0/422643397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea23a00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 -- 172.21.15.39:0/422643397 >> 172.21.15.39:0/422643397 conn(0x562c8e668300 msgr2=0x562c8ea14300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 -- 172.21.15.39:0/422643397 shutdown_connections 2024-05-04T03:16:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 1 -- 172.21.15.39:0/422643397 wait complete. 2024-05-04T03:16:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.516+0000 7fabb49c8740 20 client.15981 trim_cache size 0 max 16384 2024-05-04T03:16:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7fabb49c8740 1 Processor -- start 2024-05-04T03:16:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7fabb49c8740 1 -- start start 2024-05-04T03:16:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 0x562c8ea23ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea2ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7fabb49c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faba81a9110 con 0x562c8e6534d0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7fabb49c8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faba81aac50 con 0x562c8e64b2f0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7fabb49c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faba81a97e0 con 0x562c8e6741b0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea2ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea2ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35406/0 (socket says 172.21.15.39:35406) 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 -- 172.21.15.39:0/828417005 learned_addr learned my addr 172.21.15.39:0/828417005 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba6ffd640 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba6ffd640 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 0x562c8ea23ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 -- 172.21.15.39:0/828417005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 msgr2=0x562c8ea2b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea2b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 -- 172.21.15.39:0/828417005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 msgr2=0x562c8ea23ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 0x562c8ea23ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 -- 172.21.15.39:0/828417005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c8ea21240 con 0x562c8e6534d0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.532+0000 7faba77fe640 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea2ae30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fab98005ab0 tx=0x7fab98005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf5b59504f5824b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fab8d7fa640 1 -- 172.21.15.39:0/828417005 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98035040 con 0x562c8e6534d0 2024-05-04T03:16:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fab8d7fa640 1 -- 172.21.15.39:0/828417005 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab98031c50 con 0x562c8e6534d0 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fab8d7fa640 1 -- 172.21.15.39:0/828417005 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98046ad0 con 0x562c8e6534d0 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 -- 172.21.15.39:0/828417005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 msgr2=0x562c8ea2ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea2ae30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fab98005ab0 tx=0x7fab98005fd0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 -- 172.21.15.39:0/828417005 shutdown_connections 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea2b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e6534d0 0x562c8ea2ae30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 --2- 172.21.15.39:0/828417005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e64b2f0 0x562c8ea23ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 -- 172.21.15.39:0/828417005 >> 172.21.15.39:0/828417005 conn(0x562c8e668300 msgr2=0x562c8ea235d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 -- 172.21.15.39:0/828417005 shutdown_connections 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.533+0000 7fabb49c8740 1 -- 172.21.15.39:0/828417005 wait complete. 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 1 -- start start 2024-05-04T03:16:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 1 Processor -- start 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea28600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6534d0 0x562c8ea28b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faba81aa580 con 0x562c8e64b2f0 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faba81a9eb0 con 0x562c8e6534d0 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faba81a70f0 con 0x562c8e6741b0 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7fabb49c8740 10 client.0 _ll_register_callbacks cb 0x7fab98009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea28600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7faba6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea28600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35414/0 (socket says 172.21.15.39:35414) 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.534+0000 7faba6ffd640 1 -- 172.21.15.39:0/470866139 learned_addr learned my addr 172.21.15.39:0/470866139 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7faba77fe640 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6534d0 0x562c8ea28b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7faba6ffd640 1 -- 172.21.15.39:0/470866139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 msgr2=0x562c8ea975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7faba6ffd640 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7faba6ffd640 1 -- 172.21.15.39:0/470866139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6534d0 msgr2=0x562c8ea28b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7faba6ffd640 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6534d0 0x562c8ea28b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7faba6ffd640 1 -- 172.21.15.39:0/470866139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8ea97ae0 con 0x562c8e64b2f0 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7faba6ffd640 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea28600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab9c006010 tx=0x7fab9c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deff9a5141ddc54a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7fab86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7fab86ffd640 1 -- 172.21.15.39:0/470866139 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c036040 con 0x562c8e64b2f0 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7fab86ffd640 1 -- 172.21.15.39:0/470866139 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fab9c005030 con 0x562c8e64b2f0 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7fabb49c8740 1 -- 172.21.15.39:0/470866139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab98031c50 con 0x562c8e64b2f0 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7fab86ffd640 1 -- 172.21.15.39:0/470866139 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab9c02c040 con 0x562c8e64b2f0 2024-05-04T03:16:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7fabb49c8740 10 client.15987 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7fabb49c8740 1 -- 172.21.15.39:0/470866139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c8ea98bb0 con 0x562c8e64b2f0 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.535+0000 7fab86ffd640 1 -- 172.21.15.39:0/470866139 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab98031c50 con 0x562c8e64b2f0 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.536+0000 7fab86ffd640 1 -- 172.21.15.39:0/470866139 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fab9c027720 con 0x562c8e64b2f0 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.536+0000 7fabb49c8740 10 client.15987 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.536+0000 7fabb49c8740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.536+0000 7fabb49c8740 20 client.15987 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.536+0000 7fabb49c8740 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562c8e6081c0 0x562c8ea9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.536+0000 7fabb49c8740 4 client.15987 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.536+0000 7fabb49c8740 1 -- 172.21.15.39:0/470866139 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c8e982050 con 0x562c8e6081c0 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.536+0000 7faba77fe640 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562c8e6081c0 0x562c8ea9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.537+0000 7fab86ffd640 10 client.15987 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.537+0000 7faba77fe640 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562c8e6081c0 0x562c8ea9b570 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54efac8f32cf5b server_cookie=513fa8dd0fd140d2 in_seq=0 out_seq=0 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.545+0000 7fab86ffd640 1 -- 172.21.15.39:0/470866139 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c8e982050 con 0x562c8e6081c0 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.545+0000 7fab86ffd640 10 client.15987 handle_command_reply: tid=0 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.545+0000 7fabb49c8740 1 client.15987 shutdown 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 -- 172.21.15.39:0/470866139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 msgr2=0x562c8ea28600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea28600 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab9c006010 tx=0x7fab9c00bf20 comp rx=0 tx=0).stop 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 -- 172.21.15.39:0/470866139 shutdown_connections 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562c8e6081c0 0x562c8ea9b570 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c8e6741b0 0x562c8ea975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562c8e6534d0 0x562c8ea28b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 --2- 172.21.15.39:0/470866139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c8e64b2f0 0x562c8ea28600 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fab86ffd640 0 client.15987 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 -- 172.21.15.39:0/470866139 >> 172.21.15.39:0/470866139 conn(0x562c8e668300 msgr2=0x562c8ea113e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 -- 172.21.15.39:0/470866139 shutdown_connections 2024-05-04T03:16:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 1 -- 172.21.15.39:0/470866139 wait complete. 2024-05-04T03:16:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabb49c8740 20 client.15987 trim_cache size 0 max 16384 2024-05-04T03:16:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:16:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:16:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:16:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 -- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fab900b27e0 msgr2=0x7fab900b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fab900b27e0 0x7fab900b4cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faba00e00c0 tx=0x7faba0073210 comp rx=0 tx=0).stop 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 -- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8109bb0 msgr2=0x7faba81a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8109bb0 0x7faba81a17d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faba0009fd0 tx=0x7faba000b260 comp rx=0 tx=0).stop 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 -- 172.21.15.39:0/674105780 shutdown_connections 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fab900b27e0 0x7fab900b4cf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faba8109bb0 0x7faba81a17d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faba8104eb0 0x7faba819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.546+0000 7fabadb14640 1 --2- 172.21.15.39:0/674105780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faba8103c70 0x7faba819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.547+0000 7fabadb14640 1 -- 172.21.15.39:0/674105780 >> 172.21.15.39:0/674105780 conn(0x7faba80fee60 msgr2=0x7faba8105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.547+0000 7fabadb14640 1 -- 172.21.15.39:0/674105780 shutdown_connections 2024-05-04T03:16:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:36.547+0000 7fabadb14640 1 -- 172.21.15.39:0/674105780 wait complete. 2024-05-04T03:16:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:36 smithi167 ceph-mon[33614]: pgmap v560: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 207 KiB/s wr, 36 op/s 2024-05-04T03:16:36.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:36 smithi039 ceph-mon[29715]: pgmap v560: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 207 KiB/s wr, 36 op/s 2024-05-04T03:16:36.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 165.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:16:36.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:36 smithi066 ceph-mon[33264]: pgmap v560: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 207 KiB/s wr, 36 op/s 2024-05-04T03:16:38.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:38 smithi167 ceph-mon[33614]: pgmap v561: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 211 KiB/s wr, 38 op/s 2024-05-04T03:16:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:38 smithi039 ceph-mon[29715]: pgmap v561: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 211 KiB/s wr, 38 op/s 2024-05-04T03:16:38.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:38 smithi066 ceph-mon[33264]: pgmap v561: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 211 KiB/s wr, 38 op/s 2024-05-04T03:16:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:40 smithi167 ceph-mon[33614]: pgmap v562: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 208 KiB/s wr, 37 op/s 2024-05-04T03:16:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:40 smithi039 ceph-mon[29715]: pgmap v562: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 208 KiB/s wr, 37 op/s 2024-05-04T03:16:40.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:40 smithi066 ceph-mon[33264]: pgmap v562: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 208 KiB/s wr, 37 op/s 2024-05-04T03:16:41.346 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:16:41.613 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:41.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 170.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:16:42.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:42 smithi039 ceph-mon[29715]: pgmap v563: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 182 KiB/s wr, 34 op/s 2024-05-04T03:16:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:42 smithi167 ceph-mon[33614]: pgmap v563: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 182 KiB/s wr, 34 op/s 2024-05-04T03:16:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:42 smithi066 ceph-mon[33264]: pgmap v563: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 182 KiB/s wr, 34 op/s 2024-05-04T03:16:43.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.707+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/3258598785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 msgr2=0x7f3d38109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.707+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/3258598785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 0x7f3d38109350 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0099c0 tx=0x7f3d2c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.707+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/3258598785 shutdown_connections 2024-05-04T03:16:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.707+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/3258598785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d38109890 0x7f3d381111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.707+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/3258598785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d38102b20 0x7f3d38102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.707+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/3258598785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 0x7f3d38109350 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.707+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/3258598785 >> 172.21.15.39:0/3258598785 conn(0x7f3d380fe520 msgr2=0x7f3d38100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.708+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/3258598785 shutdown_connections 2024-05-04T03:16:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.708+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/3258598785 wait complete. 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.708+0000 7f3d3e9e4700 1 Processor -- start 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d3e9e4700 1 -- start start 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d3e9e4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d38102b20 0x7f3d38103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d3e9e4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 0x7f3d38103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d3e9e4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d38109890 0x7f3d381044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d3e9e4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38114340 con 0x7f3d38108f40 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d3e9e4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3d381141c0 con 0x7f3d38102b20 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d3e9e4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d381144c0 con 0x7f3d38109890 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d377fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 0x7f3d38103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d37fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d38102b20 0x7f3d38103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d377fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 0x7f3d38103fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35426/0 (socket says 172.21.15.39:35426) 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d377fe700 1 -- 172.21.15.39:0/1240070534 learned_addr learned my addr 172.21.15.39:0/1240070534 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.709+0000 7f3d3cf81700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d38109890 0x7f3d381044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d377fe700 1 -- 172.21.15.39:0/1240070534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d38109890 msgr2=0x7f3d381044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d377fe700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d38109890 0x7f3d381044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d377fe700 1 -- 172.21.15.39:0/1240070534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d38102b20 msgr2=0x7f3d38103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d377fe700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d38102b20 0x7f3d38103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d377fe700 1 -- 172.21.15.39:0/1240070534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d381a9090 con 0x7f3d38108f40 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d377fe700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 0x7f3d38103fb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0045e0 tx=0x7f3d2c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d357fa700 1 -- 172.21.15.39:0/1240070534 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c026070 con 0x7f3d38108f40 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d357fa700 1 -- 172.21.15.39:0/1240070534 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d2c00f460 con 0x7f3d38108f40 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d381a9320 con 0x7f3d38108f40 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d357fa700 1 -- 172.21.15.39:0/1240070534 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c009e60 con 0x7f3d38108f40 2024-05-04T03:16:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.710+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d381a9770 con 0x7f3d38108f40 2024-05-04T03:16:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.711+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d380512c0 con 0x7f3d38108f40 2024-05-04T03:16:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.713+0000 7f3d357fa700 1 -- 172.21.15.39:0/1240070534 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3d2c029070 con 0x7f3d38108f40 2024-05-04T03:16:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.714+0000 7f3d357fa700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3d200b2880 0x7f3d200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.714+0000 7f3d37fff700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3d200b2880 0x7f3d200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.714+0000 7f3d357fa700 1 -- 172.21.15.39:0/1240070534 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3d2c0e2c10 con 0x7f3d38108f40 2024-05-04T03:16:43.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.714+0000 7f3d37fff700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3d200b2880 0x7f3d200b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3d3806bc30 tx=0x7f3d28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:43.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.718+0000 7f3d357fa700 1 -- 172.21.15.39:0/1240070534 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d2c013070 con 0x7f3d38108f40 2024-05-04T03:16:43.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.985+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d3802e5f0 con 0x7f3d38108f40 2024-05-04T03:16:43.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.986+0000 7f3d357fa700 1 -- 172.21.15.39:0/1240070534 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f3d2c0aa9d0 con 0x7f3d38108f40 2024-05-04T03:16:43.987 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:16:43.987 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:16:43.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3d200b2880 msgr2=0x7f3d200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:43.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3d200b2880 0x7f3d200b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3d3806bc30 tx=0x7f3d28009260 comp rx=0 tx=0).stop 2024-05-04T03:16:43.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 msgr2=0x7f3d38103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 0x7f3d38103fb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0045e0 tx=0x7f3d2c004610 comp rx=0 tx=0).stop 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 shutdown_connections 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3d200b2880 0x7f3d200b4d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d38109890 0x7f3d381044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d38102b20 0x7f3d38103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 --2- 172.21.15.39:0/1240070534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d38108f40 0x7f3d38103fb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 >> 172.21.15.39:0/1240070534 conn(0x7f3d380fe520 msgr2=0x7f3d38100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 shutdown_connections 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:43.988+0000 7f3d3e9e4700 1 -- 172.21.15.39:0/1240070534 wait complete. 2024-05-04T03:16:43.990 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:16:44.647 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:44 smithi039 ceph-mon[29715]: pgmap v564: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 KiB/s wr, 37 op/s 2024-05-04T03:16:44.648 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:44 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1240070534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:44.675 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id d399bd5f --release --await' 2024-05-04T03:16:44.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id d399bd5f --release --await 2024-05-04T03:16:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:44 smithi167 ceph-mon[33614]: pgmap v564: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 KiB/s wr, 37 op/s 2024-05-04T03:16:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:44 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1240070534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:44.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:44 smithi066 ceph-mon[33264]: pgmap v564: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 KiB/s wr, 37 op/s 2024-05-04T03:16:44.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:44 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1240070534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:44.925 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:46 smithi167 ceph-mon[33614]: pgmap v565: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 35 op/s 2024-05-04T03:16:46.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:46 smithi039 ceph-mon[29715]: pgmap v565: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 35 op/s 2024-05-04T03:16:46.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 175.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:16:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:46 smithi066 ceph-mon[33264]: pgmap v565: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 35 op/s 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 -- 172.21.15.39:0/414802886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 msgr2=0x7f8f90103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/414802886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 0x7f8f90103d70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009990 tx=0x7f8f78009ca0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 -- 172.21.15.39:0/414802886 shutdown_connections 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/414802886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f901042b0 0x7f8f90110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/414802886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f90108200 0x7f8f90108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/414802886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 0x7f8f90103d70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 -- 172.21.15.39:0/414802886 >> 172.21.15.39:0/414802886 conn(0x7f8f900fe500 msgr2=0x7f8f90100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 -- 172.21.15.39:0/414802886 shutdown_connections 2024-05-04T03:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.858+0000 7f8f94bbe700 1 -- 172.21.15.39:0/414802886 wait complete. 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.859+0000 7f8f94bbe700 1 Processor -- start 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.859+0000 7f8f94bbe700 1 -- start start 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.859+0000 7f8f94bbe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 0x7f8f9019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.859+0000 7f8f94bbe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f901042b0 0x7f8f9019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.859+0000 7f8f94bbe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f90108200 0x7f8f90199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.859+0000 7f8f94bbe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f90113250 con 0x7f8f90103960 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.859+0000 7f8f94bbe700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f901130d0 con 0x7f8f901042b0 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.859+0000 7f8f94bbe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f901133d0 con 0x7f8f90108200 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f90108200 0x7f8f90199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 0x7f8f9019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f90108200 0x7f8f90199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:33620/0 (socket says 172.21.15.39:33620) 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8ed9d700 1 -- 172.21.15.39:0/3366941506 learned_addr learned my addr 172.21.15.39:0/3366941506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f901042b0 0x7f8f9019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8ed9d700 1 -- 172.21.15.39:0/3366941506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f901042b0 msgr2=0x7f8f9019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8ed9d700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f901042b0 0x7f8f9019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8ed9d700 1 -- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 msgr2=0x7f8f9019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8ed9d700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 0x7f8f9019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8ed9d700 1 -- 172.21.15.39:0/3366941506 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9019a050 con 0x7f8f90108200 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.860+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 0x7f8f9019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.861+0000 7f8f8ed9d700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f90108200 0x7f8f90199a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800efb0 tx=0x7f8f8800c550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.861+0000 7f8f877fe700 1 -- 172.21.15.39:0/3366941506 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88011670 con 0x7f8f90108200 2024-05-04T03:16:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.861+0000 7f8f94bbe700 1 -- 172.21.15.39:0/3366941506 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9019a340 con 0x7f8f90108200 2024-05-04T03:16:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.861+0000 7f8f94bbe700 1 -- 172.21.15.39:0/3366941506 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9019a850 con 0x7f8f90108200 2024-05-04T03:16:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.861+0000 7f8f877fe700 1 -- 172.21.15.39:0/3366941506 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f8800ce20 con 0x7f8f90108200 2024-05-04T03:16:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.861+0000 7f8f877fe700 1 -- 172.21.15.39:0/3366941506 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8801ab80 con 0x7f8f90108200 2024-05-04T03:16:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.864+0000 7f8f877fe700 1 -- 172.21.15.39:0/3366941506 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8f880233f0 con 0x7f8f90108200 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.864+0000 7f8f877fe700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8f7c0b2980 0x7f8f7c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8fa35d2b80 1 Processor -- start 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8f7c0b2980 0x7f8f7c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8f877fe700 1 -- 172.21.15.39:0/3366941506 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8f8801ae00 con 0x7f8f90108200 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8fa35d2b80 1 -- start start 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 0x5600142d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5600141e9a40 con 0x5600142c6ad0 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56001414eea0 con 0x5600142c7420 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5600141e5a00 con 0x5600142d1c40 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8f7c0b2980 0x7f8f7c0b4e90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8f9019ab50 tx=0x7f8f8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8f8e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8f8e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59386/0 (socket says 172.21.15.39:59386) 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.865+0000 7f8f8e59c700 1 -- 172.21.15.39:0/2640765509 learned_addr learned my addr 172.21.15.39:0/2640765509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8e59c700 1 -- 172.21.15.39:0/2640765509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 msgr2=0x5600142d4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8ed9d700 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 0x5600142d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 0x5600142d4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8e59c700 1 -- 172.21.15.39:0/2640765509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 msgr2=0x5600142d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8e59c700 1 -- 172.21.15.39:0/2640765509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600141369d0 con 0x5600142c6ad0 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c6ee0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bb90 tx=0x7f8f80013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf8dc4002a7dead server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f84ff9700 1 -- 172.21.15.39:0/2640765509 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8001e620 con 0x5600142c6ad0 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f84ff9700 1 -- 172.21.15.39:0/2640765509 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f8001ec50 con 0x5600142c6ad0 2024-05-04T03:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.866+0000 7f8f84ff9700 1 -- 172.21.15.39:0/2640765509 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80026ae0 con 0x5600142c6ad0 2024-05-04T03:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2640765509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 msgr2=0x5600142c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c6ee0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bb90 tx=0x7f8f80013000 comp rx=0 tx=0).stop 2024-05-04T03:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2640765509 shutdown_connections 2024-05-04T03:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c6ee0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 0x5600142d4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2640765509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2640765509 >> 172.21.15.39:0/2640765509 conn(0x560014149f40 msgr2=0x5600142c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2640765509 shutdown_connections 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.867+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2640765509 wait complete. 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 1 -- start start 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 1 Processor -- start 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c6ad0 0x5600142cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c7420 0x5600142cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 0x5600142cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5600142d8e20 con 0x5600142c7420 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5600141a2fd0 con 0x5600142d1c40 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5600142d8fa0 con 0x5600142c6ad0 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8fa35d2b80 10 client.0 _ll_register_callbacks cb 0x560014181a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8f8dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c7420 0x5600142cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8f8dd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c7420 0x5600142cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59392/0 (socket says 172.21.15.39:59392) 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8f8dd9b700 1 -- 172.21.15.39:0/2008747810 learned_addr learned my addr 172.21.15.39:0/2008747810 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8f8ed9d700 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 0x5600142cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.868+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c6ad0 0x5600142cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f8dd9b700 1 -- 172.21.15.39:0/2008747810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c6ad0 msgr2=0x5600142cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c6ad0 0x5600142cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f8dd9b700 1 -- 172.21.15.39:0/2008747810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 msgr2=0x5600142cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 0x5600142cf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f8dd9b700 1 -- 172.21.15.39:0/2008747810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600141a0860 con 0x5600142c7420 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c7420 0x5600142cf1a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009990 tx=0x7f8f78004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f891e8940434c7e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f75ffb700 1 -- 172.21.15.39:0/2008747810 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800bbe0 con 0x5600142c7420 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f75ffb700 1 -- 172.21.15.39:0/2008747810 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8f7800bd80 con 0x5600142c7420 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2008747810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8001ec50 con 0x5600142c7420 2024-05-04T03:16:46.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f75ffb700 1 -- 172.21.15.39:0/2008747810 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f78005000 con 0x5600142c7420 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8fa35d2b80 10 client.16005 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2008747810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5600142e3a70 con 0x5600142c7420 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.869+0000 7f8f75ffb700 1 -- 172.21.15.39:0/2008747810 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f8001ec50 con 0x5600142c7420 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8f75ffb700 1 -- 172.21.15.39:0/2008747810 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8f7800f8d0 con 0x5600142c7420 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8fa35d2b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8fa35d2b80 10 client.16005 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8fa35d2b80 20 client.16005 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5600142e6150 0x5600142e8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8fa35d2b80 4 client.16005 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2008747810 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600141f7420 con 0x5600142e6150 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5600142e6150 0x5600142e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5600142e6150 0x5600142e8570 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8fd3d34cf893f63 server_cookie=79394e65ddb0a013 in_seq=0 out_seq=0 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.870+0000 7f8f75ffb700 10 client.16005 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.871+0000 7f8f75ffb700 1 -- 172.21.15.39:0/2008747810 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600141f7420 con 0x5600142e6150 2024-05-04T03:16:46.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.871+0000 7f8f75ffb700 10 client.16005 handle_command_reply: tid=0 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.871+0000 7f8fa35d2b80 1 client.16005 shutdown 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.871+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2008747810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c7420 msgr2=0x5600142cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.871+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c7420 0x5600142cf1a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009990 tx=0x7f8f78004f90 comp rx=0 tx=0).stop 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2008747810 shutdown_connections 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5600142e6150 0x5600142e8570 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 0x5600142cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c6ad0 0x5600142cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8f75ffb700 0 client.16005 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/2008747810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c7420 0x5600142cf1a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2008747810 >> 172.21.15.39:0/2008747810 conn(0x560014149f40 msgr2=0x5600142c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2008747810 shutdown_connections 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/2008747810 wait complete. 2024-05-04T03:16:46.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.872+0000 7f8fa35d2b80 20 client.16005 trim_cache size 0 max 16384 2024-05-04T03:16:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8fa35d2b80 1 Processor -- start 2024-05-04T03:16:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8fa35d2b80 1 -- start start 2024-05-04T03:16:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c7420 0x5600142e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 0x5600142edd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7000d320 con 0x5600142c6ad0 2024-05-04T03:16:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7000ee60 con 0x5600142d1c40 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7000d9f0 con 0x5600142c7420 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59396/0 (socket says 172.21.15.39:59396) 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 -- 172.21.15.39:0/187840942 learned_addr learned my addr 172.21.15.39:0/187840942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c7420 0x5600142e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 -- 172.21.15.39:0/187840942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c7420 msgr2=0x5600142e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c7420 0x5600142e1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 -- 172.21.15.39:0/187840942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 msgr2=0x5600142edd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8ed9d700 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 0x5600142edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 0x5600142edd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 -- 172.21.15.39:0/187840942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600142c7850 con 0x5600142c6ad0 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.892+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c8b10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8f80011dd0 tx=0x7f8f800268c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22da3d75408949de server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8f77fff700 1 -- 172.21.15.39:0/187840942 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80026ae0 con 0x5600142c6ad0 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8f77fff700 1 -- 172.21.15.39:0/187840942 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f800054c0 con 0x5600142c6ad0 2024-05-04T03:16:46.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8f77fff700 1 -- 172.21.15.39:0/187840942 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80004a70 con 0x5600142c6ad0 2024-05-04T03:16:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/187840942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 msgr2=0x5600142c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c8b10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8f80011dd0 tx=0x7f8f800268c0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/187840942 shutdown_connections 2024-05-04T03:16:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142c8b10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142d1c40 0x5600142edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/187840942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142c7420 0x5600142e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.893+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/187840942 >> 172.21.15.39:0/187840942 conn(0x560014149f40 msgr2=0x5600142c8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/187840942 shutdown_connections 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/187840942 wait complete. 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 -- start start 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 Processor -- start 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 0x5600142db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7000e790 con 0x5600142c6ad0 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7000e0c0 con 0x5600142c7420 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7000b2c0 con 0x5600142d1c40 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8fa35d2b80 10 client.0 _ll_register_callbacks cb 0x560014181a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8f8e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8f8e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142de260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59404/0 (socket says 172.21.15.39:59404) 2024-05-04T03:16:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8f8e59c700 1 -- 172.21.15.39:0/168369853 learned_addr learned my addr 172.21.15.39:0/168369853 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.894+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f8ed9d700 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 0x5600142db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f8e59c700 1 -- 172.21.15.39:0/168369853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 msgr2=0x5600142db230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 0x5600142db230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f8e59c700 1 -- 172.21.15.39:0/168369853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 msgr2=0x5600142de7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142de7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f8e59c700 1 -- 172.21.15.39:0/168369853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600142db960 con 0x5600142c6ad0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f8e59c700 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142de260 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bf90 tx=0x7f8f800264b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33c1b699169ae09 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f75ffb700 1 -- 172.21.15.39:0/168369853 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80043020 con 0x5600142c6ad0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f75ffb700 1 -- 172.21.15.39:0/168369853 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8f800362c0 con 0x5600142c6ad0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/168369853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f800054c0 con 0x5600142c6ad0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f75ffb700 1 -- 172.21.15.39:0/168369853 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f800364c0 con 0x5600142c6ad0 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8fa35d2b80 10 client.16011 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/168369853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560014351280 con 0x5600142c6ad0 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.895+0000 7f8f75ffb700 1 -- 172.21.15.39:0/168369853 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f800054c0 con 0x5600142c6ad0 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.896+0000 7f8f75ffb700 1 -- 172.21.15.39:0/168369853 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8f8003fa80 con 0x5600142c6ad0 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.896+0000 7f8fa35d2b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.896+0000 7f8fa35d2b80 10 client.16011 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.896+0000 7f8fa35d2b80 20 client.16011 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.896+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5600142e6150 0x5600143537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.896+0000 7f8fa35d2b80 4 client.16011 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d399bd5f", "release": true, "await": true}] 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.896+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/168369853 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d399bd5f", "release": true, "await": true}) -- 0x5600141f14c0 con 0x5600142e6150 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.896+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5600142e6150 0x5600143537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.897+0000 7f8f75ffb700 10 client.16011 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.897+0000 7f8f8dd9b700 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5600142e6150 0x5600143537a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d007e8282e3b566 server_cookie=cabc8bed961e5b99 in_seq=0 out_seq=0 2024-05-04T03:16:46.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8f75ffb700 1 -- 172.21.15.39:0/168369853 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5600141f14c0 con 0x5600142e6150 2024-05-04T03:16:46.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8f75ffb700 10 client.16011 handle_command_reply: tid=0 2024-05-04T03:16:46.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 client.16011 shutdown 2024-05-04T03:16:46.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/168369853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 msgr2=0x5600142de260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142de260 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000bf90 tx=0x7f8f800264b0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/168369853 shutdown_connections 2024-05-04T03:16:46.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5600142e6150 0x5600143537a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600142c6ad0 0x5600142de260 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600142d1c40 0x5600142db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 --2- 172.21.15.39:0/168369853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600142c7420 0x5600142de7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8f75ffb700 0 client.16011 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/168369853 >> 172.21.15.39:0/168369853 conn(0x560014149f40 msgr2=0x5600142c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/168369853 shutdown_connections 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.902+0000 7f8fa35d2b80 1 -- 172.21.15.39:0/168369853 wait complete. 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8fa35d2b80 20 client.16011 trim_cache size 0 max 16384 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:16:46.905 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 16, 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "d399bd5f": { 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 16, 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 90.0, 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:16:46.906 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:16:46.907 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 -- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8f7c0b2980 msgr2=0x7f8f7c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8f7c0b2980 0x7f8f7c0b4e90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8f9019ab50 tx=0x7f8f8000b410 comp rx=0 tx=0).stop 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 -- 172.21.15.39:0/3366941506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f90108200 msgr2=0x7f8f90199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f90108200 0x7f8f90199a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800efb0 tx=0x7f8f8800c550 comp rx=0 tx=0).stop 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 -- 172.21.15.39:0/3366941506 shutdown_connections 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8f7c0b2980 0x7f8f7c0b4e90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f90103960 0x7f8f9019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f90108200 0x7f8f90199a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 --2- 172.21.15.39:0/3366941506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f901042b0 0x7f8f9019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 -- 172.21.15.39:0/3366941506 >> 172.21.15.39:0/3366941506 conn(0x7f8f900fe500 msgr2=0x7f8f90100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 -- 172.21.15.39:0/3366941506 shutdown_connections 2024-05-04T03:16:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:16:46.903+0000 7f8f94bbe700 1 -- 172.21.15.39:0/3366941506 wait complete. 2024-05-04T03:16:47.765 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd399bd5f', seconds elapsed: 44.0 2024-05-04T03:16:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:48 smithi167 ceph-mon[33614]: pgmap v566: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 187 KiB/s wr, 35 op/s 2024-05-04T03:16:48.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:48 smithi039 ceph-mon[29715]: pgmap v566: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 187 KiB/s wr, 35 op/s 2024-05-04T03:16:48.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:48 smithi066 ceph-mon[33264]: pgmap v566: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 187 KiB/s wr, 35 op/s 2024-05-04T03:16:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.704+0000 7f9770d3f640 1 -- 172.21.15.39:0/4161193700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c107800 msgr2=0x7f976c107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.704+0000 7f9770d3f640 1 --2- 172.21.15.39:0/4161193700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c107800 0x7f976c107c10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9754009860 tx=0x7f975402a340 comp rx=0 tx=0).stop 2024-05-04T03:16:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.704+0000 7f9770d3f640 1 -- 172.21.15.39:0/4161193700 shutdown_connections 2024-05-04T03:16:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.704+0000 7f9770d3f640 1 --2- 172.21.15.39:0/4161193700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c107800 0x7f976c107c10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.704+0000 7f9770d3f640 1 --2- 172.21.15.39:0/4161193700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f976c104ea0 0x7f976c1072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.704+0000 7f9770d3f640 1 --2- 172.21.15.39:0/4161193700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f976c102540 0x7f976c104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.704+0000 7f9770d3f640 1 -- 172.21.15.39:0/4161193700 >> 172.21.15.39:0/4161193700 conn(0x7f976c0fdd30 msgr2=0x7f976c100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.705+0000 7f9770d3f640 1 -- 172.21.15.39:0/4161193700 shutdown_connections 2024-05-04T03:16:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.705+0000 7f9770d3f640 1 -- 172.21.15.39:0/4161193700 wait complete. 2024-05-04T03:16:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.706+0000 7f9770d3f640 1 Processor -- start 2024-05-04T03:16:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.706+0000 7f9770d3f640 1 -- start start 2024-05-04T03:16:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f9770d3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c102540 0x7f976c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f9770d3f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f976c104ea0 0x7f976c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f9770d3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f976c107800 0x7f976c1a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f976a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c102540 0x7f976c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f9769d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f976c104ea0 0x7f976c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f976a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c102540 0x7f976c1a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:33688/0 (socket says 172.21.15.39:33688) 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f9769d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f976c104ea0 0x7f976c1a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59426/0 (socket says 172.21.15.39:59426) 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f976a59c640 1 -- 172.21.15.39:0/151102545 learned_addr learned my addr 172.21.15.39:0/151102545 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f976c115c30 con 0x7f976c104ea0 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f976c115ab0 con 0x7f976c107800 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.707+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f976c115db0 con 0x7f976c102540 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.708+0000 7f976ad9d640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f976c107800 0x7f976c1a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.708+0000 7f976a59c640 1 -- 172.21.15.39:0/151102545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f976c107800 msgr2=0x7f976c1a7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.708+0000 7f976a59c640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f976c107800 0x7f976c1a7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.708+0000 7f976a59c640 1 -- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f976c104ea0 msgr2=0x7f976c1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.708+0000 7f976a59c640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f976c104ea0 0x7f976c1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.708+0000 7f976a59c640 1 -- 172.21.15.39:0/151102545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c1a8320 con 0x7f976c102540 2024-05-04T03:16:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.708+0000 7f976a59c640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c102540 0x7f976c1a07e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e9f0 tx=0x7f975c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.709+0000 7f97637fe640 1 -- 172.21.15.39:0/151102545 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c00ccc0 con 0x7f976c102540 2024-05-04T03:16:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.709+0000 7f97637fe640 1 -- 172.21.15.39:0/151102545 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f975c0108c0 con 0x7f976c102540 2024-05-04T03:16:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.709+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976c1a8610 con 0x7f976c102540 2024-05-04T03:16:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.709+0000 7f97637fe640 1 -- 172.21.15.39:0/151102545 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0187c0 con 0x7f976c102540 2024-05-04T03:16:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.709+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976c1a8bb0 con 0x7f976c102540 2024-05-04T03:16:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.713+0000 7f97637fe640 1 -- 172.21.15.39:0/151102545 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f975c010430 con 0x7f976c102540 2024-05-04T03:16:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.713+0000 7f9777bfc740 1 Processor -- start 2024-05-04T03:16:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.713+0000 7f9777bfc740 1 -- start start 2024-05-04T03:16:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132ab01b0 0x564132e51a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a872f0 0x564132e51fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f9777bfc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564132db65d0 con 0x564132a8f4d0 2024-05-04T03:16:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f9777bfc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564132d6f890 con 0x564132a872f0 2024-05-04T03:16:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f9777bfc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564132db2590 con 0x564132ab01b0 2024-05-04T03:16:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f976ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132ab01b0 0x564132e51a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f976ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132ab01b0 0x564132e51a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:33696/0 (socket says 172.21.15.39:33696) 2024-05-04T03:16:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f976ad9d640 1 -- 172.21.15.39:0/826484995 learned_addr learned my addr 172.21.15.39:0/826484995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f9769d9b640 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.714+0000 7f976a59c640 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a872f0 0x564132e51fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.715+0000 7f97637fe640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f97580b2860 0x7f97580b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.715+0000 7f9769d9b640 1 -- 172.21.15.39:0/826484995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132ab01b0 msgr2=0x564132e51a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.715+0000 7f9769d9b640 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132ab01b0 0x564132e51a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.715+0000 7f9769d9b640 1 -- 172.21.15.39:0/826484995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a872f0 msgr2=0x564132e51fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.715+0000 7f9769d9b640 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a872f0 0x564132e51fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.715+0000 7f9769d9b640 1 -- 172.21.15.39:0/826484995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564132ac8210 con 0x564132a8f4d0 2024-05-04T03:16:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.715+0000 7f97637fe640 1 -- 172.21.15.39:0/151102545 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f975c0e23a0 con 0x7f976c102540 2024-05-04T03:16:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.716+0000 7f976ad9d640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f97580b2860 0x7f97580b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.716+0000 7f9769d9b640 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e58ee0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9764005fd0 tx=0x7f9764003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f289036c3ad72c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.716+0000 7f9760ff9640 1 -- 172.21.15.39:0/826484995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764004b20 con 0x564132a8f4d0 2024-05-04T03:16:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.716+0000 7f9760ff9640 1 -- 172.21.15.39:0/826484995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9764004cc0 con 0x564132a8f4d0 2024-05-04T03:16:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.716+0000 7f9760ff9640 1 -- 172.21.15.39:0/826484995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640126b0 con 0x564132a8f4d0 2024-05-04T03:16:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.718+0000 7f976ad9d640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f97580b2860 0x7f97580b4d70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9754005cd0 tx=0x7f975402ac80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.718+0000 7f9777bfc740 1 -- 172.21.15.39:0/826484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 msgr2=0x564132e58ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.718+0000 7f9777bfc740 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e58ee0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9764005fd0 tx=0x7f9764003e00 comp rx=0 tx=0).stop 2024-05-04T03:16:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.718+0000 7f9777bfc740 1 -- 172.21.15.39:0/826484995 shutdown_connections 2024-05-04T03:16:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.718+0000 7f9777bfc740 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e58ee0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.719+0000 7f9777bfc740 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a872f0 0x564132e51fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.719+0000 7f9777bfc740 1 --2- 172.21.15.39:0/826484995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132ab01b0 0x564132e51a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.719+0000 7f9777bfc740 1 -- 172.21.15.39:0/826484995 >> 172.21.15.39:0/826484995 conn(0x564132aa4300 msgr2=0x564132e4d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.720+0000 7f9777bfc740 1 -- 172.21.15.39:0/826484995 shutdown_connections 2024-05-04T03:16:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.720+0000 7f9777bfc740 1 -- 172.21.15.39:0/826484995 wait complete. 2024-05-04T03:16:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.721+0000 7f9777bfc740 1 -- start start 2024-05-04T03:16:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.721+0000 7f9777bfc740 1 Processor -- start 2024-05-04T03:16:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e5bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132ab01b0 0x564132e62b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f9777bfc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564132e59930 con 0x564132a8f4d0 2024-05-04T03:16:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f9777bfc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564132d73370 con 0x564132ab01b0 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f976ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132ab01b0 0x564132e62b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f976a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f976ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132ab01b0 0x564132e62b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:56038/0 (socket says 172.21.15.39:56038) 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f976a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:33698/0 (socket says 172.21.15.39:33698) 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f976ad9d640 1 -- 172.21.15.39:0/3537116396 learned_addr learned my addr 172.21.15.39:0/3537116396 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f9769d9b640 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e5bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564132e59ab0 con 0x564132a872f0 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.722+0000 7f9777bfc740 10 client.0 _ll_register_callbacks cb 0x564132ac7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.723+0000 7f976ad9d640 1 -- 172.21.15.39:0/3537116396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 msgr2=0x564132e5b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.723+0000 7f976ad9d640 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.723+0000 7f976ad9d640 1 -- 172.21.15.39:0/3537116396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 msgr2=0x564132e5bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.723+0000 7f976ad9d640 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e5bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.723+0000 7f976ad9d640 1 -- 172.21.15.39:0/3537116396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564132b94620 con 0x564132ab01b0 2024-05-04T03:16:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.723+0000 7f976a59c640 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.723+0000 7f9769d9b640 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e5bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.724+0000 7f976ad9d640 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132ab01b0 0x564132e62b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f975402a930 tx=0x7f9754005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923e02b9815417ac server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.724+0000 7f9751ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:16:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.724+0000 7f9751ffb640 1 -- 172.21.15.39:0/3537116396 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975404f020 con 0x564132ab01b0 2024-05-04T03:16:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.724+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9764004cc0 con 0x564132ab01b0 2024-05-04T03:16:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.724+0000 7f9751ffb640 1 -- 172.21.15.39:0/3537116396 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f97540055b0 con 0x564132ab01b0 2024-05-04T03:16:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.724+0000 7f9751ffb640 1 -- 172.21.15.39:0/3537116396 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9754037040 con 0x564132ab01b0 2024-05-04T03:16:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.724+0000 7f9777bfc740 10 client.25646 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.725+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564132e658a0 con 0x564132ab01b0 2024-05-04T03:16:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.726+0000 7f9751ffb640 1 -- 172.21.15.39:0/3537116396 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9754030e60 con 0x564132ab01b0 2024-05-04T03:16:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.726+0000 7f9751ffb640 1 -- 172.21.15.39:0/3537116396 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f975403d030 con 0x564132ab01b0 2024-05-04T03:16:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.726+0000 7f9777bfc740 10 client.25646 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.726+0000 7f9777bfc740 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:16:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.726+0000 7f9777bfc740 20 client.25646 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.726+0000 7f9777bfc740 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564132a6ed80 0x564132e68f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.727+0000 7f9777bfc740 4 client.25646 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:16:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.727+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9764004cc0 con 0x564132a6ed80 2024-05-04T03:16:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.727+0000 7f976a59c640 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564132a6ed80 0x564132e68f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.727+0000 7f9751ffb640 10 client.25646 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.727+0000 7f976a59c640 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564132a6ed80 0x564132e68f50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2ed335519ed218 server_cookie=5b6362d7c6507ec7 in_seq=0 out_seq=0 2024-05-04T03:16:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.728+0000 7f9751ffb640 1 -- 172.21.15.39:0/3537116396 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9764004cc0 con 0x564132a6ed80 2024-05-04T03:16:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.728+0000 7f9751ffb640 10 client.25646 handle_command_reply: tid=0 2024-05-04T03:16:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.728+0000 7f9777bfc740 1 client.25646 shutdown 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.728+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132ab01b0 msgr2=0x564132e62b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.728+0000 7f9777bfc740 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132ab01b0 0x564132e62b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f975402a930 tx=0x7f9754005e70 comp rx=0 tx=0).stop 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 shutdown_connections 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564132a6ed80 0x564132e68f50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9751ffb640 0 client.25646 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132ab01b0 0x564132e62b10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132e5bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 1 --2- 172.21.15.39:0/3537116396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 >> 172.21.15.39:0/3537116396 conn(0x564132aa4300 msgr2=0x564132e4c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 shutdown_connections 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 1 -- 172.21.15.39:0/3537116396 wait complete. 2024-05-04T03:16:49.731 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.729+0000 7f9777bfc740 20 client.25646 trim_cache size 0 max 16384 2024-05-04T03:16:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.757+0000 7f9777bfc740 1 Processor -- start 2024-05-04T03:16:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.757+0000 7f9777bfc740 1 -- start start 2024-05-04T03:16:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.757+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a6ed80 0x564132e4ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.757+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a872f0 0x564132e4d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.757+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a8f4d0 0x564132e5b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.757+0000 7f9777bfc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b1850 con 0x564132a872f0 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.757+0000 7f9777bfc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b3390 con 0x564132a8f4d0 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.757+0000 7f9777bfc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b1f20 con 0x564132a6ed80 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a8f4d0 0x564132e5b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a8f4d0 0x564132e5b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:56054/0 (socket says 172.21.15.39:56054) 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 -- 172.21.15.39:0/1041232189 learned_addr learned my addr 172.21.15.39:0/1041232189 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976a59c640 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a6ed80 0x564132e4ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f9769d9b640 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a872f0 0x564132e4d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 -- 172.21.15.39:0/1041232189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a6ed80 msgr2=0x564132e4ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a6ed80 0x564132e4ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 -- 172.21.15.39:0/1041232189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a872f0 msgr2=0x564132e4d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a872f0 0x564132e4d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 -- 172.21.15.39:0/1041232189 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564132e4dd00 con 0x564132a8f4d0 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f9769d9b640 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a872f0 0x564132e4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.758+0000 7f976ad9d640 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a8f4d0 0x564132e5b240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f975402ab30 tx=0x7f97540058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3a7b1a6be8e1fd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.759+0000 7f9753fff640 1 -- 172.21.15.39:0/1041232189 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975404f040 con 0x564132a8f4d0 2024-05-04T03:16:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.759+0000 7f9753fff640 1 -- 172.21.15.39:0/1041232189 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9754030e60 con 0x564132a8f4d0 2024-05-04T03:16:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.759+0000 7f9753fff640 1 -- 172.21.15.39:0/1041232189 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754004ac0 con 0x564132a8f4d0 2024-05-04T03:16:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.761+0000 7f9777bfc740 1 -- 172.21.15.39:0/1041232189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a8f4d0 msgr2=0x564132e5b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.761+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a8f4d0 0x564132e5b240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f975402ab30 tx=0x7f97540058b0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.762+0000 7f9777bfc740 1 -- 172.21.15.39:0/1041232189 shutdown_connections 2024-05-04T03:16:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.762+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a8f4d0 0x564132e5b240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.762+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a872f0 0x564132e4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.762+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1041232189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a6ed80 0x564132e4ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.762+0000 7f9777bfc740 1 -- 172.21.15.39:0/1041232189 >> 172.21.15.39:0/1041232189 conn(0x564132aa4300 msgr2=0x564132e55ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.762+0000 7f9777bfc740 1 -- 172.21.15.39:0/1041232189 shutdown_connections 2024-05-04T03:16:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.762+0000 7f9777bfc740 1 -- 172.21.15.39:0/1041232189 wait complete. 2024-05-04T03:16:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.762+0000 7f9777bfc740 1 -- start start 2024-05-04T03:16:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9777bfc740 1 Processor -- start 2024-05-04T03:16:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a6ed80 0x564132e5eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9777bfc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132ecf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9777bfc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b2cc0 con 0x564132a8f4d0 2024-05-04T03:16:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9777bfc740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1b25f0 con 0x564132a6ed80 2024-05-04T03:16:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9777bfc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1af830 con 0x564132a872f0 2024-05-04T03:16:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9777bfc740 10 client.0 _ll_register_callbacks cb 0x7f97540096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:16:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9769d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f976ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132ecf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9769d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:33722/0 (socket says 172.21.15.39:33722) 2024-05-04T03:16:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f9769d9b640 1 -- 172.21.15.39:0/1774499127 learned_addr learned my addr 172.21.15.39:0/1774499127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:16:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.763+0000 7f976a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a6ed80 0x564132e5eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f9769d9b640 1 -- 172.21.15.39:0/1774499127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a6ed80 msgr2=0x564132e5eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f9769d9b640 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a6ed80 0x564132e5eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f9769d9b640 1 -- 172.21.15.39:0/1774499127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 msgr2=0x564132ecf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f9769d9b640 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132ecf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f9769d9b640 1 -- 172.21.15.39:0/1774499127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564132e623b0 con 0x564132a872f0 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f976ad9d640 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132ecf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f9769d9b640 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5f3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9764005fd0 tx=0x7f9764009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8512f358aa02f520 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f9751ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:16:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.764+0000 7f9751ffb640 1 -- 172.21.15.39:0/1774499127 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976402b060 con 0x564132a872f0 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.765+0000 7f9751ffb640 1 -- 172.21.15.39:0/1774499127 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9764005250 con 0x564132a872f0 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.765+0000 7f9777bfc740 1 -- 172.21.15.39:0/1774499127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9754030e60 con 0x564132a872f0 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.765+0000 7f9751ffb640 1 -- 172.21.15.39:0/1774499127 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9764005450 con 0x564132a872f0 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.765+0000 7f9777bfc740 10 client.25654 fetch_fsmap learned FSMap version 32 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.765+0000 7f9777bfc740 1 -- 172.21.15.39:0/1774499127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564132ed0840 con 0x564132a872f0 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.765+0000 7f9751ffb640 1 -- 172.21.15.39:0/1774499127 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9754030e60 con 0x564132a872f0 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f9751ffb640 1 -- 172.21.15.39:0/1774499127 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f976400f040 con 0x564132a872f0 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f9777bfc740 10 client.25654 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f9777bfc740 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:16:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f9777bfc740 20 client.25654 populate_metadata read hostname 'smithi039' 2024-05-04T03:16:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564132ab01b0 0x564132ed3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f9777bfc740 4 client.25654 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:16:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f9777bfc740 1 -- 172.21.15.39:0/1774499127 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564132dc41e0 con 0x564132ab01b0 2024-05-04T03:16:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f976a59c640 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564132ab01b0 0x564132ed3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.766+0000 7f9751ffb640 10 client.25654 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.767+0000 7f976a59c640 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564132ab01b0 0x564132ed3050 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b66696dd3bb43b server_cookie=ac8cf0687fc5b5e4 in_seq=0 out_seq=0 2024-05-04T03:16:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.770+0000 7f9751ffb640 1 -- 172.21.15.39:0/1774499127 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564132dc41e0 con 0x564132ab01b0 2024-05-04T03:16:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.770+0000 7f9751ffb640 10 client.25654 handle_command_reply: tid=0 2024-05-04T03:16:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.770+0000 7f9777bfc740 1 client.25654 shutdown 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.770+0000 7f9777bfc740 1 -- 172.21.15.39:0/1774499127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 msgr2=0x564132e5f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.770+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5f3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9764005fd0 tx=0x7f9764009680 comp rx=0 tx=0).stop 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.771+0000 7f9777bfc740 1 -- 172.21.15.39:0/1774499127 shutdown_connections 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.771+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564132ab01b0 0x564132ed3050 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.771+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564132a8f4d0 0x564132ecf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.771+0000 7f9751ffb640 0 client.25654 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.771+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564132a872f0 0x564132e5f3f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.771+0000 7f9777bfc740 1 --2- 172.21.15.39:0/1774499127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564132a6ed80 0x564132e5eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.771+0000 7f9777bfc740 1 -- 172.21.15.39:0/1774499127 >> 172.21.15.39:0/1774499127 conn(0x564132aa4300 msgr2=0x564132e4ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.772+0000 7f9777bfc740 1 -- 172.21.15.39:0/1774499127 shutdown_connections 2024-05-04T03:16:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.772+0000 7f9777bfc740 1 -- 172.21.15.39:0/1774499127 wait complete. 2024-05-04T03:16:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.772+0000 7f9777bfc740 20 client.25654 trim_cache size 0 max 16384 2024-05-04T03:16:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:16:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:16:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:16:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.773+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f97580b2860 msgr2=0x7f97580b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.773+0000 7f9770d3f640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f97580b2860 0x7f97580b4d70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9754005cd0 tx=0x7f975402ac80 comp rx=0 tx=0).stop 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.773+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c102540 msgr2=0x7f976c1a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.773+0000 7f9770d3f640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c102540 0x7f976c1a07e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e9f0 tx=0x7f975c00eeb0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.774+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 shutdown_connections 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.774+0000 7f9770d3f640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f97580b2860 0x7f97580b4d70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.774+0000 7f9770d3f640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f976c107800 0x7f976c1a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.774+0000 7f9770d3f640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f976c104ea0 0x7f976c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.774+0000 7f9770d3f640 1 --2- 172.21.15.39:0/151102545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f976c102540 0x7f976c1a07e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.774+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 >> 172.21.15.39:0/151102545 conn(0x7f976c0fdd30 msgr2=0x7f976c102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.774+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 shutdown_connections 2024-05-04T03:16:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:16:49.774+0000 7f9770d3f640 1 -- 172.21.15.39:0/151102545 wait complete. 2024-05-04T03:16:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:50 smithi167 ceph-mon[33614]: pgmap v567: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 396 KiB/s wr, 60 op/s 2024-05-04T03:16:50.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:50 smithi039 ceph-mon[29715]: pgmap v567: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 396 KiB/s wr, 60 op/s 2024-05-04T03:16:50.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:50 smithi066 ceph-mon[33264]: pgmap v567: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 396 KiB/s wr, 60 op/s 2024-05-04T03:16:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 180.0 s, 1207.2 tps, lat 691.217 ms stddev 4708.904 2024-05-04T03:16:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:52 smithi167 ceph-mon[33614]: pgmap v568: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-05-04T03:16:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:52 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:16:52.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:52 smithi039 ceph-mon[29715]: pgmap v568: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-05-04T03:16:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:52 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:16:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:52 smithi066 ceph-mon[33264]: pgmap v568: 97 pgs: 97 active+clean; 1015 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-05-04T03:16:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:52 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:16:53.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:16:53.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:16:53.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:16:53.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:16:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:16:54.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:54 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-05-04T03:16:54.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:54 smithi167 ceph-mon[33614]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-04T03:16:54.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:54 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-05-04T03:16:54.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:54 smithi167 ceph-mon[33614]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-04T03:16:54.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:54 smithi167 ceph-mon[33614]: pgmap v569: 97 pgs: 97 active+clean; 1.0 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 466 op/s 2024-05-04T03:16:54.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:54 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-05-04T03:16:54.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:54 smithi039 ceph-mon[29715]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-04T03:16:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:54 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-05-04T03:16:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:54 smithi039 ceph-mon[29715]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-04T03:16:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:54 smithi039 ceph-mon[29715]: pgmap v569: 97 pgs: 97 active+clean; 1.0 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 466 op/s 2024-05-04T03:16:54.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:54 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-05-04T03:16:54.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:54 smithi066 ceph-mon[33264]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-04T03:16:54.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:54 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-05-04T03:16:54.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:54 smithi066 ceph-mon[33264]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-04T03:16:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:54 smithi066 ceph-mon[33264]: pgmap v569: 97 pgs: 97 active+clean; 1.0 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 466 op/s 2024-05-04T03:16:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:56 smithi167 ceph-mon[33614]: pgmap v570: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.8 MiB/s wr, 624 op/s 2024-05-04T03:16:56.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:56 smithi039 ceph-mon[29715]: pgmap v570: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.8 MiB/s wr, 624 op/s 2024-05-04T03:16:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 185.0 s, 1139.6 tps, lat 85.139 ms stddev 117.150 2024-05-04T03:16:56.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:56 smithi066 ceph-mon[33264]: pgmap v570: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.8 MiB/s wr, 624 op/s 2024-05-04T03:16:58.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:16:58 smithi167 ceph-mon[33614]: pgmap v571: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 819 op/s 2024-05-04T03:16:58.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:16:58 smithi039 ceph-mon[29715]: pgmap v571: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 819 op/s 2024-05-04T03:16:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:16:58 smithi066 ceph-mon[33264]: pgmap v571: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 819 op/s 2024-05-04T03:17:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:00 smithi167 ceph-mon[33614]: pgmap v572: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-05-04T03:17:00.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:00 smithi039 ceph-mon[29715]: pgmap v572: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-05-04T03:17:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:00 smithi066 ceph-mon[33264]: pgmap v572: 97 pgs: 97 active+clean; 1.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-05-04T03:17:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 190.0 s, 1110.8 tps, lat 92.816 ms stddev 133.042 2024-05-04T03:17:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:02 smithi167 ceph-mon[33614]: pgmap v573: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2024-05-04T03:17:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:02 smithi039 ceph-mon[29715]: pgmap v573: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2024-05-04T03:17:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:02 smithi066 ceph-mon[33264]: pgmap v573: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2024-05-04T03:17:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 -- 172.21.15.39:0/1977629459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184102b50 msgr2=0x7f3184103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 --2- 172.21.15.39:0/1977629459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184102b50 0x7f3184103850 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3174009900 tx=0x7f317402a410 comp rx=0 tx=0).stop 2024-05-04T03:17:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 -- 172.21.15.39:0/1977629459 shutdown_connections 2024-05-04T03:17:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 --2- 172.21.15.39:0/1977629459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3184108a90 0x7f318410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 --2- 172.21.15.39:0/1977629459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3184103d90 0x7f31841041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 --2- 172.21.15.39:0/1977629459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184102b50 0x7f3184103850 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 -- 172.21.15.39:0/1977629459 >> 172.21.15.39:0/1977629459 conn(0x7f31840fdd40 msgr2=0x7f31841001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 -- 172.21.15.39:0/1977629459 shutdown_connections 2024-05-04T03:17:02.915 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.912+0000 7f3188d00640 1 -- 172.21.15.39:0/1977629459 wait complete. 2024-05-04T03:17:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.913+0000 7f3188d00640 1 Processor -- start 2024-05-04T03:17:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.913+0000 7f3188d00640 1 -- start start 2024-05-04T03:17:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.913+0000 7f3188d00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3184102b50 0x7f318406af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.913+0000 7f3188d00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184103d90 0x7f318419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.913+0000 7f3188d00640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3184108a90 0x7f31841a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.913+0000 7f3188d00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f318410e500 con 0x7f3184103d90 2024-05-04T03:17:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.913+0000 7f3188d00640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f318410e380 con 0x7f3184108a90 2024-05-04T03:17:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.913+0000 7f3188d00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f318410e680 con 0x7f3184102b50 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184103d90 0x7f318419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184103d90 0x7f318419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46336/0 (socket says 172.21.15.39:46336) 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 -- 172.21.15.39:0/560164676 learned_addr learned my addr 172.21.15.39:0/560164676 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f3182d9d640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3184108a90 0x7f31841a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f318259c640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3184102b50 0x7f318406af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 -- 172.21.15.39:0/560164676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3184102b50 msgr2=0x7f318406af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3184102b50 0x7f318406af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 -- 172.21.15.39:0/560164676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3184108a90 msgr2=0x7f31841a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3184108a90 0x7f31841a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 -- 172.21.15.39:0/560164676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31841a1db0 con 0x7f3184103d90 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.914+0000 7f317bfff640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184103d90 0x7f318419f0b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f316c00d770 tx=0x7f316c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.915+0000 7f317b7fe640 1 -- 172.21.15.39:0/560164676 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c017070 con 0x7f3184103d90 2024-05-04T03:17:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.915+0000 7f317b7fe640 1 -- 172.21.15.39:0/560164676 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f316c0054b0 con 0x7f3184103d90 2024-05-04T03:17:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.915+0000 7f3188d00640 1 -- 172.21.15.39:0/560164676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31841a20a0 con 0x7f3184103d90 2024-05-04T03:17:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.915+0000 7f317b7fe640 1 -- 172.21.15.39:0/560164676 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c004c60 con 0x7f3184103d90 2024-05-04T03:17:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.915+0000 7f3188d00640 1 -- 172.21.15.39:0/560164676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31841a25a0 con 0x7f3184103d90 2024-05-04T03:17:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.917+0000 7f318fbb9740 1 Processor -- start 2024-05-04T03:17:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.917+0000 7f318fbb9740 1 -- start start 2024-05-04T03:17:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739442a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x562739442fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 0x562739449ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f318fbb9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5627393a75d0 con 0x5627390a11b0 2024-05-04T03:17:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f318fbb9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562739360890 con 0x5627390782f0 2024-05-04T03:17:02.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f318fbb9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627393a3590 con 0x5627390804d0 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f3182d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739442a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f3182d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739442a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46352/0 (socket says 172.21.15.39:46352) 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f317b7fe640 1 -- 172.21.15.39:0/560164676 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f316c010460 con 0x7f3184103d90 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f3182d9d640 1 -- 172.21.15.39:0/1650942820 learned_addr learned my addr 172.21.15.39:0/1650942820 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f318259c640 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x562739442fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f317bfff640 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 0x562739449ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f3182d9d640 1 -- 172.21.15.39:0/1650942820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 msgr2=0x562739449ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f3182d9d640 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 0x562739449ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f3182d9d640 1 -- 172.21.15.39:0/1650942820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 msgr2=0x562739442fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f3182d9d640 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x562739442fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.918+0000 7f3182d9d640 1 -- 172.21.15.39:0/1650942820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627390b9210 con 0x5627390a11b0 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.919+0000 7f3182d9d640 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739442a60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f317c007f40 tx=0x7f317c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea4c3866baf1bef server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.919+0000 7f3178ff9640 1 -- 172.21.15.39:0/1650942820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c010900 con 0x5627390a11b0 2024-05-04T03:17:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.919+0000 7f317b7fe640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f31700b27e0 0x7f31700b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.919+0000 7f3178ff9640 1 -- 172.21.15.39:0/1650942820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f317c0183f0 con 0x5627390a11b0 2024-05-04T03:17:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.919+0000 7f3178ff9640 1 -- 172.21.15.39:0/1650942820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c017600 con 0x5627390a11b0 2024-05-04T03:17:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.919+0000 7f318259c640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f31700b27e0 0x7f31700b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.919+0000 7f317b7fe640 1 -- 172.21.15.39:0/560164676 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f316c0e1c20 con 0x7f3184103d90 2024-05-04T03:17:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 -- 172.21.15.39:0/1650942820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 msgr2=0x562739442a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318259c640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f31700b27e0 0x7f31700b4cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3174005760 tx=0x7f3174000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739442a60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f317c007f40 tx=0x7f317c00f850 comp rx=0 tx=0).stop 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 -- 172.21.15.39:0/1650942820 shutdown_connections 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 0x562739449ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x562739442fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 --2- 172.21.15.39:0/1650942820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739442a60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 -- 172.21.15.39:0/1650942820 >> 172.21.15.39:0/1650942820 conn(0x562739095300 msgr2=0x56273943e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 -- 172.21.15.39:0/1650942820 shutdown_connections 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.920+0000 7f318fbb9740 1 -- 172.21.15.39:0/1650942820 wait complete. 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.921+0000 7f318fbb9740 1 -- start start 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.921+0000 7f318fbb9740 1 Processor -- start 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x56273944c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 0x56273944cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739453bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318fbb9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56273944a930 con 0x5627390a11b0 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318fbb9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562739364370 con 0x5627390782f0 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318fbb9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56273944aab0 con 0x5627390804d0 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f3182d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739453bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f3182d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739453bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46358/0 (socket says 172.21.15.39:46358) 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f3182d9d640 1 -- 172.21.15.39:0/257737128 learned_addr learned my addr 172.21.15.39:0/257737128 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318259c640 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x56273944c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318fbb9740 10 client.0 _ll_register_callbacks cb 0x5627390b8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f317bfff640 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 0x56273944cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f3182d9d640 1 -- 172.21.15.39:0/257737128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 msgr2=0x56273944cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f3182d9d640 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 0x56273944cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f3182d9d640 1 -- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 msgr2=0x56273944c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f3182d9d640 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x56273944c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f3182d9d640 1 -- 172.21.15.39:0/257737128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562739185620 con 0x5627390a11b0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.922+0000 7f318259c640 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x56273944c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f3182d9d640 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739453bb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f317c01a040 tx=0x7f317c018a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41e8a67b1aa0003 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f3169ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f3169ffb640 1 -- 172.21.15.39:0/257737128 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c025020 con 0x5627390a11b0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f3169ffb640 1 -- 172.21.15.39:0/257737128 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f317c004d10 con 0x5627390a11b0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f318fbb9740 1 -- 172.21.15.39:0/257737128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f317c0183f0 con 0x5627390a11b0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f3169ffb640 1 -- 172.21.15.39:0/257737128 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f317c003970 con 0x5627390a11b0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f318fbb9740 10 client.16041 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f318fbb9740 1 -- 172.21.15.39:0/257737128 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562739456ac0 con 0x5627390a11b0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.923+0000 7f3169ffb640 1 -- 172.21.15.39:0/257737128 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f317c0183f0 con 0x5627390a11b0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.924+0000 7f3169ffb640 1 -- 172.21.15.39:0/257737128 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f317c004040 con 0x5627390a11b0 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.924+0000 7f318fbb9740 10 client.16041 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.924+0000 7f318fbb9740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.924+0000 7f318fbb9740 20 client.16041 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.924+0000 7f318fbb9740 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56273905fd80 0x562739459f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.924+0000 7f318fbb9740 4 client.16041 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:17:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.924+0000 7f318fbb9740 1 -- 172.21.15.39:0/257737128 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627393b51e0 con 0x56273905fd80 2024-05-04T03:17:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.924+0000 7f318259c640 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56273905fd80 0x562739459f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.925+0000 7f318259c640 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56273905fd80 0x562739459f40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea7cd711aa49389 server_cookie=12220ba7a9ce692b in_seq=0 out_seq=0 2024-05-04T03:17:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.925+0000 7f3169ffb640 10 client.16041 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.925+0000 7f3169ffb640 1 -- 172.21.15.39:0/257737128 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5627393b51e0 con 0x56273905fd80 2024-05-04T03:17:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.925+0000 7f3169ffb640 10 client.16041 handle_command_reply: tid=0 2024-05-04T03:17:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.925+0000 7f318fbb9740 1 client.16041 shutdown 2024-05-04T03:17:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 -- 172.21.15.39:0/257737128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 msgr2=0x562739453bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739453bb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f317c01a040 tx=0x7f317c018a80 comp rx=0 tx=0).stop 2024-05-04T03:17:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 -- 172.21.15.39:0/257737128 shutdown_connections 2024-05-04T03:17:02.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56273905fd80 0x562739459f40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390a11b0 0x562739453bb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390804d0 0x56273944cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 --2- 172.21.15.39:0/257737128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x56273944c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 -- 172.21.15.39:0/257737128 >> 172.21.15.39:0/257737128 conn(0x562739095300 msgr2=0x56273943d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f3169ffb640 0 client.16041 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 -- 172.21.15.39:0/257737128 shutdown_connections 2024-05-04T03:17:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 1 -- 172.21.15.39:0/257737128 wait complete. 2024-05-04T03:17:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.926+0000 7f318fbb9740 20 client.16041 trim_cache size 0 max 16384 2024-05-04T03:17:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.949+0000 7f318fbb9740 1 Processor -- start 2024-05-04T03:17:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.949+0000 7f318fbb9740 1 -- start start 2024-05-04T03:17:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562739095300 0x5627394461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x562739453ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390804d0 0x562739454010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f318fbb9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31841ab350 con 0x5627390804d0 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f318fbb9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31841ace90 con 0x5627390782f0 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f318fbb9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31841aba20 con 0x562739095300 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f3182d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390804d0 0x562739454010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f318259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562739095300 0x5627394461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f3182d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390804d0 0x562739454010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46374/0 (socket says 172.21.15.39:46374) 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f3182d9d640 1 -- 172.21.15.39:0/2432780893 learned_addr learned my addr 172.21.15.39:0/2432780893 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f318259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562739095300 0x5627394461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:32776/0 (socket says 172.21.15.39:32776) 2024-05-04T03:17:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f317bfff640 1 --2- 172.21.15.39:0/2432780893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x562739453ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f3182d9d640 1 -- 172.21.15.39:0/2432780893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562739095300 msgr2=0x5627394461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f3182d9d640 1 --2- 172.21.15.39:0/2432780893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562739095300 0x5627394461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f3182d9d640 1 -- 172.21.15.39:0/2432780893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 msgr2=0x562739453ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f3182d9d640 1 --2- 172.21.15.39:0/2432780893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x562739453ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.950+0000 7f3182d9d640 1 -- 172.21.15.39:0/2432780893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273943ed00 con 0x5627390804d0 2024-05-04T03:17:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.951+0000 7f3182d9d640 1 --2- 172.21.15.39:0/2432780893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390804d0 0x562739454010 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f317c00fd50 tx=0x7f317c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb46fee9f7227dc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.951+0000 7f316bfff640 1 -- 172.21.15.39:0/2432780893 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c039070 con 0x5627390804d0 2024-05-04T03:17:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.951+0000 7f316bfff640 1 -- 172.21.15.39:0/2432780893 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f317c0183f0 con 0x5627390804d0 2024-05-04T03:17:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.951+0000 7f316bfff640 1 -- 172.21.15.39:0/2432780893 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c025070 con 0x5627390804d0 2024-05-04T03:17:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 -- 172.21.15.39:0/2432780893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390804d0 msgr2=0x562739454010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 --2- 172.21.15.39:0/2432780893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390804d0 0x562739454010 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f317c00fd50 tx=0x7f317c0079a0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 -- 172.21.15.39:0/2432780893 shutdown_connections 2024-05-04T03:17:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 --2- 172.21.15.39:0/2432780893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627390804d0 0x562739454010 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 --2- 172.21.15.39:0/2432780893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390782f0 0x562739453ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 --2- 172.21.15.39:0/2432780893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562739095300 0x5627394461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 -- 172.21.15.39:0/2432780893 >> 172.21.15.39:0/2432780893 conn(0x562739447380 msgr2=0x562739445ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 -- 172.21.15.39:0/2432780893 shutdown_connections 2024-05-04T03:17:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.953+0000 7f318fbb9740 1 -- 172.21.15.39:0/2432780893 wait complete. 2024-05-04T03:17:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.954+0000 7f318fbb9740 1 -- start start 2024-05-04T03:17:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.954+0000 7f318fbb9740 1 Processor -- start 2024-05-04T03:17:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.954+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390782f0 0x56273944ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.954+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390804d0 0x562739454290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.954+0000 7f318fbb9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562739095300 0x56273944f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f318fbb9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31841ac7c0 con 0x562739095300 2024-05-04T03:17:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f318fbb9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f31841ac0f0 con 0x5627390804d0 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f318fbb9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31841a9330 con 0x5627390782f0 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f318fbb9740 10 client.0 _ll_register_callbacks cb 0x7f3174009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f3182d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562739095300 0x56273944f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f3182d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562739095300 0x56273944f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46384/0 (socket says 172.21.15.39:46384) 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f3182d9d640 1 -- 172.21.15.39:0/849330857 learned_addr learned my addr 172.21.15.39:0/849330857 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f317bfff640 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390804d0 0x562739454290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f318259c640 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390782f0 0x56273944ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f317bfff640 1 -- 172.21.15.39:0/849330857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390782f0 msgr2=0x56273944ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f317bfff640 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390782f0 0x56273944ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f317bfff640 1 -- 172.21.15.39:0/849330857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562739095300 msgr2=0x56273944f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f317bfff640 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562739095300 0x56273944f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f317bfff640 1 -- 172.21.15.39:0/849330857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627394c4640 con 0x5627390804d0 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f3182d9d640 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562739095300 0x56273944f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f317bfff640 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390804d0 0x562739454290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f316c0df780 tx=0x7f316c009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=652d2e835dde2ae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.955+0000 7f3169ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f3169ffb640 1 -- 172.21.15.39:0/849330857 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c020040 con 0x5627390804d0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f3169ffb640 1 -- 172.21.15.39:0/849330857 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f316c0ad120 con 0x5627390804d0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 1 -- 172.21.15.39:0/849330857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f317c0183f0 con 0x5627390804d0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f3169ffb640 1 -- 172.21.15.39:0/849330857 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f316c0a8e10 con 0x5627390804d0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 10 client.25667 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 1 -- 172.21.15.39:0/849330857 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5627394c5930 con 0x5627390804d0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f3169ffb640 1 -- 172.21.15.39:0/849330857 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f317c0183f0 con 0x5627390804d0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f3169ffb640 1 -- 172.21.15.39:0/849330857 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f316c0a8560 con 0x5627390804d0 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 10 client.25667 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 20 client.25667 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5627390a11b0 0x5627394c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 4 client.25667 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:17:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.956+0000 7f318fbb9740 1 -- 172.21.15.39:0/849330857 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627393af050 con 0x5627390a11b0 2024-05-04T03:17:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.957+0000 7f318259c640 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5627390a11b0 0x5627394c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.957+0000 7f318259c640 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5627390a11b0 0x5627394c8300 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2713bd70e5a6410c server_cookie=217c4f4e4a847c31 in_seq=0 out_seq=0 2024-05-04T03:17:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.957+0000 7f3169ffb640 10 client.25667 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f3169ffb640 1 -- 172.21.15.39:0/849330857 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627393af050 con 0x5627390a11b0 2024-05-04T03:17:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f3169ffb640 10 client.25667 handle_command_reply: tid=0 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 client.25667 shutdown 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 -- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390804d0 msgr2=0x562739454290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390804d0 0x562739454290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f316c0df780 tx=0x7f316c009e60 comp rx=0 tx=0).stop 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 -- 172.21.15.39:0/849330857 shutdown_connections 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5627390a11b0 0x5627394c8300 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562739095300 0x56273944f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f3169ffb640 0 client.25667 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5627390804d0 0x562739454290 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 --2- 172.21.15.39:0/849330857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627390782f0 0x56273944ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 -- 172.21.15.39:0/849330857 >> 172.21.15.39:0/849330857 conn(0x562739447380 msgr2=0x56273944c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 -- 172.21.15.39:0/849330857 shutdown_connections 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.966+0000 7f318fbb9740 1 -- 172.21.15.39:0/849330857 wait complete. 2024-05-04T03:17:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f318fbb9740 20 client.25667 trim_cache size 0 max 16384 2024-05-04T03:17:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:17:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:17:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:17:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:17:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 -- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f31700b27e0 msgr2=0x7f31700b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f31700b27e0 0x7f31700b4cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3174005760 tx=0x7f3174000bc0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 -- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184103d90 msgr2=0x7f318419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184103d90 0x7f318419f0b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f316c00d770 tx=0x7f316c00dc30 comp rx=0 tx=0).stop 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 -- 172.21.15.39:0/560164676 shutdown_connections 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f31700b27e0 0x7f31700b4cf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3184108a90 0x7f31841a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3184103d90 0x7f318419f0b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 --2- 172.21.15.39:0/560164676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3184102b50 0x7f318406af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.967+0000 7f3188d00640 1 -- 172.21.15.39:0/560164676 >> 172.21.15.39:0/560164676 conn(0x7f31840fdd40 msgr2=0x7f3184104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.968+0000 7f3188d00640 1 -- 172.21.15.39:0/560164676 shutdown_connections 2024-05-04T03:17:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:02.968+0000 7f3188d00640 1 -- 172.21.15.39:0/560164676 wait complete. 2024-05-04T03:17:04.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:04 smithi167 ceph-mon[33614]: pgmap v574: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 24 MiB/s wr, 1.32k op/s 2024-05-04T03:17:04.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:04 smithi039 ceph-mon[29715]: pgmap v574: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 24 MiB/s wr, 1.32k op/s 2024-05-04T03:17:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:04 smithi066 ceph-mon[33264]: pgmap v574: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 24 MiB/s wr, 1.32k op/s 2024-05-04T03:17:06.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:06 smithi167 ceph-mon[33614]: pgmap v575: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-05-04T03:17:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:06 smithi039 ceph-mon[29715]: pgmap v575: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-05-04T03:17:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 195.0 s, 1240.2 tps, lat 78.399 ms stddev 103.959 2024-05-04T03:17:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:06 smithi066 ceph-mon[33264]: pgmap v575: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-05-04T03:17:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:08 smithi167 ceph-mon[33614]: pgmap v576: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 28 MiB/s wr, 1.22k op/s 2024-05-04T03:17:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:08 smithi039 ceph-mon[29715]: pgmap v576: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 28 MiB/s wr, 1.22k op/s 2024-05-04T03:17:08.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:08 smithi066 ceph-mon[33264]: pgmap v576: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 28 MiB/s wr, 1.22k op/s 2024-05-04T03:17:10.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:10 smithi167 ceph-mon[33614]: pgmap v577: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 1.30k op/s 2024-05-04T03:17:10.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:10 smithi039 ceph-mon[29715]: pgmap v577: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 1.30k op/s 2024-05-04T03:17:10.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:10 smithi066 ceph-mon[33264]: pgmap v577: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 28 MiB/s wr, 1.30k op/s 2024-05-04T03:17:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 200.0 s, 1099.6 tps, lat 88.681 ms stddev 122.299 2024-05-04T03:17:12.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:12 smithi167 ceph-mon[33614]: pgmap v578: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 25 MiB/s wr, 1.21k op/s 2024-05-04T03:17:12.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:12 smithi039 ceph-mon[29715]: pgmap v578: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 25 MiB/s wr, 1.21k op/s 2024-05-04T03:17:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:12 smithi066 ceph-mon[33264]: pgmap v578: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 25 MiB/s wr, 1.21k op/s 2024-05-04T03:17:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:14 smithi167 ceph-mon[33614]: pgmap v579: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 25 MiB/s wr, 1.31k op/s 2024-05-04T03:17:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:14 smithi039 ceph-mon[29715]: pgmap v579: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 25 MiB/s wr, 1.31k op/s 2024-05-04T03:17:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:14 smithi066 ceph-mon[33264]: pgmap v579: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 25 MiB/s wr, 1.31k op/s 2024-05-04T03:17:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 -- 172.21.15.39:0/4093983771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c1c110a10 msgr2=0x7f2c1c112e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 --2- 172.21.15.39:0/4093983771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c1c110a10 0x7f2c1c112e30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009860 tx=0x7f2c1002a340 comp rx=0 tx=0).stop 2024-05-04T03:17:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 -- 172.21.15.39:0/4093983771 shutdown_connections 2024-05-04T03:17:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 --2- 172.21.15.39:0/4093983771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c1c110a10 0x7f2c1c112e30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 --2- 172.21.15.39:0/4093983771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c10bd10 0x7f2c1c10c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 --2- 172.21.15.39:0/4093983771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c1c10ade0 0x7f2c1c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 -- 172.21.15.39:0/4093983771 >> 172.21.15.39:0/4093983771 conn(0x7f2c1c070a90 msgr2=0x7f2c1c072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 -- 172.21.15.39:0/4093983771 shutdown_connections 2024-05-04T03:17:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.127+0000 7f2c238d1640 1 -- 172.21.15.39:0/4093983771 wait complete. 2024-05-04T03:17:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.128+0000 7f2c238d1640 1 Processor -- start 2024-05-04T03:17:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.128+0000 7f2c238d1640 1 -- start start 2024-05-04T03:17:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c238d1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c1c10ade0 0x7f2c1c1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c238d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c10bd10 0x7f2c1c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c238d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c1c110a10 0x7f2c1c1a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c238d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c115ca0 con 0x7f2c1c110a10 2024-05-04T03:17:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c238d1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c115b20 con 0x7f2c1c10ade0 2024-05-04T03:17:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c238d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c115e20 con 0x7f2c1c10bd10 2024-05-04T03:17:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c20e6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c10bd10 0x7f2c1c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c20e6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c10bd10 0x7f2c1c1a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:43558/0 (socket says 172.21.15.39:43558) 2024-05-04T03:17:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c20e6c640 1 -- 172.21.15.39:0/77381536 learned_addr learned my addr 172.21.15.39:0/77381536 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c1c110a10 0x7f2c1c1a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.129+0000 7f2c2166d640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c1c10ade0 0x7f2c1c1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.130+0000 7f2c21e6e640 1 -- 172.21.15.39:0/77381536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c10bd10 msgr2=0x7f2c1c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.130+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c10bd10 0x7f2c1c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.130+0000 7f2c21e6e640 1 -- 172.21.15.39:0/77381536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c1c10ade0 msgr2=0x7f2c1c1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.130+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c1c10ade0 0x7f2c1c1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.130+0000 7f2c21e6e640 1 -- 172.21.15.39:0/77381536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c1a83f0 con 0x7f2c1c110a10 2024-05-04T03:17:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.131+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c10bd10 0x7f2c1c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.131+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c1c110a10 0x7f2c1c1a7d80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2c1002a860 tx=0x7f2c10005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.131+0000 7f2c1a7fc640 1 -- 172.21.15.39:0/77381536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10037070 con 0x7f2c1c110a10 2024-05-04T03:17:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.131+0000 7f2c1a7fc640 1 -- 172.21.15.39:0/77381536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c10038040 con 0x7f2c1c110a10 2024-05-04T03:17:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.131+0000 7f2c1a7fc640 1 -- 172.21.15.39:0/77381536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10004e50 con 0x7f2c1c110a10 2024-05-04T03:17:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.132+0000 7f2c238d1640 1 -- 172.21.15.39:0/77381536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c1a8680 con 0x7f2c1c110a10 2024-05-04T03:17:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.132+0000 7f2c238d1640 1 -- 172.21.15.39:0/77381536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c1a8b90 con 0x7f2c1c110a10 2024-05-04T03:17:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.136+0000 7f2c1a7fc640 1 -- 172.21.15.39:0/77381536 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2c100035f0 con 0x7f2c1c110a10 2024-05-04T03:17:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c1a7fc640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c040b2a00 0x7f2c040b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c2a782740 1 Processor -- start 2024-05-04T03:17:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c2a782740 1 -- start start 2024-05-04T03:17:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 0x5642e4acaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4acafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c2a782740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5642e4a2f5d0 con 0x5642e47084d0 2024-05-04T03:17:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c2a782740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5642e49e8890 con 0x5642e47291b0 2024-05-04T03:17:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.137+0000 7f2c2a782740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5642e4a2b590 con 0x5642e47002f0 2024-05-04T03:17:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c21e6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c20e6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4acafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c20e6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4acafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:43574/0 (socket says 172.21.15.39:43574) 2024-05-04T03:17:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c20e6c640 1 -- 172.21.15.39:0/1672288774 learned_addr learned my addr 172.21.15.39:0/1672288774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c2166d640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c040b2a00 0x7f2c040b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c20e6c640 1 -- 172.21.15.39:0/1672288774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 msgr2=0x5642e4acaa60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:17:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/1672288774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 0x5642e4acaa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c20e6c640 1 -- 172.21.15.39:0/1672288774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 msgr2=0x5642e4ad1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/1672288774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c20e6c640 1 -- 172.21.15.39:0/1672288774 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e4741210 con 0x5642e47002f0 2024-05-04T03:17:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/1672288774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.138+0000 7f2c1a7fc640 1 -- 172.21.15.39:0/77381536 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2c10103d30 con 0x7f2c1c110a10 2024-05-04T03:17:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.139+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/1672288774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4acafe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00afd0 tx=0x7f2c0c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f1eebf3d1ae721b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.139+0000 7f2c2166d640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c040b2a00 0x7f2c040b4f10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5642e4acbc30 tx=0x7f2c14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.139+0000 7f2c037fe640 1 -- 172.21.15.39:0/1672288774 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c01d070 con 0x5642e47002f0 2024-05-04T03:17:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.139+0000 7f2c037fe640 1 -- 172.21.15.39:0/1672288774 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c0c018410 con 0x5642e47002f0 2024-05-04T03:17:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.139+0000 7f2c037fe640 1 -- 172.21.15.39:0/1672288774 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00c5d0 con 0x5642e47002f0 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 -- 172.21.15.39:0/1672288774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 msgr2=0x5642e4acafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 --2- 172.21.15.39:0/1672288774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4acafe0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00afd0 tx=0x7f2c0c00edc0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 -- 172.21.15.39:0/1672288774 shutdown_connections 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 --2- 172.21.15.39:0/1672288774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 --2- 172.21.15.39:0/1672288774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4acafe0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 --2- 172.21.15.39:0/1672288774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 0x5642e4acaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 -- 172.21.15.39:0/1672288774 >> 172.21.15.39:0/1672288774 conn(0x5642e471d300 msgr2=0x5642e4ac63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 -- 172.21.15.39:0/1672288774 shutdown_connections 2024-05-04T03:17:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.140+0000 7f2c2a782740 1 -- 172.21.15.39:0/1672288774 wait complete. 2024-05-04T03:17:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.141+0000 7f2c2a782740 1 -- start start 2024-05-04T03:17:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.141+0000 7f2c2a782740 1 Processor -- start 2024-05-04T03:17:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2166d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47291b0 0x5642e4adbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2a782740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5642e4ad2980 con 0x5642e47084d0 2024-05-04T03:17:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2a782740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5642e49ec370 con 0x5642e47002f0 2024-05-04T03:17:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2a782740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5642e4ad2b00 con 0x5642e47291b0 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2166d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48570/0 (socket says 172.21.15.39:48570) 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2166d640 1 -- 172.21.15.39:0/2369491576 learned_addr learned my addr 172.21.15.39:0/2369491576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c2a782740 10 client.0 _ll_register_callbacks cb 0x5642e4740a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.142+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47291b0 0x5642e4adbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c2166d640 1 -- 172.21.15.39:0/2369491576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47291b0 msgr2=0x5642e4adbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c2166d640 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47291b0 0x5642e4adbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c2166d640 1 -- 172.21.15.39:0/2369491576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 msgr2=0x5642e4ad4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c2166d640 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c2166d640 1 -- 172.21.15.39:0/2369491576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e480d620 con 0x5642e47002f0 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c2166d640 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2c14012fd0 tx=0x7f2c14004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c9950f8b8cbb59 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2369491576 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14032060 con 0x5642e47002f0 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2369491576 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2c1400ec80 con 0x5642e47002f0 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.143+0000 7f2c2a782740 1 -- 172.21.15.39:0/2369491576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c0c018410 con 0x5642e47002f0 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2369491576 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2c1402d420 con 0x5642e47002f0 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c2a782740 10 client.25676 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c2a782740 1 -- 172.21.15.39:0/2369491576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642e4addf50 con 0x5642e47002f0 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2369491576 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c0c018410 con 0x5642e47002f0 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2369491576 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2c1400f450 con 0x5642e47002f0 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c2a782740 10 client.25676 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c2a782740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c2a782740 20 client.25676 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5642e46e7d80 0x5642e4adf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c2a782740 4 client.25676 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:17:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.144+0000 7f2c2a782740 1 -- 172.21.15.39:0/2369491576 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642e4a3d1e0 con 0x5642e46e7d80 2024-05-04T03:17:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.145+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5642e46e7d80 0x5642e4adf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.145+0000 7f2c00ff9640 10 client.25676 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.145+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5642e46e7d80 0x5642e4adf6f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a1fa423be41b87 server_cookie=d26e965ceaec7639 in_seq=0 out_seq=0 2024-05-04T03:17:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.146+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2369491576 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642e4a3d1e0 con 0x5642e46e7d80 2024-05-04T03:17:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.146+0000 7f2c00ff9640 10 client.25676 handle_command_reply: tid=0 2024-05-04T03:17:16.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.146+0000 7f2c2a782740 1 client.25676 shutdown 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c2a782740 1 -- 172.21.15.39:0/2369491576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 msgr2=0x5642e4ad46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2c14012fd0 tx=0x7f2c14004730 comp rx=0 tx=0).stop 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c2a782740 1 -- 172.21.15.39:0/2369491576 shutdown_connections 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5642e46e7d80 0x5642e4adf6f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47291b0 0x5642e4adbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4ad4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2369491576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad46f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c2a782740 1 -- 172.21.15.39:0/2369491576 >> 172.21.15.39:0/2369491576 conn(0x5642e471d300 msgr2=0x5642e4ac9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.147+0000 7f2c00ff9640 0 client.25676 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.148+0000 7f2c2a782740 1 -- 172.21.15.39:0/2369491576 shutdown_connections 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.148+0000 7f2c2a782740 1 -- 172.21.15.39:0/2369491576 wait complete. 2024-05-04T03:17:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.148+0000 7f2c2a782740 20 client.25676 trim_cache size 0 max 16384 2024-05-04T03:17:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.178+0000 7f2c2a782740 1 Processor -- start 2024-05-04T03:17:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.178+0000 7f2c2a782740 1 -- start start 2024-05-04T03:17:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4adfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4adbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 0x5642e4ae03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c2a782740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1b1830 con 0x5642e47084d0 2024-05-04T03:17:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c2a782740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1b3370 con 0x5642e47291b0 2024-05-04T03:17:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c2a782740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1b1f00 con 0x5642e47002f0 2024-05-04T03:17:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c20e6c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4adbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c20e6c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4adbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57298/0 (socket says 172.21.15.39:57298) 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c20e6c640 1 -- 172.21.15.39:0/356146945 learned_addr learned my addr 172.21.15.39:0/356146945 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c2166d640 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4adfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 0x5642e4ae03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c20e6c640 1 -- 172.21.15.39:0/356146945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 msgr2=0x5642e4adfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4adfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c20e6c640 1 -- 172.21.15.39:0/356146945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 msgr2=0x5642e4ae03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.179+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 0x5642e4ae03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.180+0000 7f2c20e6c640 1 -- 172.21.15.39:0/356146945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e4ac1780 con 0x5642e47084d0 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.180+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 0x5642e4ae03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.180+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4adbb20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c005b00 tx=0x7f2c0c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fe644dc8e2c44 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.180+0000 7f2c02ffd640 1 -- 172.21.15.39:0/356146945 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c02d070 con 0x5642e47084d0 2024-05-04T03:17:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.180+0000 7f2c02ffd640 1 -- 172.21.15.39:0/356146945 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c0c028070 con 0x5642e47084d0 2024-05-04T03:17:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.180+0000 7f2c02ffd640 1 -- 172.21.15.39:0/356146945 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c01d040 con 0x5642e47084d0 2024-05-04T03:17:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.182+0000 7f2c2a782740 1 -- 172.21.15.39:0/356146945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 msgr2=0x5642e4adbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.182+0000 7f2c2a782740 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4adbb20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c005b00 tx=0x7f2c0c005a50 comp rx=0 tx=0).stop 2024-05-04T03:17:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.182+0000 7f2c2a782740 1 -- 172.21.15.39:0/356146945 shutdown_connections 2024-05-04T03:17:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.182+0000 7f2c2a782740 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47291b0 0x5642e4ae03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.182+0000 7f2c2a782740 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47084d0 0x5642e4adbb20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.182+0000 7f2c2a782740 1 --2- 172.21.15.39:0/356146945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47002f0 0x5642e4adfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.182+0000 7f2c2a782740 1 -- 172.21.15.39:0/356146945 >> 172.21.15.39:0/356146945 conn(0x5642e471d300 msgr2=0x5642e4aceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.183+0000 7f2c2a782740 1 -- 172.21.15.39:0/356146945 shutdown_connections 2024-05-04T03:17:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.183+0000 7f2c2a782740 1 -- 172.21.15.39:0/356146945 wait complete. 2024-05-04T03:17:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.183+0000 7f2c2a782740 1 -- start start 2024-05-04T03:17:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.184+0000 7f2c2a782740 1 Processor -- start 2024-05-04T03:17:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.184+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.184+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47084d0 0x5642e4ad9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c2a782740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47291b0 0x5642e4adabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c2166d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c2166d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48582/0 (socket says 172.21.15.39:48582) 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c20e6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47084d0 0x5642e4ad9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c20e6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47084d0 0x5642e4ad9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:43600/0 (socket says 172.21.15.39:43600) 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c2166d640 1 -- 172.21.15.39:0/2797382190 learned_addr learned my addr 172.21.15.39:0/2797382190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1b2ca0 con 0x5642e47291b0 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1b25d0 con 0x5642e47002f0 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47291b0 0x5642e4adabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1af810 con 0x5642e47084d0 2024-05-04T03:17:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.185+0000 7f2c2a782740 10 client.0 _ll_register_callbacks cb 0x7f2c14009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c2166d640 1 -- 172.21.15.39:0/2797382190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47084d0 msgr2=0x5642e4ad9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c2166d640 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47084d0 0x5642e4ad9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c2166d640 1 -- 172.21.15.39:0/2797382190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47291b0 msgr2=0x5642e4adabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c2166d640 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47291b0 0x5642e4adabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c2166d640 1 -- 172.21.15.39:0/2797382190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e4adb1b0 con 0x5642e47002f0 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c21e6e640 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47291b0 0x5642e4adabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c2166d640 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad9780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2c14004700 tx=0x7f2c1400fe50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c16512269f9701e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.186+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2797382190 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14047020 con 0x5642e47002f0 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.187+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2797382190 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2c1400ec50 con 0x5642e47002f0 2024-05-04T03:17:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.187+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c0c028070 con 0x5642e47002f0 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.187+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2797382190 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2c1403d040 con 0x5642e47002f0 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.187+0000 7f2c2a782740 10 client.25682 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.187+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5642e4b490d0 con 0x5642e47002f0 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.188+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2797382190 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c1402d5c0 con 0x5642e47002f0 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.188+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2797382190 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2c14040030 con 0x5642e47002f0 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.188+0000 7f2c2a782740 10 client.25682 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.188+0000 7f2c2a782740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.188+0000 7f2c2a782740 20 client.25682 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.188+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5642e46bd1c0 0x5642e4b4baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.188+0000 7f2c2a782740 4 client.25682 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:17:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.188+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2c0c028070 con 0x5642e46bd1c0 2024-05-04T03:17:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.189+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5642e46bd1c0 0x5642e4b4baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.189+0000 7f2c00ff9640 10 client.25682 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.189+0000 7f2c20e6c640 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5642e46bd1c0 0x5642e4b4baf0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f21ab969a8bfb5d server_cookie=8bd6381c8aa528b8 in_seq=0 out_seq=0 2024-05-04T03:17:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.197+0000 7f2c00ff9640 1 -- 172.21.15.39:0/2797382190 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2c0c028070 con 0x5642e46bd1c0 2024-05-04T03:17:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.197+0000 7f2c00ff9640 10 client.25682 handle_command_reply: tid=0 2024-05-04T03:17:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.198+0000 7f2c2a782740 1 client.25682 shutdown 2024-05-04T03:17:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.198+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 msgr2=0x5642e4ad9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.198+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad9780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2c14004700 tx=0x7f2c1400fe50 comp rx=0 tx=0).stop 2024-05-04T03:17:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.198+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 shutdown_connections 2024-05-04T03:17:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.198+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5642e46bd1c0 0x5642e4b4baf0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.198+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5642e47291b0 0x5642e4adabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.198+0000 7f2c00ff9640 0 client.25682 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.199+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642e47084d0 0x5642e4ad9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.199+0000 7f2c2a782740 1 --2- 172.21.15.39:0/2797382190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642e47002f0 0x5642e4ad9780 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.199+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 >> 172.21.15.39:0/2797382190 conn(0x5642e471d300 msgr2=0x5642e4adfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.199+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 shutdown_connections 2024-05-04T03:17:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.199+0000 7f2c2a782740 1 -- 172.21.15.39:0/2797382190 wait complete. 2024-05-04T03:17:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.199+0000 7f2c2a782740 20 client.25682 trim_cache size 0 max 16384 2024-05-04T03:17:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:17:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:17:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:17:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:17:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.200+0000 7f2c238d1640 1 -- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c040b2a00 msgr2=0x7f2c040b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.200+0000 7f2c238d1640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c040b2a00 0x7f2c040b4f10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5642e4acbc30 tx=0x7f2c14009260 comp rx=0 tx=0).stop 2024-05-04T03:17:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.200+0000 7f2c238d1640 1 -- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c1c110a10 msgr2=0x7f2c1c1a7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.200+0000 7f2c238d1640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c1c110a10 0x7f2c1c1a7d80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2c1002a860 tx=0x7f2c10005040 comp rx=0 tx=0).stop 2024-05-04T03:17:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.200+0000 7f2c238d1640 1 -- 172.21.15.39:0/77381536 shutdown_connections 2024-05-04T03:17:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.200+0000 7f2c238d1640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2c040b2a00 0x7f2c040b4f10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.200+0000 7f2c238d1640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c1c110a10 0x7f2c1c1a7d80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.201+0000 7f2c238d1640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c1c10bd10 0x7f2c1c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.201+0000 7f2c238d1640 1 --2- 172.21.15.39:0/77381536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c1c10ade0 0x7f2c1c1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.201+0000 7f2c238d1640 1 -- 172.21.15.39:0/77381536 >> 172.21.15.39:0/77381536 conn(0x7f2c1c070a90 msgr2=0x7f2c1c10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.201+0000 7f2c238d1640 1 -- 172.21.15.39:0/77381536 shutdown_connections 2024-05-04T03:17:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:16.201+0000 7f2c238d1640 1 -- 172.21.15.39:0/77381536 wait complete. 2024-05-04T03:17:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:16 smithi167 ceph-mon[33614]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 24 MiB/s wr, 1.23k op/s 2024-05-04T03:17:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:16 smithi039 ceph-mon[29715]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 24 MiB/s wr, 1.23k op/s 2024-05-04T03:17:16.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 205.0 s, 1211.1 tps, lat 78.740 ms stddev 92.044 2024-05-04T03:17:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:16 smithi066 ceph-mon[33264]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 24 MiB/s wr, 1.23k op/s 2024-05-04T03:17:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:18 smithi167 ceph-mon[33614]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 25 MiB/s wr, 1.31k op/s 2024-05-04T03:17:18.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:18 smithi039 ceph-mon[29715]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 25 MiB/s wr, 1.31k op/s 2024-05-04T03:17:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:18 smithi066 ceph-mon[33264]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 25 MiB/s wr, 1.31k op/s 2024-05-04T03:17:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:20 smithi167 ceph-mon[33614]: pgmap v582: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 24 MiB/s wr, 1.35k op/s 2024-05-04T03:17:20.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:20 smithi039 ceph-mon[29715]: pgmap v582: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 24 MiB/s wr, 1.35k op/s 2024-05-04T03:17:20.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:20 smithi066 ceph-mon[33264]: pgmap v582: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 24 MiB/s wr, 1.35k op/s 2024-05-04T03:17:21.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 210.0 s, 1307.9 tps, lat 84.064 ms stddev 101.766 2024-05-04T03:17:22.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:22 smithi167 ceph-mon[33614]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-05-04T03:17:22.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:22 smithi039 ceph-mon[29715]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-05-04T03:17:22.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:22 smithi066 ceph-mon[33264]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-05-04T03:17:24.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:24 smithi167 ceph-mon[33614]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 23 MiB/s wr, 1.37k op/s 2024-05-04T03:17:24.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:24 smithi039 ceph-mon[29715]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 23 MiB/s wr, 1.37k op/s 2024-05-04T03:17:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:24 smithi066 ceph-mon[33264]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 23 MiB/s wr, 1.37k op/s 2024-05-04T03:17:26.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:26 smithi167 ceph-mon[33614]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 23 MiB/s wr, 1.28k op/s 2024-05-04T03:17:26.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:26 smithi039 ceph-mon[29715]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 23 MiB/s wr, 1.28k op/s 2024-05-04T03:17:26.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 215.0 s, 1183.2 tps, lat 84.265 ms stddev 105.011 2024-05-04T03:17:26.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:26 smithi066 ceph-mon[33264]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 23 MiB/s wr, 1.28k op/s 2024-05-04T03:17:28.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:28 smithi039 ceph-mon[29715]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 25 MiB/s wr, 1.39k op/s 2024-05-04T03:17:28.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:28 smithi066 ceph-mon[33264]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 25 MiB/s wr, 1.39k op/s 2024-05-04T03:17:28.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:28 smithi167 ceph-mon[33614]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 25 MiB/s wr, 1.39k op/s 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.341+0000 7f6da49b2640 1 -- 172.21.15.39:0/1845415175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 msgr2=0x7f6da0104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.341+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1845415175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 0x7f6da0104190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009860 tx=0x7f6d8802a340 comp rx=0 tx=0).stop 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.342+0000 7f6da49b2640 1 -- 172.21.15.39:0/1845415175 shutdown_connections 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.342+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1845415175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da0108a80 0x7f6da010aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.342+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1845415175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 0x7f6da0104190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.342+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1845415175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6da0102b40 0x7f6da0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.342+0000 7f6da49b2640 1 -- 172.21.15.39:0/1845415175 >> 172.21.15.39:0/1845415175 conn(0x7f6da00fdd30 msgr2=0x7f6da01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.342+0000 7f6da49b2640 1 -- 172.21.15.39:0/1845415175 shutdown_connections 2024-05-04T03:17:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.342+0000 7f6da49b2640 1 -- 172.21.15.39:0/1845415175 wait complete. 2024-05-04T03:17:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.343+0000 7f6da49b2640 1 Processor -- start 2024-05-04T03:17:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.343+0000 7f6da49b2640 1 -- start start 2024-05-04T03:17:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6da49b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6da0102b40 0x7f6da019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6da49b2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 0x7f6da019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6da49b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da0108a80 0x7f6da01a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6da49b2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6da010e4f0 con 0x7f6da0102b40 2024-05-04T03:17:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6da49b2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6da010e370 con 0x7f6da0103d80 2024-05-04T03:17:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6da49b2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6da010e670 con 0x7f6da0108a80 2024-05-04T03:17:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6d9ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da0108a80 0x7f6da01a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6da0102b40 0x7f6da019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6da0102b40 0x7f6da019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59048/0 (socket says 172.21.15.39:59048) 2024-05-04T03:17:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6d9e59c640 1 -- 172.21.15.39:0/1240859180 learned_addr learned my addr 172.21.15.39:0/1240859180 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 0x7f6da019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6d9ed9d640 1 -- 172.21.15.39:0/1240859180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 msgr2=0x7f6da019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.344+0000 7f6d9ed9d640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 0x7f6da019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.345+0000 7f6d9ed9d640 1 -- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6da0102b40 msgr2=0x7f6da019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.345+0000 7f6d9ed9d640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6da0102b40 0x7f6da019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.345+0000 7f6d9ed9d640 1 -- 172.21.15.39:0/1240859180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da01a1dc0 con 0x7f6da0108a80 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.345+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6da0102b40 0x7f6da019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.345+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 0x7f6da019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.346+0000 7f6d9ed9d640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da0108a80 0x7f6da01a1750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009fd0 tx=0x7f6d9800eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.346+0000 7f6d977fe640 1 -- 172.21.15.39:0/1240859180 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004220 con 0x7f6da0108a80 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.346+0000 7f6d977fe640 1 -- 172.21.15.39:0/1240859180 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d980043c0 con 0x7f6da0108a80 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.346+0000 7f6da49b2640 1 -- 172.21.15.39:0/1240859180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da01a20b0 con 0x7f6da0108a80 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.346+0000 7f6da49b2640 1 -- 172.21.15.39:0/1240859180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da01a2650 con 0x7f6da0108a80 2024-05-04T03:17:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.347+0000 7f6d977fe640 1 -- 172.21.15.39:0/1240859180 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d980047a0 con 0x7f6da0108a80 2024-05-04T03:17:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.350+0000 7f6dab868740 1 Processor -- start 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.350+0000 7f6d977fe640 1 -- 172.21.15.39:0/1240859180 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6d98004960 con 0x7f6da0108a80 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6dab868740 1 -- start start 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6dab868740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 0x55f4dcab2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6dab868740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6dab868740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcaafad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6dab868740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f4dca145d0 con 0x55f4dc6e52f0 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6dab868740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc9cd890 con 0x55f4dc70e1b0 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6dab868740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f4dca10590 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59060/0 (socket says 172.21.15.39:59060) 2024-05-04T03:17:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6d9e59c640 1 -- 172.21.15.39:0/2613063838 learned_addr learned my addr 172.21.15.39:0/2613063838 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.351+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/2613063838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 0x55f4dcab2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.352+0000 7f6d9e59c640 1 -- 172.21.15.39:0/2613063838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 msgr2=0x55f4dcaafad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.352+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/2613063838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcaafad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.352+0000 7f6d9e59c640 1 -- 172.21.15.39:0/2613063838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 msgr2=0x55f4dcab2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.352+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/2613063838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 0x55f4dcab2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.352+0000 7f6d9e59c640 1 -- 172.21.15.39:0/2613063838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dc726210 con 0x55f4dc6e52f0 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.352+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/2613063838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab33c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6d90011900 tx=0x7f6d90011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2773903d8bcfd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.352+0000 7f6d94ff9640 1 -- 172.21.15.39:0/2613063838 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d900112f0 con 0x55f4dc6e52f0 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.353+0000 7f6d94ff9640 1 -- 172.21.15.39:0/2613063838 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d90010430 con 0x55f4dc6e52f0 2024-05-04T03:17:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.353+0000 7f6d94ff9640 1 -- 172.21.15.39:0/2613063838 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90019640 con 0x55f4dc6e52f0 2024-05-04T03:17:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.353+0000 7f6d977fe640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d8c0b28e0 0x7f6d8c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.354+0000 7f6d977fe640 1 -- 172.21.15.39:0/1240859180 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d980e1fe0 con 0x7f6da0108a80 2024-05-04T03:17:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.354+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d8c0b28e0 0x7f6d8c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.354+0000 7f6dab868740 1 -- 172.21.15.39:0/2613063838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 msgr2=0x55f4dcab33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.354+0000 7f6dab868740 1 --2- 172.21.15.39:0/2613063838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab33c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6d90011900 tx=0x7f6d90011dc0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.354+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d8c0b28e0 0x7f6d8c0b4df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009780 tx=0x7f6d88000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.355+0000 7f6dab868740 1 -- 172.21.15.39:0/2613063838 shutdown_connections 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.355+0000 7f6dab868740 1 --2- 172.21.15.39:0/2613063838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcaafad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.355+0000 7f6dab868740 1 --2- 172.21.15.39:0/2613063838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab33c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.355+0000 7f6dab868740 1 --2- 172.21.15.39:0/2613063838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 0x55f4dcab2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.355+0000 7f6dab868740 1 -- 172.21.15.39:0/2613063838 >> 172.21.15.39:0/2613063838 conn(0x55f4dc702300 msgr2=0x55f4dcaab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.355+0000 7f6dab868740 1 -- 172.21.15.39:0/2613063838 shutdown_connections 2024-05-04T03:17:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.355+0000 7f6dab868740 1 -- 172.21.15.39:0/2613063838 wait complete. 2024-05-04T03:17:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.356+0000 7f6dab868740 1 -- start start 2024-05-04T03:17:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.356+0000 7f6dab868740 1 Processor -- start 2024-05-04T03:17:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6dab868740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6dab868740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcab9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6d9dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcab9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6d9dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcab9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36982/0 (socket says 172.21.15.39:36982) 2024-05-04T03:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6dab868740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 0x55f4dcabe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59068/0 (socket says 172.21.15.39:59068) 2024-05-04T03:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6d9dd9b640 1 -- 172.21.15.39:0/3590865161 learned_addr learned my addr 172.21.15.39:0/3590865161 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc9d1370 con 0x55f4dc6e52f0 2024-05-04T03:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f4dcab3aa0 con 0x55f4dc70e1b0 2024-05-04T03:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f4dcabeef0 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.357+0000 7f6dab868740 10 client.0 _ll_register_callbacks cb 0x55f4dc725a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.358+0000 7f6d9ed9d640 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 0x55f4dcabe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.358+0000 7f6d9dd9b640 1 -- 172.21.15.39:0/3590865161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 msgr2=0x55f4dcabe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.358+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 0x55f4dcabe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.358+0000 7f6d9dd9b640 1 -- 172.21.15.39:0/3590865161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 msgr2=0x55f4dcab9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.358+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.358+0000 7f6d9dd9b640 1 -- 172.21.15.39:0/3590865161 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dc7f2620 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.359+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcab9a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005d00 tx=0x7f6d8802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a6f23f3838868d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.359+0000 7f6d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:17:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.359+0000 7f6d85ffb640 1 -- 172.21.15.39:0/3590865161 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8804f060 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.359+0000 7f6d85ffb640 1 -- 172.21.15.39:0/3590865161 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d88004450 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.359+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d90010430 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.359+0000 7f6d85ffb640 1 -- 172.21.15.39:0/3590865161 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6d88037050 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.359+0000 7f6dab868740 10 client.25699 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.359+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4dcad4940 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.360+0000 7f6d85ffb640 1 -- 172.21.15.39:0/3590865161 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d90010430 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.360+0000 7f6d85ffb640 1 -- 172.21.15.39:0/3590865161 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6d88004ad0 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.360+0000 7f6dab868740 10 client.25699 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.360+0000 7f6dab868740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.360+0000 7f6dab868740 20 client.25699 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.360+0000 7f6dab868740 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f4dc6ccd80 0x55f4dcad8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.360+0000 7f6dab868740 4 client.25699 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.360+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4dca221e0 con 0x55f4dc6ccd80 2024-05-04T03:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.361+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f4dc6ccd80 0x55f4dcad8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.362+0000 7f6d85ffb640 10 client.25699 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.362+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f4dc6ccd80 0x55f4dcad8140 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2928522187da630 server_cookie=fed4664a59c18890 in_seq=0 out_seq=0 2024-05-04T03:17:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6d85ffb640 1 -- 172.21.15.39:0/3590865161 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4dca221e0 con 0x55f4dc6ccd80 2024-05-04T03:17:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6d85ffb640 10 client.25699 handle_command_reply: tid=0 2024-05-04T03:17:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 client.25699 shutdown 2024-05-04T03:17:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 msgr2=0x55f4dcab9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcab9a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005d00 tx=0x7f6d8802a340 comp rx=0 tx=0).stop 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 shutdown_connections 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f4dc6ccd80 0x55f4dcad8140 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc70e1b0 0x55f4dcabe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcab9a60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 --2- 172.21.15.39:0/3590865161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcab9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 >> 172.21.15.39:0/3590865161 conn(0x55f4dc702300 msgr2=0x55f4dcaaa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6d85ffb640 0 client.25699 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 shutdown_connections 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.363+0000 7f6dab868740 1 -- 172.21.15.39:0/3590865161 wait complete. 2024-05-04T03:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.364+0000 7f6dab868740 20 client.25699 trim_cache size 0 max 16384 2024-05-04T03:17:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.393+0000 7f6dab868740 1 Processor -- start 2024-05-04T03:17:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.393+0000 7f6dab868740 1 -- start start 2024-05-04T03:17:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6dab868740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6e52f0 0x55f4dcaab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6dab868740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcaae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6dab868740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc70e1b0 0x55f4dcaae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6e52f0 0x55f4dcaab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6d9e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6e52f0 0x55f4dcaab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36986/0 (socket says 172.21.15.39:36986) 2024-05-04T03:17:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6d9e59c640 1 -- 172.21.15.39:0/1293311367 learned_addr learned my addr 172.21.15.39:0/1293311367 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6dab868740 1 -- 172.21.15.39:0/1293311367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8000e430 con 0x55f4dc70e1b0 2024-05-04T03:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcaae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.394+0000 7f6d9ed9d640 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc70e1b0 0x55f4dcaae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6dab868740 1 -- 172.21.15.39:0/1293311367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8000ff70 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6dab868740 1 -- 172.21.15.39:0/1293311367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8000eb00 con 0x55f4dc6e52f0 2024-05-04T03:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6d9dd9b640 1 -- 172.21.15.39:0/1293311367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6e52f0 msgr2=0x55f4dcaab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6e52f0 0x55f4dcaab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6d9dd9b640 1 -- 172.21.15.39:0/1293311367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc70e1b0 msgr2=0x55f4dcaae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc70e1b0 0x55f4dcaae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6d9dd9b640 1 -- 172.21.15.39:0/1293311367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dcaaff30 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6e52f0 0x55f4dcaab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.395+0000 7f6d9ed9d640 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc70e1b0 0x55f4dcaae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.396+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcaae2a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d88004ef0 tx=0x7f6d880097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c90f03fc4d647991 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.396+0000 7f6d87fff640 1 -- 172.21.15.39:0/1293311367 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8804f070 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.396+0000 7f6d87fff640 1 -- 172.21.15.39:0/1293311367 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d88046040 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.396+0000 7f6d87fff640 1 -- 172.21.15.39:0/1293311367 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8805a040 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.397+0000 7f6dab868740 1 -- 172.21.15.39:0/1293311367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 msgr2=0x55f4dcaae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.397+0000 7f6dab868740 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcaae2a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d88004ef0 tx=0x7f6d880097b0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 -- 172.21.15.39:0/1293311367 shutdown_connections 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc70e1b0 0x55f4dcaae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcaae2a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 --2- 172.21.15.39:0/1293311367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc6e52f0 0x55f4dcaab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 -- 172.21.15.39:0/1293311367 >> 172.21.15.39:0/1293311367 conn(0x55f4dc702300 msgr2=0x55f4dcaaacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 -- 172.21.15.39:0/1293311367 shutdown_connections 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 -- 172.21.15.39:0/1293311367 wait complete. 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 -- start start 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.398+0000 7f6dab868740 1 Processor -- start 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6dab868740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcae7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6dab868740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcae7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6dab868740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc70e1b0 0x55f4dcae8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6dab868740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8000f8a0 con 0x55f4dc6e52f0 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6dab868740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8000f1d0 con 0x55f4dc6ed4d0 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6dab868740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8000c410 con 0x55f4dc70e1b0 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6dab868740 10 client.0 _ll_register_callbacks cb 0x7f6d88035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc70e1b0 0x55f4dcae8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc70e1b0 0x55f4dcae8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:37002/0 (socket says 172.21.15.39:37002) 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9ed9d640 1 -- 172.21.15.39:0/1249962948 learned_addr learned my addr 172.21.15.39:0/1249962948 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcae7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcae7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9e59c640 1 -- 172.21.15.39:0/1249962948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc70e1b0 msgr2=0x55f4dcae8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc70e1b0 0x55f4dcae8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9e59c640 1 -- 172.21.15.39:0/1249962948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 msgr2=0x55f4dcae7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcae7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9e59c640 1 -- 172.21.15.39:0/1249962948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dcae8a50 con 0x55f4dc6e52f0 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.399+0000 7f6d9ed9d640 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc70e1b0 0x55f4dcae8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.400+0000 7f6d9e59c640 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcae7020 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55f4dcab4f60 tx=0x7f6d90010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32a887a2e2539f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.400+0000 7f6d85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.400+0000 7f6d85ffb640 1 -- 172.21.15.39:0/1249962948 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90034060 con 0x55f4dc6e52f0 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.400+0000 7f6d85ffb640 1 -- 172.21.15.39:0/1249962948 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d9000faa0 con 0x55f4dc6e52f0 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.400+0000 7f6dab868740 1 -- 172.21.15.39:0/1249962948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d88046040 con 0x55f4dc6e52f0 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.400+0000 7f6d85ffb640 1 -- 172.21.15.39:0/1249962948 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6d900109d0 con 0x55f4dc6e52f0 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.400+0000 7f6dab868740 10 client.16083 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.400+0000 7f6dab868740 1 -- 172.21.15.39:0/1249962948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4dcb40120 con 0x55f4dc6e52f0 2024-05-04T03:17:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.401+0000 7f6d85ffb640 1 -- 172.21.15.39:0/1249962948 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d88046040 con 0x55f4dc6e52f0 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.401+0000 7f6d85ffb640 1 -- 172.21.15.39:0/1249962948 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6d90018070 con 0x55f4dc6e52f0 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.401+0000 7f6dab868740 10 client.16083 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.401+0000 7f6dab868740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.401+0000 7f6dab868740 20 client.16083 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.401+0000 7f6dab868740 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f4dc6a21c0 0x55f4dcb42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.401+0000 7f6dab868740 4 client.16083 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.401+0000 7f6dab868740 1 -- 172.21.15.39:0/1249962948 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4dca1c050 con 0x55f4dc6a21c0 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.402+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f4dc6a21c0 0x55f4dcb42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.403+0000 7f6d9dd9b640 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f4dc6a21c0 0x55f4dcb42830 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d300802c3fc9bee server_cookie=da85724e7414a6d4 in_seq=0 out_seq=0 2024-05-04T03:17:29.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.403+0000 7f6d85ffb640 10 client.16083 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.410+0000 7f6d85ffb640 1 -- 172.21.15.39:0/1249962948 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4dca1c050 con 0x55f4dc6a21c0 2024-05-04T03:17:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.410+0000 7f6d85ffb640 10 client.16083 handle_command_reply: tid=0 2024-05-04T03:17:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.410+0000 7f6dab868740 1 client.16083 shutdown 2024-05-04T03:17:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 -- 172.21.15.39:0/1249962948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 msgr2=0x55f4dcae7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcae7020 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55f4dcab4f60 tx=0x7f6d90010660 comp rx=0 tx=0).stop 2024-05-04T03:17:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 -- 172.21.15.39:0/1249962948 shutdown_connections 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f4dc6a21c0 0x55f4dcb42830 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6d85ffb640 0 client.16083 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f4dc70e1b0 0x55f4dcae8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f4dc6ed4d0 0x55f4dcae7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 --2- 172.21.15.39:0/1249962948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4dc6e52f0 0x55f4dcae7020 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 -- 172.21.15.39:0/1249962948 >> 172.21.15.39:0/1249962948 conn(0x55f4dc702300 msgr2=0x55f4dcaae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 -- 172.21.15.39:0/1249962948 shutdown_connections 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.411+0000 7f6dab868740 1 -- 172.21.15.39:0/1249962948 wait complete. 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.412+0000 7f6dab868740 20 client.16083 trim_cache size 0 max 16384 2024-05-04T03:17:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:17:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:17:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:17:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:17:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.412+0000 7f6da49b2640 1 -- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d8c0b28e0 msgr2=0x7f6d8c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d8c0b28e0 0x7f6d8c0b4df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009780 tx=0x7f6d88000bc0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 -- 172.21.15.39:0/1240859180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da0108a80 msgr2=0x7f6da01a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da0108a80 0x7f6da01a1750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009fd0 tx=0x7f6d9800eba0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 -- 172.21.15.39:0/1240859180 shutdown_connections 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d8c0b28e0 0x7f6d8c0b4df0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6da0108a80 0x7f6da01a1750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6da0103d80 0x7f6da019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 --2- 172.21.15.39:0/1240859180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6da0102b40 0x7f6da019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 -- 172.21.15.39:0/1240859180 >> 172.21.15.39:0/1240859180 conn(0x7f6da00fdd30 msgr2=0x7f6da0074f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 -- 172.21.15.39:0/1240859180 shutdown_connections 2024-05-04T03:17:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:29.413+0000 7f6da49b2640 1 -- 172.21.15.39:0/1240859180 wait complete. 2024-05-04T03:17:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:30 smithi039 ceph-mon[29715]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-05-04T03:17:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:30 smithi066 ceph-mon[33264]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-05-04T03:17:30.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:30 smithi167 ceph-mon[33614]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-05-04T03:17:31.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 220.0 s, 1227.0 tps, lat 81.717 ms stddev 77.109 2024-05-04T03:17:31.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:31 smithi167 ceph-mon[33614]: pgmap v588: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 22 MiB/s wr, 1.33k op/s 2024-05-04T03:17:31.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:31 smithi039 ceph-mon[29715]: pgmap v588: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 22 MiB/s wr, 1.33k op/s 2024-05-04T03:17:32.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:31 smithi066 ceph-mon[33264]: pgmap v588: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 22 MiB/s wr, 1.33k op/s 2024-05-04T03:17:34.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:34 smithi066 ceph-mon[33264]: pgmap v589: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-04T03:17:34.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:34 smithi167 ceph-mon[33614]: pgmap v589: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-04T03:17:34.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:34 smithi039 ceph-mon[29715]: pgmap v589: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2024-05-04T03:17:36.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:36 smithi066 ceph-mon[33264]: pgmap v590: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-04T03:17:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:36 smithi167 ceph-mon[33614]: pgmap v590: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-04T03:17:36.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:36 smithi039 ceph-mon[29715]: pgmap v590: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-05-04T03:17:36.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 225.0 s, 1208.2 tps, lat 82.614 ms stddev 104.156 2024-05-04T03:17:38.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:38 smithi066 ceph-mon[33264]: pgmap v591: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-05-04T03:17:38.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:38 smithi167 ceph-mon[33614]: pgmap v591: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-05-04T03:17:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:38 smithi039 ceph-mon[29715]: pgmap v591: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-05-04T03:17:40.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:40 smithi066 ceph-mon[33264]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-04T03:17:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:40 smithi167 ceph-mon[33614]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-04T03:17:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:40 smithi039 ceph-mon[29715]: pgmap v592: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-04T03:17:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 230.0 s, 1335.4 tps, lat 74.803 ms stddev 72.219 2024-05-04T03:17:42.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:42 smithi066 ceph-mon[33264]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.32k op/s 2024-05-04T03:17:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.574+0000 7f4c8d48c640 1 -- 172.21.15.39:0/3165131851 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80030bb0 con 0x7f4c88118ba0 2024-05-04T03:17:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.575+0000 7f4c906f2640 1 -- 172.21.15.39:0/3165131851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 msgr2=0x7f4c8811afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.575+0000 7f4c906f2640 1 --2- 172.21.15.39:0/3165131851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 0x7f4c8811afc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009ca0 tx=0x7f4c8002a8b0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.575+0000 7f4c906f2640 1 -- 172.21.15.39:0/3165131851 shutdown_connections 2024-05-04T03:17:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.575+0000 7f4c906f2640 1 --2- 172.21.15.39:0/3165131851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 0x7f4c8811afc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.575+0000 7f4c906f2640 1 --2- 172.21.15.39:0/3165131851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8806d4f0 0x7f4c88118660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.575+0000 7f4c906f2640 1 --2- 172.21.15.39:0/3165131851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8806cba0 0x7f4c8806cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.575+0000 7f4c906f2640 1 -- 172.21.15.39:0/3165131851 >> 172.21.15.39:0/3165131851 conn(0x7f4c88072860 msgr2=0x7f4c88109cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.576+0000 7f4c906f2640 1 -- 172.21.15.39:0/3165131851 shutdown_connections 2024-05-04T03:17:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.576+0000 7f4c906f2640 1 -- 172.21.15.39:0/3165131851 wait complete. 2024-05-04T03:17:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.576+0000 7f4c906f2640 1 Processor -- start 2024-05-04T03:17:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c906f2640 1 -- start start 2024-05-04T03:17:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c906f2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8806cba0 0x7f4c881a9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c906f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8806d4f0 0x7f4c881a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c906f2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 0x7f4c881b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c906f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811e450 con 0x7f4c8806d4f0 2024-05-04T03:17:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c906f2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811e2d0 con 0x7f4c88118ba0 2024-05-04T03:17:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c906f2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8811e5d0 con 0x7f4c8806cba0 2024-05-04T03:17:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c8ec8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 0x7f4c881b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c8e48e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8806cba0 0x7f4c881a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c8ec8f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 0x7f4c881b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:60966/0 (socket says 172.21.15.39:60966) 2024-05-04T03:17:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.577+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/4241133631 learned_addr learned my addr 172.21.15.39:0/4241133631 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c8dc8d640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8806d4f0 0x7f4c881a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/4241133631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8806cba0 msgr2=0x7f4c881a9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8806cba0 0x7f4c881a9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8806d4f0 msgr2=0x7f4c881a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8806d4f0 0x7f4c881a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/4241133631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c881b0b70 con 0x7f4c88118ba0 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c8dc8d640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8806d4f0 0x7f4c881a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 0x7f4c881b0500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4c80030750 tx=0x7f4c80003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.578+0000 7f4c7f7fe640 1 -- 172.21.15.39:0/4241133631 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004240 con 0x7f4c88118ba0 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.579+0000 7f4c7f7fe640 1 -- 172.21.15.39:0/4241133631 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c80040070 con 0x7f4c88118ba0 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.579+0000 7f4c7f7fe640 1 -- 172.21.15.39:0/4241133631 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80045a50 con 0x7f4c88118ba0 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.579+0000 7f4c906f2640 1 -- 172.21.15.39:0/4241133631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c881b0e00 con 0x7f4c88118ba0 2024-05-04T03:17:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.579+0000 7f4c906f2640 1 -- 172.21.15.39:0/4241133631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c881b1210 con 0x7f4c88118ba0 2024-05-04T03:17:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.583+0000 7f4c7f7fe640 1 -- 172.21.15.39:0/4241133631 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4c80031440 con 0x7f4c88118ba0 2024-05-04T03:17:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.583+0000 7f4c975a1740 1 Processor -- start 2024-05-04T03:17:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.583+0000 7f4c975a1740 1 -- start start 2024-05-04T03:17:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e377ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e377cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3783ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c975a1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5618e36e15d0 con 0x5618e33db1b0 2024-05-04T03:17:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c975a1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5618e369a890 con 0x5618e33b22f0 2024-05-04T03:17:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c975a1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5618e36dd590 con 0x5618e33ba4d0 2024-05-04T03:17:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c8e48e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e377cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c8dc8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e377ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c8e48e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e377cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:60980/0 (socket says 172.21.15.39:60980) 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c8dc8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e377ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51370/0 (socket says 172.21.15.39:51370) 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c8e48e640 1 -- 172.21.15.39:0/3190600797 learned_addr learned my addr 172.21.15.39:0/3190600797 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.584+0000 7f4c8dc8d640 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3783ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c8e48e640 1 -- 172.21.15.39:0/3190600797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 msgr2=0x5618e3783ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3783ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c8e48e640 1 -- 172.21.15.39:0/3190600797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 msgr2=0x5618e377ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e377ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c8e48e640 1 -- 172.21.15.39:0/3190600797 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e33f3210 con 0x5618e33b22f0 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c8dc8d640 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3783ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e377cfe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009c90 tx=0x7f4c84011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7cade1ad04f830e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c7cff9640 1 -- 172.21.15.39:0/3190600797 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8401e070 con 0x5618e33b22f0 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c7cff9640 1 -- 172.21.15.39:0/3190600797 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c84019ac0 con 0x5618e33b22f0 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.585+0000 7f4c7cff9640 1 -- 172.21.15.39:0/3190600797 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400e900 con 0x5618e33b22f0 2024-05-04T03:17:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.586+0000 7f4c7f7fe640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c740b2860 0x7f4c740b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.586+0000 7f4c8dc8d640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c740b2860 0x7f4c740b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.586+0000 7f4c7f7fe640 1 -- 172.21.15.39:0/4241133631 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4c80103750 con 0x7f4c88118ba0 2024-05-04T03:17:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.587+0000 7f4c975a1740 1 -- 172.21.15.39:0/3190600797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 msgr2=0x5618e377cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.587+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e377cfe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009c90 tx=0x7f4c84011810 comp rx=0 tx=0).stop 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.587+0000 7f4c8dc8d640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c740b2860 0x7f4c740b4d70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5618e377dc30 tx=0x7f4c78009660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.587+0000 7f4c975a1740 1 -- 172.21.15.39:0/3190600797 shutdown_connections 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.587+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3783ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.587+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e377cfe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.587+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3190600797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e377ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.587+0000 7f4c975a1740 1 -- 172.21.15.39:0/3190600797 >> 172.21.15.39:0/3190600797 conn(0x5618e33cf300 msgr2=0x5618e37783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.588+0000 7f4c975a1740 1 -- 172.21.15.39:0/3190600797 shutdown_connections 2024-05-04T03:17:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.588+0000 7f4c975a1740 1 -- 172.21.15.39:0/3190600797 wait complete. 2024-05-04T03:17:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.588+0000 7f4c975a1740 1 -- start start 2024-05-04T03:17:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.589+0000 7f4c975a1740 1 Processor -- start 2024-05-04T03:17:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.589+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e37866e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.589+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3786c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.589+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e378db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c975a1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5618e3784930 con 0x5618e33db1b0 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c975a1740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5618e369e370 con 0x5618e33b22f0 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c975a1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5618e3784ab0 con 0x5618e33ba4d0 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8dc8d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3786c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c975a1740 10 client.0 _ll_register_callbacks cb 0x5618e33f2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8dc8d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3786c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:52792/0 (socket says 172.21.15.39:52792) 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8ec8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e378db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8dc8d640 1 -- 172.21.15.39:0/3158965163 learned_addr learned my addr 172.21.15.39:0/3158965163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e37866e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/3158965163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 msgr2=0x5618e3786c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3786c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/3158965163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 msgr2=0x5618e37866e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e37866e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.590+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/3158965163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e34bf620 con 0x5618e33db1b0 2024-05-04T03:17:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.591+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e378db80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4c800cb8a0 tx=0x7f4c80030f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ac2e469774bdd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.591+0000 7f4c71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:17:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.591+0000 7f4c71ffb640 1 -- 172.21.15.39:0/3158965163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800e1060 con 0x5618e33db1b0 2024-05-04T03:17:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.591+0000 7f4c71ffb640 1 -- 172.21.15.39:0/3158965163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4c800cc010 con 0x5618e33db1b0 2024-05-04T03:17:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.591+0000 7f4c975a1740 1 -- 172.21.15.39:0/3158965163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c84019ac0 con 0x5618e33db1b0 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.591+0000 7f4c71ffb640 1 -- 172.21.15.39:0/3158965163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c800cc2a0 con 0x5618e33db1b0 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.591+0000 7f4c975a1740 10 client.16092 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.591+0000 7f4c975a1740 1 -- 172.21.15.39:0/3158965163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618e3790a00 con 0x5618e33db1b0 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.592+0000 7f4c71ffb640 1 -- 172.21.15.39:0/3158965163 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c80036040 con 0x5618e33db1b0 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.592+0000 7f4c71ffb640 1 -- 172.21.15.39:0/3158965163 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c800d4570 con 0x5618e33db1b0 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.592+0000 7f4c975a1740 10 client.16092 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.592+0000 7f4c975a1740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.592+0000 7f4c975a1740 20 client.16092 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.592+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5618e3399d80 0x5618e37941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.592+0000 7f4c975a1740 4 client.16092 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.592+0000 7f4c975a1740 1 -- 172.21.15.39:0/3158965163 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c84019ac0 con 0x5618e3399d80 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.593+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5618e3399d80 0x5618e37941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.593+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5618e3399d80 0x5618e37941a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a4edd4a563371a server_cookie=92bc2a623416c862 in_seq=0 out_seq=0 2024-05-04T03:17:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.593+0000 7f4c71ffb640 10 client.16092 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.594+0000 7f4c71ffb640 1 -- 172.21.15.39:0/3158965163 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4c84019ac0 con 0x5618e3399d80 2024-05-04T03:17:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.594+0000 7f4c71ffb640 10 client.16092 handle_command_reply: tid=0 2024-05-04T03:17:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.594+0000 7f4c975a1740 1 client.16092 shutdown 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 -- 172.21.15.39:0/3158965163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 msgr2=0x5618e378db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e378db80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4c800cb8a0 tx=0x7f4c80030f80 comp rx=0 tx=0).stop 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 -- 172.21.15.39:0/3158965163 shutdown_connections 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5618e3399d80 0x5618e37941a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e378db80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33ba4d0 0x5618e3786c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3158965163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33b22f0 0x5618e37866e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 -- 172.21.15.39:0/3158965163 >> 172.21.15.39:0/3158965163 conn(0x5618e33cf300 msgr2=0x5618e377b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c71ffb640 0 client.16092 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 -- 172.21.15.39:0/3158965163 shutdown_connections 2024-05-04T03:17:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 1 -- 172.21.15.39:0/3158965163 wait complete. 2024-05-04T03:17:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.595+0000 7f4c975a1740 20 client.16092 trim_cache size 0 max 16384 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.625+0000 7f4c975a1740 1 Processor -- start 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c975a1740 1 -- start start 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33b22f0 0x5618e3777bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e3778100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c8e48e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33b22f0 0x5618e3777bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c8dc8d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c8e48e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33b22f0 0x5618e3777bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:52806/0 (socket says 172.21.15.39:52806) 2024-05-04T03:17:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c8e48e640 1 -- 172.21.15.39:0/3129787393 learned_addr learned my addr 172.21.15.39:0/3129787393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.626+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e3778100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.627+0000 7f4c975a1740 1 -- 172.21.15.39:0/3129787393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881ba000 con 0x5618e33db1b0 2024-05-04T03:17:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.627+0000 7f4c975a1740 1 -- 172.21.15.39:0/3129787393 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881bbb40 con 0x5618e33ba4d0 2024-05-04T03:17:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.627+0000 7f4c975a1740 1 -- 172.21.15.39:0/3129787393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881ba6d0 con 0x5618e33b22f0 2024-05-04T03:17:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.628+0000 7f4c8e48e640 1 -- 172.21.15.39:0/3129787393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 msgr2=0x5618e378daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.628+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.628+0000 7f4c8e48e640 1 -- 172.21.15.39:0/3129787393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 msgr2=0x5618e3778100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.628+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e3778100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.628+0000 7f4c8e48e640 1 -- 172.21.15.39:0/3129787393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618e3794760 con 0x5618e33b22f0 2024-05-04T03:17:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.628+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e3778100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.628+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33b22f0 0x5618e3777bc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4c8401c040 tx=0x7f4c8401af30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9644a2028ef7e60 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.629+0000 7f4c73fff640 1 -- 172.21.15.39:0/3129787393 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8401e040 con 0x5618e33b22f0 2024-05-04T03:17:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.629+0000 7f4c73fff640 1 -- 172.21.15.39:0/3129787393 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c84029070 con 0x5618e33b22f0 2024-05-04T03:17:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.629+0000 7f4c73fff640 1 -- 172.21.15.39:0/3129787393 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84003ce0 con 0x5618e33b22f0 2024-05-04T03:17:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.630+0000 7f4c975a1740 1 -- 172.21.15.39:0/3129787393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33b22f0 msgr2=0x5618e3777bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.630+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33b22f0 0x5618e3777bc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4c8401c040 tx=0x7f4c8401af30 comp rx=0 tx=0).stop 2024-05-04T03:17:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.631+0000 7f4c975a1740 1 -- 172.21.15.39:0/3129787393 shutdown_connections 2024-05-04T03:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.631+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33db1b0 0x5618e3778100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.631+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.631+0000 7f4c975a1740 1 --2- 172.21.15.39:0/3129787393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33b22f0 0x5618e3777bc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.631+0000 7f4c975a1740 1 -- 172.21.15.39:0/3129787393 >> 172.21.15.39:0/3129787393 conn(0x5618e33cf300 msgr2=0x5618e3780ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.631+0000 7f4c975a1740 1 -- 172.21.15.39:0/3129787393 shutdown_connections 2024-05-04T03:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.631+0000 7f4c975a1740 1 -- 172.21.15.39:0/3129787393 wait complete. 2024-05-04T03:17:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.632+0000 7f4c975a1740 1 -- start start 2024-05-04T03:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.632+0000 7f4c975a1740 1 Processor -- start 2024-05-04T03:17:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.633+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33b22f0 0x5618e378ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.633+0000 7f4c975a1740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.633+0000 7f4c8e48e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33b22f0 0x5618e378ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.633+0000 7f4c8dc8d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.633+0000 7f4c8dc8d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:60994/0 (socket says 172.21.15.39:60994) 2024-05-04T03:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.633+0000 7f4c8dc8d640 1 -- 172.21.15.39:0/955362881 learned_addr learned my addr 172.21.15.39:0/955362881 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c975a1740 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33db1b0 0x5618e378c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881bb470 con 0x5618e33b22f0 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881bada0 con 0x5618e33ba4d0 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881b7fe0 con 0x5618e33db1b0 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33db1b0 0x5618e378c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c975a1740 10 client.0 _ll_register_callbacks cb 0x7f4c80003f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/955362881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 msgr2=0x5618e378b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/955362881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33b22f0 msgr2=0x5618e378ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33b22f0 0x5618e378ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.634+0000 7f4c8ec8f640 1 -- 172.21.15.39:0/955362881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618e378c7d0 con 0x5618e33db1b0 2024-05-04T03:17:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c8ec8f640 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33db1b0 0x5618e378c210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c80003fb0 tx=0x7f4c8002afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82fd60ad85b90c8f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c71ffb640 1 -- 172.21.15.39:0/955362881 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800e1070 con 0x5618e33db1b0 2024-05-04T03:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c71ffb640 1 -- 172.21.15.39:0/955362881 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4c800cb010 con 0x5618e33db1b0 2024-05-04T03:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c84029070 con 0x5618e33db1b0 2024-05-04T03:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c71ffb640 1 -- 172.21.15.39:0/955362881 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c800cbba0 con 0x5618e33db1b0 2024-05-04T03:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c975a1740 10 client.25720 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5618e378dad0 con 0x5618e33db1b0 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.636+0000 7f4c71ffb640 1 -- 172.21.15.39:0/955362881 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c80036070 con 0x5618e33db1b0 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.637+0000 7f4c71ffb640 1 -- 172.21.15.39:0/955362881 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c800ca870 con 0x5618e33db1b0 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.637+0000 7f4c975a1740 10 client.25720 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.637+0000 7f4c975a1740 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.637+0000 7f4c975a1740 20 client.25720 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.637+0000 7f4c975a1740 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5618e336f1c0 0x5618e37fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.637+0000 7f4c975a1740 4 client.25720 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.637+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4c84029070 con 0x5618e336f1c0 2024-05-04T03:17:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.637+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5618e336f1c0 0x5618e37fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.638+0000 7f4c71ffb640 10 client.25720 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.638+0000 7f4c8e48e640 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5618e336f1c0 0x5618e37fe8a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce5b65566b5db1a server_cookie=9b6a50fbd42ec7ce in_seq=0 out_seq=0 2024-05-04T03:17:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c71ffb640 1 -- 172.21.15.39:0/955362881 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4c84029070 con 0x5618e336f1c0 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c71ffb640 10 client.25720 handle_command_reply: tid=0 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 client.25720 shutdown 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33db1b0 msgr2=0x5618e378c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33db1b0 0x5618e378c210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c80003fb0 tx=0x7f4c8002afb0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 shutdown_connections 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5618e336f1c0 0x5618e37fe8a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c71ffb640 0 client.25720 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5618e33db1b0 0x5618e378c210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5618e33ba4d0 0x5618e378b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 --2- 172.21.15.39:0/955362881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618e33b22f0 0x5618e378ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.647+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 >> 172.21.15.39:0/955362881 conn(0x5618e33cf300 msgr2=0x5618e377b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.648+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 shutdown_connections 2024-05-04T03:17:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.648+0000 7f4c975a1740 1 -- 172.21.15.39:0/955362881 wait complete. 2024-05-04T03:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.648+0000 7f4c975a1740 20 client.25720 trim_cache size 0 max 16384 2024-05-04T03:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:17:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 -- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c740b2860 msgr2=0x7f4c740b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c740b2860 0x7f4c740b4d70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5618e377dc30 tx=0x7f4c78009660 comp rx=0 tx=0).stop 2024-05-04T03:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 -- 172.21.15.39:0/4241133631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 msgr2=0x7f4c881b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 0x7f4c881b0500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4c80030750 tx=0x7f4c80003b80 comp rx=0 tx=0).stop 2024-05-04T03:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 -- 172.21.15.39:0/4241133631 shutdown_connections 2024-05-04T03:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c740b2860 0x7f4c740b4d70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c88118ba0 0x7f4c881b0500 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8806d4f0 0x7f4c881a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 --2- 172.21.15.39:0/4241133631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8806cba0 0x7f4c881a9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 -- 172.21.15.39:0/4241133631 >> 172.21.15.39:0/4241133631 conn(0x7f4c88072860 msgr2=0x7f4c8810b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 -- 172.21.15.39:0/4241133631 shutdown_connections 2024-05-04T03:17:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:42.649+0000 7f4c906f2640 1 -- 172.21.15.39:0/4241133631 wait complete. 2024-05-04T03:17:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:42 smithi167 ceph-mon[33614]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.32k op/s 2024-05-04T03:17:42.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:42 smithi039 ceph-mon[29715]: pgmap v593: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.32k op/s 2024-05-04T03:17:44.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:44 smithi066 ceph-mon[33264]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2024-05-04T03:17:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:44 smithi167 ceph-mon[33614]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2024-05-04T03:17:44.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:44 smithi039 ceph-mon[29715]: pgmap v594: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2024-05-04T03:17:46.494 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:46 smithi039 ceph-mon[29715]: pgmap v595: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 19 MiB/s wr, 1.30k op/s 2024-05-04T03:17:46.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:46 smithi066 ceph-mon[33264]: pgmap v595: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 19 MiB/s wr, 1.30k op/s 2024-05-04T03:17:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:46 smithi167 ceph-mon[33614]: pgmap v595: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 19 MiB/s wr, 1.30k op/s 2024-05-04T03:17:46.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 235.0 s, 1205.8 tps, lat 82.823 ms stddev 79.325 2024-05-04T03:17:48.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:48 smithi066 ceph-mon[33264]: pgmap v596: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-05-04T03:17:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:48 smithi167 ceph-mon[33614]: pgmap v596: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-05-04T03:17:48.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:48 smithi039 ceph-mon[29715]: pgmap v596: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-05-04T03:17:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:50 smithi066 ceph-mon[33264]: pgmap v597: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 19 MiB/s wr, 1.37k op/s 2024-05-04T03:17:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:50 smithi167 ceph-mon[33614]: pgmap v597: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 19 MiB/s wr, 1.37k op/s 2024-05-04T03:17:50.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:50 smithi039 ceph-mon[29715]: pgmap v597: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 19 MiB/s wr, 1.37k op/s 2024-05-04T03:17:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 240.0 s, 1359.4 tps, lat 73.765 ms stddev 18.206 2024-05-04T03:17:52.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:52 smithi066 ceph-mon[33264]: pgmap v598: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-05-04T03:17:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:52 smithi167 ceph-mon[33614]: pgmap v598: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-05-04T03:17:52.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:52 smithi039 ceph-mon[29715]: pgmap v598: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2024-05-04T03:17:53.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:53 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:17:53.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:53 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:17:53.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:53 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:17:54.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:54 smithi066 ceph-mon[33264]: pgmap v599: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-04T03:17:54.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:54 smithi167 ceph-mon[33614]: pgmap v599: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-04T03:17:54.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:54 smithi039 ceph-mon[29715]: pgmap v599: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-05-04T03:17:55.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:17:55.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:17:55.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:17:55.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:17:55.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:17:55.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:17:55.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:17:55.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:17:55.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:17:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.810+0000 7f9120c3b640 1 -- 172.21.15.39:0/4273960358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 msgr2=0x7f911c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.810+0000 7f9120c3b640 1 --2- 172.21.15.39:0/4273960358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 0x7f911c103be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9114009be0 tx=0x7f911402a6c0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.810+0000 7f9120c3b640 1 -- 172.21.15.39:0/4273960358 shutdown_connections 2024-05-04T03:17:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.810+0000 7f9120c3b640 1 --2- 172.21.15.39:0/4273960358 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 0x7f911c103be0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.810+0000 7f9120c3b640 1 --2- 172.21.15.39:0/4273960358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c102e80 0x7f911c103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.810+0000 7f9120c3b640 1 --2- 172.21.15.39:0/4273960358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c102530 0x7f911c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.810+0000 7f9120c3b640 1 -- 172.21.15.39:0/4273960358 >> 172.21.15.39:0/4273960358 conn(0x7f911c0fdd40 msgr2=0x7f911c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.810+0000 7f9120c3b640 1 -- 172.21.15.39:0/4273960358 shutdown_connections 2024-05-04T03:17:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.811+0000 7f9120c3b640 1 -- 172.21.15.39:0/4273960358 wait complete. 2024-05-04T03:17:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.812+0000 7f9120c3b640 1 Processor -- start 2024-05-04T03:17:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.812+0000 7f9120c3b640 1 -- start start 2024-05-04T03:17:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.812+0000 7f9120c3b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c102530 0x7f911c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.812+0000 7f9120c3b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c102e80 0x7f911c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c102530 0x7f911c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c102530 0x7f911c19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:49290/0 (socket says 172.21.15.39:49290) 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f9120c3b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 0x7f911c1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f9119d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c102e80 0x7f911c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911a59c640 1 -- 172.21.15.39:0/93911650 learned_addr learned my addr 172.21.15.39:0/93911650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f911c111810 con 0x7f911c102e80 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f911c111690 con 0x7f911c1037d0 2024-05-04T03:17:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f911c111990 con 0x7f911c102530 2024-05-04T03:17:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911ad9d640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 0x7f911c1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911ad9d640 1 -- 172.21.15.39:0/93911650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c102530 msgr2=0x7f911c19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911ad9d640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c102530 0x7f911c19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911ad9d640 1 -- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c102e80 msgr2=0x7f911c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911ad9d640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c102e80 0x7f911c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.813+0000 7f911ad9d640 1 -- 172.21.15.39:0/93911650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c1a3f60 con 0x7f911c1037d0 2024-05-04T03:17:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.814+0000 7f911ad9d640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 0x7f911c1a38f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9114034040 tx=0x7f91140305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.814+0000 7f910b7fe640 1 -- 172.21.15.39:0/93911650 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114041020 con 0x7f911c1037d0 2024-05-04T03:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.814+0000 7f910b7fe640 1 -- 172.21.15.39:0/93911650 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9114037040 con 0x7f911c1037d0 2024-05-04T03:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.814+0000 7f910b7fe640 1 -- 172.21.15.39:0/93911650 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114036070 con 0x7f911c1037d0 2024-05-04T03:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.814+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c1a41f0 con 0x7f911c1037d0 2024-05-04T03:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.814+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c1a4660 con 0x7f911c1037d0 2024-05-04T03:17:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.819+0000 7f9127bc6740 1 Processor -- start 2024-05-04T03:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.821+0000 7f910b7fe640 1 -- 172.21.15.39:0/93911650 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9114044020 con 0x7f911c1037d0 2024-05-04T03:17:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.822+0000 7f9127bc6740 1 -- start start 2024-05-04T03:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.822+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41921b0 0x556ee4533a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.822+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41692f0 0x556ee4533fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.822+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41714d0 0x556ee453aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.822+0000 7f9127bc6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556ee44985d0 con 0x556ee41714d0 2024-05-04T03:17:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.822+0000 7f9127bc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556ee4451890 con 0x556ee41692f0 2024-05-04T03:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.822+0000 7f9127bc6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556ee4494590 con 0x556ee41921b0 2024-05-04T03:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41921b0 0x556ee4533a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41921b0 0x556ee4533a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:49294/0 (socket says 172.21.15.39:49294) 2024-05-04T03:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 -- 172.21.15.39:0/2615132812 learned_addr learned my addr 172.21.15.39:0/2615132812 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f911a59c640 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41692f0 0x556ee4533fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f910b7fe640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91040b2930 0x7f91040b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41714d0 0x556ee453aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f911ad9d640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91040b2930 0x7f91040b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 -- 172.21.15.39:0/2615132812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41692f0 msgr2=0x556ee4533fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41692f0 0x556ee4533fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 -- 172.21.15.39:0/2615132812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41714d0 msgr2=0x556ee453aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41714d0 0x556ee453aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 -- 172.21.15.39:0/2615132812 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ee41aa210 con 0x556ee41921b0 2024-05-04T03:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f910b7fe640 1 -- 172.21.15.39:0/93911650 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f91141028b0 con 0x7f911c1037d0 2024-05-04T03:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.823+0000 7f9119d9b640 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41921b0 0x556ee4533a60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f910c011470 tx=0x7f910c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=783d9b0c4827fab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.824+0000 7f9108ff9640 1 -- 172.21.15.39:0/2615132812 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c005020 con 0x556ee41921b0 2024-05-04T03:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.824+0000 7f9108ff9640 1 -- 172.21.15.39:0/2615132812 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f910c0047a0 con 0x556ee41921b0 2024-05-04T03:17:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.824+0000 7f9108ff9640 1 -- 172.21.15.39:0/2615132812 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c00e5e0 con 0x556ee41921b0 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f911ad9d640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91040b2930 0x7f91040b4e40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f91140c9ef0 tx=0x7f91140c4520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 -- 172.21.15.39:0/2615132812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41921b0 msgr2=0x556ee4533a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41921b0 0x556ee4533a60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f910c011470 tx=0x7f910c0099e0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 -- 172.21.15.39:0/2615132812 shutdown_connections 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41714d0 0x556ee453aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41692f0 0x556ee4533fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 --2- 172.21.15.39:0/2615132812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41921b0 0x556ee4533a60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 -- 172.21.15.39:0/2615132812 >> 172.21.15.39:0/2615132812 conn(0x556ee4186300 msgr2=0x556ee452f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 -- 172.21.15.39:0/2615132812 shutdown_connections 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.825+0000 7f9127bc6740 1 -- 172.21.15.39:0/2615132812 wait complete. 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.826+0000 7f9127bc6740 1 -- start start 2024-05-04T03:17:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.826+0000 7f9127bc6740 1 Processor -- start 2024-05-04T03:17:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41692f0 0x556ee453d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41714d0 0x556ee453dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9119d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41692f0 0x556ee453d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41921b0 0x556ee4544b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9127bc6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556ee453b930 con 0x556ee41692f0 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9119d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41692f0 0x556ee453d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41182/0 (socket says 172.21.15.39:41182) 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9127bc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556ee4455370 con 0x556ee41921b0 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9119d9b640 1 -- 172.21.15.39:0/1465452966 learned_addr learned my addr 172.21.15.39:0/1465452966 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556ee453bab0 con 0x556ee41714d0 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f911a59c640 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41714d0 0x556ee453dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9119d9b640 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41921b0 0x556ee4544b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9119d9b640 1 -- 172.21.15.39:0/1465452966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41714d0 msgr2=0x556ee453dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.827+0000 7f9127bc6740 10 client.0 _ll_register_callbacks cb 0x556ee41a9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9119d9b640 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41714d0 0x556ee453dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9119d9b640 1 -- 172.21.15.39:0/1465452966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41921b0 msgr2=0x556ee4544b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9119d9b640 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41921b0 0x556ee4544b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9119d9b640 1 -- 172.21.15.39:0/1465452966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ee4276620 con 0x556ee41692f0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f911a59c640 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41714d0 0x556ee453dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910c0047a0 con 0x556ee41692f0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9119d9b640 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41692f0 0x556ee453d6a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f910c004e80 tx=0x7f910c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12822b74f5ddc375 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9101ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9101ffb640 1 -- 172.21.15.39:0/1465452966 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c022030 con 0x556ee41692f0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9101ffb640 1 -- 172.21.15.39:0/1465452966 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f910c030be0 con 0x556ee41692f0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9101ffb640 1 -- 172.21.15.39:0/1465452966 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f910c030de0 con 0x556ee41692f0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9127bc6740 10 client.16110 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.828+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({fsmap=32}) -- 0x556ee45479e0 con 0x556ee41692f0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.829+0000 7f9101ffb640 1 -- 172.21.15.39:0/1465452966 <== mon.0 v2:172.21.15.39:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f910c01a040 con 0x556ee41692f0 2024-05-04T03:17:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.829+0000 7f9127bc6740 10 client.16110 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.829+0000 7f9127bc6740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.829+0000 7f9127bc6740 20 client.16110 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.829+0000 7f9127bc6740 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556ee4150d80 0x556ee454ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.829+0000 7f9127bc6740 4 client.16110 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.829+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ee44a61e0 con 0x556ee4150d80 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.829+0000 7f911a59c640 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556ee4150d80 0x556ee454ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.830+0000 7f911a59c640 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556ee4150d80 0x556ee454ad40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d297d346939553 server_cookie=d79d932b75df3abb in_seq=0 out_seq=0 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.830+0000 7f9101ffb640 10 client.16110 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.831+0000 7f9101ffb640 1 -- 172.21.15.39:0/1465452966 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ee44a61e0 con 0x556ee4150d80 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.831+0000 7f9101ffb640 10 client.16110 handle_command_reply: tid=0 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.831+0000 7f9127bc6740 1 client.16110 shutdown 2024-05-04T03:17:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.831+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41692f0 msgr2=0x556ee453d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.831+0000 7f9127bc6740 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41692f0 0x556ee453d6a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f910c004e80 tx=0x7f910c009260 comp rx=0 tx=0).stop 2024-05-04T03:17:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.831+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 shutdown_connections 2024-05-04T03:17:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9127bc6740 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556ee4150d80 0x556ee454ad40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9127bc6740 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41921b0 0x556ee4544b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9101ffb640 0 client.16110 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9127bc6740 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41714d0 0x556ee453dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9127bc6740 1 --2- 172.21.15.39:0/1465452966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee41692f0 0x556ee453d6a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 >> 172.21.15.39:0/1465452966 conn(0x556ee4186300 msgr2=0x556ee4532300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 shutdown_connections 2024-05-04T03:17:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9127bc6740 1 -- 172.21.15.39:0/1465452966 wait complete. 2024-05-04T03:17:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.832+0000 7f9127bc6740 20 client.16110 trim_cache size 0 max 16384 2024-05-04T03:17:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.858+0000 7f9127bc6740 1 Processor -- start 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.858+0000 7f9127bc6740 1 -- start start 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee453d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee453d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee454f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9127bc6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1a8fb0 con 0x556ee4150d80 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9127bc6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1aaaf0 con 0x556ee41714d0 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9127bc6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1a9680 con 0x556ee41692f0 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee453d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f911a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee453d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee453d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41192/0 (socket says 172.21.15.39:41192) 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 -- 172.21.15.39:0/530127428 learned_addr learned my addr 172.21.15.39:0/530127428 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 --2- 172.21.15.39:0/530127428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee454f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 -- 172.21.15.39:0/530127428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 msgr2=0x556ee453d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 --2- 172.21.15.39:0/530127428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee453d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 -- 172.21.15.39:0/530127428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 msgr2=0x556ee454f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 --2- 172.21.15.39:0/530127428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee454f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.859+0000 7f9119d9b640 1 -- 172.21.15.39:0/530127428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ee453e710 con 0x556ee4150d80 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.860+0000 7f9119d9b640 1 --2- 172.21.15.39:0/530127428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee453d270 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f910c000c00 tx=0x7f910c027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44271165313302b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.861+0000 7f9103fff640 1 -- 172.21.15.39:0/530127428 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c022040 con 0x556ee4150d80 2024-05-04T03:17:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.861+0000 7f9103fff640 1 -- 172.21.15.39:0/530127428 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f910c0047a0 con 0x556ee4150d80 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.861+0000 7f9103fff640 1 -- 172.21.15.39:0/530127428 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c0035f0 con 0x556ee4150d80 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.862+0000 7f9127bc6740 1 -- 172.21.15.39:0/530127428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 msgr2=0x556ee453d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.862+0000 7f9127bc6740 1 --2- 172.21.15.39:0/530127428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee453d270 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f910c000c00 tx=0x7f910c027000 comp rx=0 tx=0).stop 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.863+0000 7f9127bc6740 1 -- 172.21.15.39:0/530127428 shutdown_connections 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.863+0000 7f9127bc6740 1 --2- 172.21.15.39:0/530127428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee454f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.863+0000 7f9127bc6740 1 --2- 172.21.15.39:0/530127428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee453d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.863+0000 7f9127bc6740 1 --2- 172.21.15.39:0/530127428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee453d270 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.863+0000 7f9127bc6740 1 -- 172.21.15.39:0/530127428 >> 172.21.15.39:0/530127428 conn(0x556ee4186300 msgr2=0x556ee4537b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.863+0000 7f9127bc6740 1 -- 172.21.15.39:0/530127428 shutdown_connections 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.863+0000 7f9127bc6740 1 -- 172.21.15.39:0/530127428 wait complete. 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.863+0000 7f9127bc6740 1 -- start start 2024-05-04T03:17:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.864+0000 7f9127bc6740 1 Processor -- start 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee4543140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee4543680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f9127bc6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee45b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f911a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee4543680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f9119d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee4543140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f911a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee4543680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:49320/0 (socket says 172.21.15.39:49320) 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f911a59c640 1 -- 172.21.15.39:0/3231388117 learned_addr learned my addr 172.21.15.39:0/3231388117 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f9127bc6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1aa420 con 0x556ee4150d80 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1a9d50 con 0x556ee41714d0 2024-05-04T03:17:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.865+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1a6f90 con 0x556ee41692f0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f9127bc6740 10 client.0 _ll_register_callbacks cb 0x7f9110009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f9119d9b640 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee45b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f911a59c640 1 -- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 msgr2=0x556ee45b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f911a59c640 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee45b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f911a59c640 1 -- 172.21.15.39:0/3231388117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 msgr2=0x556ee4543140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f911a59c640 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee4543140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f911a59c640 1 -- 172.21.15.39:0/3231388117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ee45b1620 con 0x556ee41692f0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f9119d9b640 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee45b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f911a59c640 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee4543680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x556ee4534e40 tx=0x7f9110010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3138c06a4518870 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.866+0000 7f9101ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.867+0000 7f9101ffb640 1 -- 172.21.15.39:0/3231388117 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110026060 con 0x556ee41692f0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.867+0000 7f9101ffb640 1 -- 172.21.15.39:0/3231388117 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9110004440 con 0x556ee41692f0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.867+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910c0047a0 con 0x556ee41692f0 2024-05-04T03:17:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.867+0000 7f9101ffb640 1 -- 172.21.15.39:0/3231388117 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f911000f870 con 0x556ee41692f0 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.867+0000 7f9127bc6740 10 client.25738 fetch_fsmap learned FSMap version 32 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.867+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556ee45b2840 con 0x556ee41692f0 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.867+0000 7f9101ffb640 1 -- 172.21.15.39:0/3231388117 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f910c0047a0 con 0x556ee41692f0 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.867+0000 7f9101ffb640 1 -- 172.21.15.39:0/3231388117 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9110003c40 con 0x556ee41692f0 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.868+0000 7f9127bc6740 10 client.25738 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.868+0000 7f9127bc6740 10 client.25738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.868+0000 7f9127bc6740 20 client.25738 populate_metadata read hostname 'smithi039' 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.868+0000 7f9127bc6740 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556ee41921b0 0x556ee45b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.868+0000 7f9127bc6740 4 client.25738 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.868+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ee44a0050 con 0x556ee41921b0 2024-05-04T03:17:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.868+0000 7f9119d9b640 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556ee41921b0 0x556ee45b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.868+0000 7f9101ffb640 10 client.25738 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.869+0000 7f9119d9b640 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556ee41921b0 0x556ee45b51b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bead8092e6f53445 server_cookie=ae0d6eeeafa8dc2a in_seq=0 out_seq=0 2024-05-04T03:17:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.878+0000 7f9101ffb640 1 -- 172.21.15.39:0/3231388117 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ee44a0050 con 0x556ee41921b0 2024-05-04T03:17:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.878+0000 7f9101ffb640 10 client.25738 handle_command_reply: tid=0 2024-05-04T03:17:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.878+0000 7f9127bc6740 1 client.25738 shutdown 2024-05-04T03:17:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.878+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 msgr2=0x556ee4543680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.878+0000 7f9127bc6740 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee4543680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x556ee4534e40 tx=0x7f9110010c10 comp rx=0 tx=0).stop 2024-05-04T03:17:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.878+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 shutdown_connections 2024-05-04T03:17:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.878+0000 7f9127bc6740 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556ee41921b0 0x556ee45b51b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.879+0000 7f9127bc6740 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556ee41714d0 0x556ee45b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.879+0000 7f9101ffb640 0 client.25738 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:17:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.879+0000 7f9127bc6740 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556ee41692f0 0x556ee4543680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.879+0000 7f9127bc6740 1 --2- 172.21.15.39:0/3231388117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ee4150d80 0x556ee4543140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.879+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 >> 172.21.15.39:0/3231388117 conn(0x556ee4186300 msgr2=0x556ee453d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.879+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 shutdown_connections 2024-05-04T03:17:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.879+0000 7f9127bc6740 1 -- 172.21.15.39:0/3231388117 wait complete. 2024-05-04T03:17:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.879+0000 7f9127bc6740 20 client.25738 trim_cache size 0 max 16384 2024-05-04T03:17:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:17:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:17:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:17:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:17:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91040b2930 msgr2=0x7f91040b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91040b2930 0x7f91040b4e40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f91140c9ef0 tx=0x7f91140c4520 comp rx=0 tx=0).stop 2024-05-04T03:17:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 msgr2=0x7f911c1a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 0x7f911c1a38f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9114034040 tx=0x7f91140305d0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 shutdown_connections 2024-05-04T03:17:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91040b2930 0x7f91040b4e40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f911c1037d0 0x7f911c1a38f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c102e80 0x7f911c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 --2- 172.21.15.39:0/93911650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c102530 0x7f911c19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.881+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 >> 172.21.15.39:0/93911650 conn(0x7f911c0fdd40 msgr2=0x7f911c10a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.882+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 shutdown_connections 2024-05-04T03:17:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:17:55.882+0000 7f9120c3b640 1 -- 172.21.15.39:0/93911650 wait complete. 2024-05-04T03:17:56.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:56 smithi167 ceph-mon[33614]: pgmap v600: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-05-04T03:17:56.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:56 smithi039 ceph-mon[29715]: pgmap v600: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-05-04T03:17:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 245.0 s, 1390.0 tps, lat 71.870 ms stddev 21.214 2024-05-04T03:17:56.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:56 smithi066 ceph-mon[33264]: pgmap v600: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2024-05-04T03:17:58.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:17:58 smithi167 ceph-mon[33614]: pgmap v601: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-05-04T03:17:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:17:58 smithi039 ceph-mon[29715]: pgmap v601: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-05-04T03:17:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:17:58 smithi066 ceph-mon[33264]: pgmap v601: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 16 MiB/s wr, 1.42k op/s 2024-05-04T03:18:00.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:00 smithi167 ceph-mon[33614]: pgmap v602: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-05-04T03:18:00.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:00 smithi039 ceph-mon[29715]: pgmap v602: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-05-04T03:18:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:00 smithi066 ceph-mon[33264]: pgmap v602: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-05-04T03:18:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 250.0 s, 1476.6 tps, lat 67.726 ms stddev 18.179 2024-05-04T03:18:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:02 smithi167 ceph-mon[33614]: pgmap v603: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.41k op/s 2024-05-04T03:18:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:02 smithi039 ceph-mon[29715]: pgmap v603: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.41k op/s 2024-05-04T03:18:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:02 smithi066 ceph-mon[33264]: pgmap v603: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.41k op/s 2024-05-04T03:18:04.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:04 smithi167 ceph-mon[33614]: pgmap v604: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-05-04T03:18:04.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:04 smithi039 ceph-mon[29715]: pgmap v604: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-05-04T03:18:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:04 smithi066 ceph-mon[33264]: pgmap v604: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 17 MiB/s wr, 1.62k op/s 2024-05-04T03:18:06.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:06 smithi167 ceph-mon[33614]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-05-04T03:18:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:06 smithi039 ceph-mon[29715]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-05-04T03:18:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 255.0 s, 1504.8 tps, lat 66.428 ms stddev 16.547 2024-05-04T03:18:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:06 smithi066 ceph-mon[33264]: pgmap v605: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 15 MiB/s wr, 1.46k op/s 2024-05-04T03:18:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:08 smithi167 ceph-mon[33614]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-05-04T03:18:08.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:08 smithi039 ceph-mon[29715]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-05-04T03:18:08.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:08 smithi066 ceph-mon[33264]: pgmap v606: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-05-04T03:18:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.045+0000 7ff5fa868640 1 -- 172.21.15.39:0/4070466700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 msgr2=0x7ff5f410aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.045+0000 7ff5fa868640 1 --2- 172.21.15.39:0/4070466700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 0x7ff5f410aee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009ca0 tx=0x7ff5ec02a8c0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.045+0000 7ff5fa868640 1 -- 172.21.15.39:0/4070466700 shutdown_connections 2024-05-04T03:18:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.045+0000 7ff5fa868640 1 --2- 172.21.15.39:0/4070466700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 0x7ff5f410aee0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.045+0000 7ff5fa868640 1 --2- 172.21.15.39:0/4070466700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5f4103dc0 0x7ff5f41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.045+0000 7ff5fa868640 1 --2- 172.21.15.39:0/4070466700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5f4102b80 0x7ff5f4103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.045+0000 7ff5fa868640 1 -- 172.21.15.39:0/4070466700 >> 172.21.15.39:0/4070466700 conn(0x7ff5f40fdd90 msgr2=0x7ff5f4100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.046+0000 7ff5fa868640 1 -- 172.21.15.39:0/4070466700 shutdown_connections 2024-05-04T03:18:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.046+0000 7ff5fa868640 1 -- 172.21.15.39:0/4070466700 wait complete. 2024-05-04T03:18:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.047+0000 7ff5fa868640 1 Processor -- start 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.047+0000 7ff5fa868640 1 -- start start 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.047+0000 7ff5fa868640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5f4102b80 0x7ff5f41a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.047+0000 7ff5fa868640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5f4103dc0 0x7ff5f41a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.047+0000 7ff5fa868640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 0x7ff5f41a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5fa868640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f410d490 con 0x7ff5f4102b80 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5fa868640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f410d310 con 0x7ff5f4103dc0 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5fa868640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f410d610 con 0x7ff5f4108ac0 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5f4102b80 0x7ff5f41a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5f4102b80 0x7ff5f41a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57138/0 (socket says 172.21.15.39:57138) 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5f4103dc0 0x7ff5f41a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f3fff640 1 -- 172.21.15.39:0/2776290654 learned_addr learned my addr 172.21.15.39:0/2776290654 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 0x7ff5f41a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f37fe640 1 -- 172.21.15.39:0/2776290654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 msgr2=0x7ff5f41a7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 0x7ff5f41a7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f37fe640 1 -- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5f4102b80 msgr2=0x7ff5f41a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5f4102b80 0x7ff5f41a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.048+0000 7ff5f37fe640 1 -- 172.21.15.39:0/2776290654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f41a8290 con 0x7ff5f4103dc0 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.049+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 0x7ff5f41a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.049+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5f4102b80 0x7ff5f41a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.049+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5f4103dc0 0x7ff5f41a0c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800e9f0 tx=0x7ff5e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.049+0000 7ff5f17fa640 1 -- 172.21.15.39:0/2776290654 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800ccc0 con 0x7ff5f4103dc0 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.049+0000 7ff5f17fa640 1 -- 172.21.15.39:0/2776290654 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5e8005020 con 0x7ff5f4103dc0 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.049+0000 7ff5f17fa640 1 -- 172.21.15.39:0/2776290654 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800ce60 con 0x7ff5f4103dc0 2024-05-04T03:18:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.049+0000 7ff5fa868640 1 -- 172.21.15.39:0/2776290654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f41a8580 con 0x7ff5f4103dc0 2024-05-04T03:18:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.049+0000 7ff5fa868640 1 -- 172.21.15.39:0/2776290654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f41a8ad0 con 0x7ff5f4103dc0 2024-05-04T03:18:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.052+0000 7ff6017f2740 1 Processor -- start 2024-05-04T03:18:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff6017f2740 1 -- start start 2024-05-04T03:18:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff5f17fa640 1 -- 172.21.15.39:0/2776290654 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff5e8004a60 con 0x7ff5f4103dc0 2024-05-04T03:18:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe7b1b0 0x55aacc21ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc21cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe5a4d0 0x55aacc223ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff6017f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55aacc1815d0 con 0x55aacbe7b1b0 2024-05-04T03:18:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff6017f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55aacc13a890 con 0x55aacbe522f0 2024-05-04T03:18:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff6017f2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aacc17d590 con 0x55aacbe5a4d0 2024-05-04T03:18:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff5f8e05640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe7b1b0 0x55aacc21ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff5f8e05640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe7b1b0 0x55aacc21ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57150/0 (socket says 172.21.15.39:57150) 2024-05-04T03:18:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.053+0000 7ff5f8e05640 1 -- 172.21.15.39:0/2198592466 learned_addr learned my addr 172.21.15.39:0/2198592466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc21cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe5a4d0 0x55aacc223ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f8e05640 1 -- 172.21.15.39:0/2198592466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe5a4d0 msgr2=0x55aacc223ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe5a4d0 0x55aacc223ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f8e05640 1 -- 172.21.15.39:0/2198592466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 msgr2=0x55aacc21cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc21cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f8e05640 1 -- 172.21.15.39:0/2198592466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacbe93210 con 0x55aacbe7b1b0 2024-05-04T03:18:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc21cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:18:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.054+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe7b1b0 0x55aacc21ca60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec004af0 tx=0x7ff5ec005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2500de41a3fc6cb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.055+0000 7ff5da7fc640 1 -- 172.21.15.39:0/2198592466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec031650 con 0x55aacbe7b1b0 2024-05-04T03:18:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.055+0000 7ff5da7fc640 1 -- 172.21.15.39:0/2198592466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5ec036070 con 0x55aacbe7b1b0 2024-05-04T03:18:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.055+0000 7ff5da7fc640 1 -- 172.21.15.39:0/2198592466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec004760 con 0x55aacbe7b1b0 2024-05-04T03:18:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.055+0000 7ff5f17fa640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5dc0b28a0 0x7ff5dc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.056+0000 7ff5f17fa640 1 -- 172.21.15.39:0/2776290654 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff5e80e2210 con 0x7ff5f4103dc0 2024-05-04T03:18:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.056+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5dc0b28a0 0x7ff5dc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.057+0000 7ff6017f2740 1 -- 172.21.15.39:0/2198592466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe7b1b0 msgr2=0x55aacc21ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.057+0000 7ff6017f2740 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe7b1b0 0x55aacc21ca60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec004af0 tx=0x7ff5ec005bd0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.057+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5dc0b28a0 0x7ff5dc0b4db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40095f0 tx=0x7ff5e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.057+0000 7ff6017f2740 1 -- 172.21.15.39:0/2198592466 shutdown_connections 2024-05-04T03:18:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.057+0000 7ff6017f2740 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe5a4d0 0x55aacc223ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.057+0000 7ff6017f2740 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc21cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.057+0000 7ff6017f2740 1 --2- 172.21.15.39:0/2198592466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe7b1b0 0x55aacc21ca60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.057+0000 7ff6017f2740 1 -- 172.21.15.39:0/2198592466 >> 172.21.15.39:0/2198592466 conn(0x55aacbe6f300 msgr2=0x55aacc2183a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.058+0000 7ff6017f2740 1 -- 172.21.15.39:0/2198592466 shutdown_connections 2024-05-04T03:18:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.058+0000 7ff6017f2740 1 -- 172.21.15.39:0/2198592466 wait complete. 2024-05-04T03:18:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.058+0000 7ff6017f2740 1 -- start start 2024-05-04T03:18:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff6017f2740 1 Processor -- start 2024-05-04T03:18:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc226cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe7b1b0 0x55aacc22dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff5f3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff6017f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55aacc2248d0 con 0x55aacbe5a4d0 2024-05-04T03:18:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff6017f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55aacc13e370 con 0x55aacbe7b1b0 2024-05-04T03:18:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff6017f2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aacc224a50 con 0x55aacbe522f0 2024-05-04T03:18:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f8e05640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe7b1b0 0x55aacc22dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f8e05640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe7b1b0 0x55aacc22dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37034/0 (socket says 172.21.15.39:37034) 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f8e05640 1 -- 172.21.15.39:0/3441686271 learned_addr learned my addr 172.21.15.39:0/3441686271 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.059+0000 7ff6017f2740 10 client.0 _ll_register_callbacks cb 0x55aacbe92a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc226cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f8e05640 1 -- 172.21.15.39:0/3441686271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 msgr2=0x55aacc226790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f8e05640 1 -- 172.21.15.39:0/3441686271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 msgr2=0x55aacc226cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc226cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.060+0000 7ff5f8e05640 1 -- 172.21.15.39:0/3441686271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacbf5f620 con 0x55aacbe7b1b0 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.061+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc226cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.061+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.061+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe7b1b0 0x55aacc22dbc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009bf0 tx=0x7ff5ec0314d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e99b70d511af5e54 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.061+0000 7ff5d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.061+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3441686271 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec052060 con 0x55aacbe7b1b0 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.061+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3441686271 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff5ec0046f0 con 0x55aacbe7b1b0 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.061+0000 7ff6017f2740 1 -- 172.21.15.39:0/3441686271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5ec036070 con 0x55aacbe7b1b0 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.062+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3441686271 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff5ec0316c0 con 0x55aacbe7b1b0 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.062+0000 7ff6017f2740 10 client.25739 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.062+0000 7ff6017f2740 1 -- 172.21.15.39:0/3441686271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aacc2248d0 con 0x55aacbe7b1b0 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.062+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3441686271 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5ec036070 con 0x55aacbe7b1b0 2024-05-04T03:18:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.062+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3441686271 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff5ec0309f0 con 0x55aacbe7b1b0 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.062+0000 7ff6017f2740 10 client.25739 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.063+0000 7ff6017f2740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.063+0000 7ff6017f2740 20 client.25739 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.063+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55aacbe39d80 0x55aacc233ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.063+0000 7ff6017f2740 4 client.25739 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.063+0000 7ff6017f2740 1 -- 172.21.15.39:0/3441686271 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aacc18f1e0 con 0x55aacbe39d80 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.063+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55aacbe39d80 0x55aacc233ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.063+0000 7ff5d37fe640 10 client.25739 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.063+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55aacbe39d80 0x55aacc233ce0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e86bdd5a6398c1 server_cookie=92cb4efe7833732f in_seq=0 out_seq=0 2024-05-04T03:18:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3441686271 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aacc18f1e0 con 0x55aacbe39d80 2024-05-04T03:18:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff5d37fe640 10 client.25739 handle_command_reply: tid=0 2024-05-04T03:18:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 client.25739 shutdown 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 -- 172.21.15.39:0/3441686271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe7b1b0 msgr2=0x55aacc22dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe7b1b0 0x55aacc22dbc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009bf0 tx=0x7ff5ec0314d0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 -- 172.21.15.39:0/3441686271 shutdown_connections 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55aacbe39d80 0x55aacc233ce0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe7b1b0 0x55aacc22dbc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc226cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff5d37fe640 0 client.25739 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3441686271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 -- 172.21.15.39:0/3441686271 >> 172.21.15.39:0/3441686271 conn(0x55aacbe6f300 msgr2=0x55aacc2173a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 -- 172.21.15.39:0/3441686271 shutdown_connections 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.065+0000 7ff6017f2740 1 -- 172.21.15.39:0/3441686271 wait complete. 2024-05-04T03:18:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.066+0000 7ff6017f2740 20 client.25739 trim_cache size 0 max 16384 2024-05-04T03:18:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.103+0000 7ff6017f2740 1 Processor -- start 2024-05-04T03:18:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.103+0000 7ff6017f2740 1 -- start start 2024-05-04T03:18:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.103+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe39d80 0x55aacc226760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.103+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc217d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe39d80 0x55aacc226760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe39d80 0x55aacc226760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37036/0 (socket says 172.21.15.39:37036) 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f3fff640 1 -- 172.21.15.39:0/3587921520 learned_addr learned my addr 172.21.15.39:0/3587921520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff6017f2740 1 -- 172.21.15.39:0/3587921520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41b17c0 con 0x55aacbe5a4d0 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff6017f2740 1 -- 172.21.15.39:0/3587921520 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41b3300 con 0x55aacbe39d80 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff6017f2740 1 -- 172.21.15.39:0/3587921520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41b1e90 con 0x55aacbe522f0 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc217d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f37fe640 1 -- 172.21.15.39:0/3587921520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe39d80 msgr2=0x55aacc226760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe39d80 0x55aacc226760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f37fe640 1 -- 172.21.15.39:0/3587921520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 msgr2=0x55aacc217d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc217d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.104+0000 7ff5f37fe640 1 -- 172.21.15.39:0/3587921520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacc224000 con 0x55aacbe522f0 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.105+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc217d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.105+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80dfd70 tx=0x7ff5e80a5040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e283ba94e3de00 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.105+0000 7ff5d9ffb640 1 -- 172.21.15.39:0/3587921520 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80a9350 con 0x55aacbe522f0 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.105+0000 7ff5d9ffb640 1 -- 172.21.15.39:0/3587921520 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5e80a9f40 con 0x55aacbe522f0 2024-05-04T03:18:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.105+0000 7ff5d9ffb640 1 -- 172.21.15.39:0/3587921520 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80cc0b0 con 0x55aacbe522f0 2024-05-04T03:18:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.106+0000 7ff6017f2740 1 -- 172.21.15.39:0/3587921520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 msgr2=0x55aacc226ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.106+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80dfd70 tx=0x7ff5e80a5040 comp rx=0 tx=0).stop 2024-05-04T03:18:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.106+0000 7ff6017f2740 1 -- 172.21.15.39:0/3587921520 shutdown_connections 2024-05-04T03:18:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.106+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc217d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.106+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe522f0 0x55aacc226ca0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.106+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3587921520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe39d80 0x55aacc226760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.106+0000 7ff6017f2740 1 -- 172.21.15.39:0/3587921520 >> 172.21.15.39:0/3587921520 conn(0x55aacbe6f300 msgr2=0x55aacc220c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.107+0000 7ff6017f2740 1 -- 172.21.15.39:0/3587921520 shutdown_connections 2024-05-04T03:18:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.107+0000 7ff6017f2740 1 -- 172.21.15.39:0/3587921520 wait complete. 2024-05-04T03:18:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.107+0000 7ff6017f2740 1 -- start start 2024-05-04T03:18:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.108+0000 7ff6017f2740 1 Processor -- start 2024-05-04T03:18:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe39d80 0x55aacc22a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc22ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff6017f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc22bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff5f3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe39d80 0x55aacc22a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff5f3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe39d80 0x55aacc22a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:54246/0 (socket says 172.21.15.39:54246) 2024-05-04T03:18:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff5f37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc22ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff5f37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc22ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37038/0 (socket says 172.21.15.39:37038) 2024-05-04T03:18:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff5f8e05640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc22bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff5f8e05640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc22bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57188/0 (socket says 172.21.15.39:57188) 2024-05-04T03:18:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41b2c30 con 0x55aacbe5a4d0 2024-05-04T03:18:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff5f3fff640 1 -- 172.21.15.39:0/3476524150 learned_addr learned my addr 172.21.15.39:0/3476524150 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41b2560 con 0x55aacbe522f0 2024-05-04T03:18:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.109+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41af7a0 con 0x55aacbe39d80 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff5f3fff640 1 -- 172.21.15.39:0/3476524150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 msgr2=0x55aacc22ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc22ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff5f3fff640 1 -- 172.21.15.39:0/3476524150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 msgr2=0x55aacc22bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff6017f2740 10 client.0 _ll_register_callbacks cb 0x7ff5e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc22bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff5f3fff640 1 -- 172.21.15.39:0/3476524150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacc22c2d0 con 0x55aacbe39d80 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff5f8e05640 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc22bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e80a9f40 con 0x55aacbe39d80 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff5f3fff640 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe39d80 0x55aacc22a8a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4011cf0 tx=0x7ff5e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60049e099e3635fb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.110+0000 7ff5d37fe640 10 client.25753 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.111+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3476524150 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e401eb90 con 0x55aacbe39d80 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.111+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3476524150 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff5e401ed30 con 0x55aacbe39d80 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.111+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3476524150 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff5e402a380 con 0x55aacbe39d80 2024-05-04T03:18:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.111+0000 7ff6017f2740 10 client.25753 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.111+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55aacc22d4d0 con 0x55aacbe39d80 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.112+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3476524150 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff5e402a520 con 0x55aacbe39d80 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.112+0000 7ff6017f2740 10 client.25753 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.112+0000 7ff6017f2740 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.112+0000 7ff6017f2740 20 client.25753 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.112+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55aacbe7b1b0 0x55aacc29e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.112+0000 7ff6017f2740 4 client.25753 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.112+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aacc189050 con 0x55aacbe7b1b0 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.112+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55aacbe7b1b0 0x55aacc29e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.113+0000 7ff5f37fe640 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55aacbe7b1b0 0x55aacc29e6b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbfc151fd5b04944 server_cookie=30c470d81e87dbc2 in_seq=0 out_seq=0 2024-05-04T03:18:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.113+0000 7ff5d37fe640 10 client.25753 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.121+0000 7ff5d37fe640 1 -- 172.21.15.39:0/3476524150 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aacc189050 con 0x55aacbe7b1b0 2024-05-04T03:18:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.121+0000 7ff5d37fe640 10 client.25753 handle_command_reply: tid=0 2024-05-04T03:18:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.121+0000 7ff6017f2740 1 client.25753 shutdown 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.121+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe39d80 msgr2=0x55aacc22a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.121+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe39d80 0x55aacc22a8a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4011cf0 tx=0x7ff5e40046c0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.121+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 shutdown_connections 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55aacbe7b1b0 0x55aacc29e6b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55aacbe5a4d0 0x55aacc22bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff5d37fe640 0 client.25753 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aacbe522f0 0x55aacc22ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff6017f2740 1 --2- 172.21.15.39:0/3476524150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aacbe39d80 0x55aacc22a8a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 >> 172.21.15.39:0/3476524150 conn(0x55aacbe6f300 msgr2=0x55aacc217d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 shutdown_connections 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff6017f2740 1 -- 172.21.15.39:0/3476524150 wait complete. 2024-05-04T03:18:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.122+0000 7ff6017f2740 20 client.25753 trim_cache size 0 max 16384 2024-05-04T03:18:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:18:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:18:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:18:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.123+0000 7ff5fa868640 1 -- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5dc0b28a0 msgr2=0x7ff5dc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.123+0000 7ff5fa868640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5dc0b28a0 0x7ff5dc0b4db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40095f0 tx=0x7ff5e4005f90 comp rx=0 tx=0).stop 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.123+0000 7ff5fa868640 1 -- 172.21.15.39:0/2776290654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5f4103dc0 msgr2=0x7ff5f41a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.123+0000 7ff5fa868640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5f4103dc0 0x7ff5f41a0c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800e9f0 tx=0x7ff5e800eeb0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.124+0000 7ff5fa868640 1 -- 172.21.15.39:0/2776290654 shutdown_connections 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.124+0000 7ff5fa868640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5dc0b28a0 0x7ff5dc0b4db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.124+0000 7ff5fa868640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f4108ac0 0x7ff5f41a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.124+0000 7ff5fa868640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5f4103dc0 0x7ff5f41a0c90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.124+0000 7ff5fa868640 1 --2- 172.21.15.39:0/2776290654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5f4102b80 0x7ff5f41a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.124+0000 7ff5fa868640 1 -- 172.21.15.39:0/2776290654 >> 172.21.15.39:0/2776290654 conn(0x7ff5f40fdd90 msgr2=0x7ff5f4100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.124+0000 7ff5fa868640 1 -- 172.21.15.39:0/2776290654 shutdown_connections 2024-05-04T03:18:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:09.124+0000 7ff5fa868640 1 -- 172.21.15.39:0/2776290654 wait complete. 2024-05-04T03:18:10.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:10 smithi167 ceph-mon[33614]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-05-04T03:18:10.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:10 smithi039 ceph-mon[29715]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-05-04T03:18:10.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:10 smithi066 ceph-mon[33264]: pgmap v607: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 16 MiB/s wr, 1.58k op/s 2024-05-04T03:18:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 260.0 s, 1432.2 tps, lat 69.684 ms stddev 19.584 2024-05-04T03:18:12.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:12 smithi167 ceph-mon[33614]: pgmap v608: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-05-04T03:18:12.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:12 smithi039 ceph-mon[29715]: pgmap v608: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-05-04T03:18:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:12 smithi066 ceph-mon[33264]: pgmap v608: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 15 MiB/s wr, 1.49k op/s 2024-05-04T03:18:14.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:14 smithi167 ceph-mon[33614]: pgmap v609: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-05-04T03:18:14.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:14 smithi039 ceph-mon[29715]: pgmap v609: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-05-04T03:18:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:14 smithi066 ceph-mon[33264]: pgmap v609: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-05-04T03:18:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:16 smithi167 ceph-mon[33614]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-05-04T03:18:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:16 smithi039 ceph-mon[29715]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-05-04T03:18:16.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 265.0 s, 1408.0 tps, lat 71.125 ms stddev 38.310 2024-05-04T03:18:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:16 smithi066 ceph-mon[33264]: pgmap v610: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-05-04T03:18:18.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:18 smithi167 ceph-mon[33614]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:18:18.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:18 smithi039 ceph-mon[29715]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:18:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:18 smithi066 ceph-mon[33264]: pgmap v611: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:18:20.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:20 smithi167 ceph-mon[33614]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-05-04T03:18:20.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:20 smithi039 ceph-mon[29715]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-05-04T03:18:20.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:20 smithi066 ceph-mon[33264]: pgmap v612: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.63k op/s 2024-05-04T03:18:21.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 270.0 s, 1535.6 tps, lat 65.060 ms stddev 16.697 2024-05-04T03:18:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.286+0000 7f3dedffb640 1 -- 172.21.15.39:0/3599654237 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de40307b0 con 0x7f3df0102920 2024-05-04T03:18:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 -- 172.21.15.39:0/3599654237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 msgr2=0x7f3df0102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 --2- 172.21.15.39:0/3599654237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 0x7f3df0102df0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009860 tx=0x7f3de402a340 comp rx=0 tx=0).stop 2024-05-04T03:18:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 -- 172.21.15.39:0/3599654237 shutdown_connections 2024-05-04T03:18:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 --2- 172.21.15.39:0/3599654237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0107a90 0x7f3df0107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 --2- 172.21.15.39:0/3599654237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 0x7f3df0102df0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 --2- 172.21.15.39:0/3599654237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df0101ee0 0x7f3df0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 -- 172.21.15.39:0/3599654237 >> 172.21.15.39:0/3599654237 conn(0x7f3df00fd4d0 msgr2=0x7f3df00ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 -- 172.21.15.39:0/3599654237 shutdown_connections 2024-05-04T03:18:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.287+0000 7f3df5292640 1 -- 172.21.15.39:0/3599654237 wait complete. 2024-05-04T03:18:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.288+0000 7f3df5292640 1 Processor -- start 2024-05-04T03:18:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.288+0000 7f3df5292640 1 -- start start 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3df5292640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0101ee0 0x7f3df01a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3df5292640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 0x7f3df01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3df5292640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df0107a90 0x7f3df01a7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3df5292640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0115c60 con 0x7f3df0107a90 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3df5292640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0115ae0 con 0x7f3df0101ee0 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3df5292640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0115de0 con 0x7f3df0102920 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0101ee0 0x7f3df01a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0101ee0 0x7f3df01a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:36786/0 (socket says 172.21.15.39:36786) 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3dee7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 0x7f3df01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3deeffd640 1 -- 172.21.15.39:0/584728729 learned_addr learned my addr 172.21.15.39:0/584728729 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.289+0000 7f3def7fe640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df0107a90 0x7f3df01a7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dee7fc640 1 -- 172.21.15.39:0/584728729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0101ee0 msgr2=0x7f3df01a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0101ee0 0x7f3df01a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dee7fc640 1 -- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df0107a90 msgr2=0x7f3df01a7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df0107a90 0x7f3df01a7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dee7fc640 1 -- 172.21.15.39:0/584728729 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df01a8380 con 0x7f3df0102920 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 0x7f3df01a0d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3de402a860 tx=0x7f3de402ad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dd7fff640 1 -- 172.21.15.39:0/584728729 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4040030 con 0x7f3df0102920 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dd7fff640 1 -- 172.21.15.39:0/584728729 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3de40305f0 con 0x7f3df0102920 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3dd7fff640 1 -- 172.21.15.39:0/584728729 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4037040 con 0x7f3df0102920 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3df5292640 1 -- 172.21.15.39:0/584728729 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df01a8610 con 0x7f3df0102920 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.290+0000 7f3df5292640 1 -- 172.21.15.39:0/584728729 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a8a80 con 0x7f3df0102920 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.294+0000 7f3dd7fff640 1 -- 172.21.15.39:0/584728729 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3de4043070 con 0x7f3df0102920 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.295+0000 7f3dd7fff640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3dd80b28e0 0x7f3dd80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.295+0000 7f3dfc21d740 1 Processor -- start 2024-05-04T03:18:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.295+0000 7f3dd7fff640 1 -- 172.21.15.39:0/584728729 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3de4101ed0 con 0x7f3df0102920 2024-05-04T03:18:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.295+0000 7f3deeffd640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3dd80b28e0 0x7f3dd80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.295+0000 7f3dfc21d740 1 -- start start 2024-05-04T03:18:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.296+0000 7f3deeffd640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3dd80b28e0 0x7f3dd80b4df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3de00058c0 tx=0x7f3de0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.296+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57d91b0 0x5645f5b7aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b7afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f57b84d0 0x5645f5b81ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645f5adf5d0 con 0x5645f57b84d0 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5645f5a98890 con 0x5645f57d91b0 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5645f5adb590 con 0x5645f57b02f0 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57d91b0 0x5645f5b7aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57d91b0 0x5645f5b7aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:36794/0 (socket says 172.21.15.39:36794) 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3deeffd640 1 -- 172.21.15.39:0/262094887 learned_addr learned my addr 172.21.15.39:0/262094887 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3def7fe640 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f57b84d0 0x5645f5b81ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b7afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3deeffd640 1 -- 172.21.15.39:0/262094887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 msgr2=0x5645f5b7afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3deeffd640 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b7afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3deeffd640 1 -- 172.21.15.39:0/262094887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f57b84d0 msgr2=0x5645f5b81ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3deeffd640 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f57b84d0 0x5645f5b81ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.297+0000 7f3deeffd640 1 -- 172.21.15.39:0/262094887 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f57f1210 con 0x5645f57d91b0 2024-05-04T03:18:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.298+0000 7f3deeffd640 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57d91b0 0x5645f5b7aa60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009fc0 tx=0x7f3de0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8856ce6dd4f1e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.298+0000 7f3dd57fa640 1 -- 172.21.15.39:0/262094887 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0005220 con 0x5645f57d91b0 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.298+0000 7f3dd57fa640 1 -- 172.21.15.39:0/262094887 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3de0004c30 con 0x5645f57d91b0 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.298+0000 7f3dd57fa640 1 -- 172.21.15.39:0/262094887 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00267c0 con 0x5645f57d91b0 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 -- 172.21.15.39:0/262094887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57d91b0 msgr2=0x5645f5b7aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57d91b0 0x5645f5b7aa60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009fc0 tx=0x7f3de0014000 comp rx=0 tx=0).stop 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 -- 172.21.15.39:0/262094887 shutdown_connections 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f57b84d0 0x5645f5b81ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b7afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/262094887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57d91b0 0x5645f5b7aa60 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009fc0 tx=0x7f3de0014000 comp rx=0 tx=0).stop 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 -- 172.21.15.39:0/262094887 >> 172.21.15.39:0/262094887 conn(0x5645f57cd300 msgr2=0x5645f5b763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 -- 172.21.15.39:0/262094887 shutdown_connections 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 -- 172.21.15.39:0/262094887 wait complete. 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.300+0000 7f3dfc21d740 1 -- start start 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.301+0000 7f3dfc21d740 1 Processor -- start 2024-05-04T03:18:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.301+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b84880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.301+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57b84d0 0x5645f5b84dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.301+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f5797d80 0x5645f5b8bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.301+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645f5b82930 con 0x5645f5797d80 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.301+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5645f5a9c370 con 0x5645f57b84d0 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.301+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5645f5b82ab0 con 0x5645f57b02f0 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.301+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b84880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b84880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:38174/0 (socket says 172.21.15.39:38174) 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f5797d80 0x5645f5b8bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3deeffd640 1 -- 172.21.15.39:0/2680206852 learned_addr learned my addr 172.21.15.39:0/2680206852 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57b84d0 0x5645f5b84dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dfc21d740 10 client.0 _ll_register_callbacks cb 0x5645f57f0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dee7fc640 1 -- 172.21.15.39:0/2680206852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 msgr2=0x5645f5b84880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b84880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dee7fc640 1 -- 172.21.15.39:0/2680206852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f5797d80 msgr2=0x5645f5b8bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f5797d80 0x5645f5b8bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dee7fc640 1 -- 172.21.15.39:0/2680206852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f58bd620 con 0x5645f57b84d0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3def7fe640 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f5797d80 0x5645f5b8bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dfc21d740 1 -- 172.21.15.39:0/2680206852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de0004c30 con 0x5645f57b84d0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57b84d0 0x5645f5b84dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3de40ff980 tx=0x7f3de40c8f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19e2884a67636c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.302+0000 7f3dd2ff5640 10 client.25757 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.303+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/2680206852 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de40e3060 con 0x5645f57b84d0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.303+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/2680206852 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3de40ffa90 con 0x5645f57b84d0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.303+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/2680206852 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3de40307a0 con 0x5645f57b84d0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.303+0000 7f3dfc21d740 10 client.25757 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.303+0000 7f3dfc21d740 1 -- 172.21.15.39:0/2680206852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5645f5b8eab0 con 0x5645f57b84d0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.304+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/2680206852 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3de404d060 con 0x5645f57b84d0 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.304+0000 7f3dfc21d740 10 client.25757 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.304+0000 7f3dfc21d740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.304+0000 7f3dfc21d740 20 client.25757 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.304+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645f576d1c0 0x5645f5b91fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.304+0000 7f3dfc21d740 4 client.25757 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.304+0000 7f3dfc21d740 1 -- 172.21.15.39:0/2680206852 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645f5aed1e0 con 0x5645f576d1c0 2024-05-04T03:18:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.304+0000 7f3deeffd640 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645f576d1c0 0x5645f5b91fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.305+0000 7f3deeffd640 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645f576d1c0 0x5645f5b91fb0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4f72ef14c9e4a1 server_cookie=424dc345f5684d32 in_seq=0 out_seq=0 2024-05-04T03:18:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.305+0000 7f3dd2ff5640 10 client.25757 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.306+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/2680206852 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645f5aed1e0 con 0x5645f576d1c0 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.306+0000 7f3dd2ff5640 10 client.25757 handle_command_reply: tid=0 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.306+0000 7f3dfc21d740 1 client.25757 shutdown 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.306+0000 7f3dfc21d740 1 -- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57b84d0 msgr2=0x5645f5b84dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.306+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57b84d0 0x5645f5b84dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3de40ff980 tx=0x7f3de40c8f40 comp rx=0 tx=0).stop 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.306+0000 7f3dfc21d740 1 -- 172.21.15.39:0/2680206852 shutdown_connections 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.306+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645f576d1c0 0x5645f5b91fb0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.307+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f5797d80 0x5645f5b8bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.307+0000 7f3dd2ff5640 0 client.25757 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.307+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f57b84d0 0x5645f5b84dc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.307+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/2680206852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b84880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.307+0000 7f3dfc21d740 1 -- 172.21.15.39:0/2680206852 >> 172.21.15.39:0/2680206852 conn(0x5645f57cd300 msgr2=0x5645f5b79300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.307+0000 7f3dfc21d740 1 -- 172.21.15.39:0/2680206852 shutdown_connections 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.307+0000 7f3dfc21d740 1 -- 172.21.15.39:0/2680206852 wait complete. 2024-05-04T03:18:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.307+0000 7f3dfc21d740 20 client.25757 trim_cache size 0 max 16384 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dfc21d740 1 Processor -- start 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dfc21d740 1 -- start start 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 0x5645f5b79900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b97750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b97c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dee7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b97750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dee7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b97750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:36814/0 (socket says 172.21.15.39:36814) 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dee7fc640 1 -- 172.21.15.39:0/1429275964 learned_addr learned my addr 172.21.15.39:0/1429275964 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.335+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b1950 con 0x5645f576d1c0 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3def7fe640 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3deeffd640 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 0x5645f5b79900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3dfc21d740 1 -- 172.21.15.39:0/1429275964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b3490 con 0x5645f5797d80 2024-05-04T03:18:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3dfc21d740 1 -- 172.21.15.39:0/1429275964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b2020 con 0x5645f57b02f0 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3def7fe640 1 -- 172.21.15.39:0/1429275964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 msgr2=0x5645f5b97750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3def7fe640 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b97750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3def7fe640 1 -- 172.21.15.39:0/1429275964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 msgr2=0x5645f5b79900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3def7fe640 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 0x5645f5b79900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3def7fe640 1 -- 172.21.15.39:0/1429275964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645f5b76d00 con 0x5645f57b02f0 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.336+0000 7f3def7fe640 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b97c90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3de8005fd0 tx=0x7f3de8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb998e88ae56d0c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.337+0000 7f3dd4ff9640 1 -- 172.21.15.39:0/1429275964 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8004d20 con 0x5645f57b02f0 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.337+0000 7f3dd4ff9640 1 -- 172.21.15.39:0/1429275964 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3de8005020 con 0x5645f57b02f0 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.337+0000 7f3dd4ff9640 1 -- 172.21.15.39:0/1429275964 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de801e6b0 con 0x5645f57b02f0 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 -- 172.21.15.39:0/1429275964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 msgr2=0x5645f5b97c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b97c90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3de8005fd0 tx=0x7f3de8009500 comp rx=0 tx=0).stop 2024-05-04T03:18:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 -- 172.21.15.39:0/1429275964 shutdown_connections 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5b97c90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b97750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/1429275964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 0x5645f5b79900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 -- 172.21.15.39:0/1429275964 >> 172.21.15.39:0/1429275964 conn(0x5645f57cd300 msgr2=0x5645f5b791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 -- 172.21.15.39:0/1429275964 shutdown_connections 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.338+0000 7f3dfc21d740 1 -- 172.21.15.39:0/1429275964 wait complete. 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 1 -- start start 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 1 Processor -- start 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 0x5645f5b8aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b8af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5c09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b2dc0 con 0x5645f576d1c0 2024-05-04T03:18:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b26f0 con 0x5645f5797d80 2024-05-04T03:18:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01af930 con 0x5645f57b02f0 2024-05-04T03:18:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.339+0000 7f3dfc21d740 10 client.0 _ll_register_callbacks cb 0x7f3de4005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3dee7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b8af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5c09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 0x5645f5b8aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5c09770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:38196/0 (socket says 172.21.15.39:38196) 2024-05-04T03:18:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3def7fe640 1 -- 172.21.15.39:0/4235220371 learned_addr learned my addr 172.21.15.39:0/4235220371 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3dee7fc640 1 -- 172.21.15.39:0/4235220371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 msgr2=0x5645f5c09770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5c09770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3dee7fc640 1 -- 172.21.15.39:0/4235220371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 msgr2=0x5645f5b8aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 0x5645f5b8aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.340+0000 7f3dee7fc640 1 -- 172.21.15.39:0/4235220371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645f5c09d30 con 0x5645f5797d80 2024-05-04T03:18:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.341+0000 7f3dee7fc640 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b8af70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3de402ab90 tx=0x7f3de40ff850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b52af8b85bb515db server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.341+0000 7f3dd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:18:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.341+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/4235220371 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de40e3040 con 0x5645f5797d80 2024-05-04T03:18:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.341+0000 7f3dfc21d740 1 -- 172.21.15.39:0/4235220371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de8005020 con 0x5645f5797d80 2024-05-04T03:18:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.341+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/4235220371 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3de40d2080 con 0x5645f5797d80 2024-05-04T03:18:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.342+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/4235220371 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3de40c9580 con 0x5645f5797d80 2024-05-04T03:18:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.342+0000 7f3dfc21d740 10 client.25763 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.342+0000 7f3dfc21d740 1 -- 172.21.15.39:0/4235220371 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5645f5c0afc0 con 0x5645f5797d80 2024-05-04T03:18:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.342+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/4235220371 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3de8005020 con 0x5645f5797d80 2024-05-04T03:18:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/4235220371 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3de40c8790 con 0x5645f5797d80 2024-05-04T03:18:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3dfc21d740 10 client.25763 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3dfc21d740 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3dfc21d740 20 client.25763 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645f57b84d0 0x5645f5c0d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3dfc21d740 4 client.25763 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:18:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3dfc21d740 1 -- 172.21.15.39:0/4235220371 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645f5ae7050 con 0x5645f57b84d0 2024-05-04T03:18:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3deeffd640 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645f57b84d0 0x5645f5c0d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.343+0000 7f3dd2ff5640 10 client.25763 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.344+0000 7f3deeffd640 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645f57b84d0 0x5645f5c0d590 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=111089c06565f7 server_cookie=d5488c71c6dadd9 in_seq=0 out_seq=0 2024-05-04T03:18:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dd2ff5640 1 -- 172.21.15.39:0/4235220371 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645f5ae7050 con 0x5645f57b84d0 2024-05-04T03:18:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dd2ff5640 10 client.25763 handle_command_reply: tid=0 2024-05-04T03:18:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 client.25763 shutdown 2024-05-04T03:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 -- 172.21.15.39:0/4235220371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 msgr2=0x5645f5b8af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b8af70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3de402ab90 tx=0x7f3de40ff850 comp rx=0 tx=0).stop 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 -- 172.21.15.39:0/4235220371 shutdown_connections 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5645f57b84d0 0x5645f5c0d590 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645f57b02f0 0x5645f5c09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5645f5797d80 0x5645f5b8af70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dd2ff5640 0 client.25763 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 --2- 172.21.15.39:0/4235220371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645f576d1c0 0x5645f5b8aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 -- 172.21.15.39:0/4235220371 >> 172.21.15.39:0/4235220371 conn(0x5645f57cd300 msgr2=0x5645f5b82e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 -- 172.21.15.39:0/4235220371 shutdown_connections 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.353+0000 7f3dfc21d740 1 -- 172.21.15.39:0/4235220371 wait complete. 2024-05-04T03:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.354+0000 7f3dfc21d740 20 client.25763 trim_cache size 0 max 16384 2024-05-04T03:18:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:18:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:18:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:18:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:18:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.355+0000 7f3df5292640 1 -- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3dd80b28e0 msgr2=0x7f3dd80b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.355+0000 7f3df5292640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3dd80b28e0 0x7f3dd80b4df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3de00058c0 tx=0x7f3de0009260 comp rx=0 tx=0).stop 2024-05-04T03:18:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.355+0000 7f3df5292640 1 -- 172.21.15.39:0/584728729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 msgr2=0x7f3df01a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.355+0000 7f3df5292640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 0x7f3df01a0d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3de402a860 tx=0x7f3de402ad70 comp rx=0 tx=0).stop 2024-05-04T03:18:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.356+0000 7f3df5292640 1 -- 172.21.15.39:0/584728729 shutdown_connections 2024-05-04T03:18:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.356+0000 7f3df5292640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3dd80b28e0 0x7f3dd80b4df0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.356+0000 7f3df5292640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df0107a90 0x7f3df01a7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.356+0000 7f3df5292640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3df0102920 0x7f3df01a0d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.356+0000 7f3df5292640 1 --2- 172.21.15.39:0/584728729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3df0101ee0 0x7f3df01a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.356+0000 7f3df5292640 1 -- 172.21.15.39:0/584728729 >> 172.21.15.39:0/584728729 conn(0x7f3df00fd4d0 msgr2=0x7f3df0103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.356+0000 7f3df5292640 1 -- 172.21.15.39:0/584728729 shutdown_connections 2024-05-04T03:18:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:22.356+0000 7f3df5292640 1 -- 172.21.15.39:0/584728729 wait complete. 2024-05-04T03:18:22.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:22 smithi167 ceph-mon[33614]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:18:22.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:22 smithi039 ceph-mon[29715]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:18:22.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:22 smithi066 ceph-mon[33264]: pgmap v613: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:18:24.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:24 smithi167 ceph-mon[33614]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.59k op/s 2024-05-04T03:18:24.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:24 smithi039 ceph-mon[29715]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.59k op/s 2024-05-04T03:18:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:24 smithi066 ceph-mon[33264]: pgmap v614: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.59k op/s 2024-05-04T03:18:26.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:26 smithi167 ceph-mon[33614]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:18:26.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:26 smithi039 ceph-mon[29715]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:18:26.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 275.0 s, 1411.2 tps, lat 70.913 ms stddev 18.037 2024-05-04T03:18:26.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:26 smithi066 ceph-mon[33264]: pgmap v615: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:18:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:28 smithi167 ceph-mon[33614]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-05-04T03:18:28.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:28 smithi039 ceph-mon[29715]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-05-04T03:18:28.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:28 smithi066 ceph-mon[33264]: pgmap v616: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-05-04T03:18:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:30 smithi167 ceph-mon[33614]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.65k op/s 2024-05-04T03:18:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:30 smithi039 ceph-mon[29715]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.65k op/s 2024-05-04T03:18:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:30 smithi066 ceph-mon[33264]: pgmap v617: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 16 MiB/s wr, 1.65k op/s 2024-05-04T03:18:31.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 280.0 s, 1463.6 tps, lat 68.341 ms stddev 17.616 2024-05-04T03:18:32.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:32 smithi167 ceph-mon[33614]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:18:32.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:32 smithi039 ceph-mon[29715]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:18:32.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:32 smithi066 ceph-mon[33264]: pgmap v618: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:18:34.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:34 smithi167 ceph-mon[33614]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.58k op/s 2024-05-04T03:18:34.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:34 smithi039 ceph-mon[29715]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.58k op/s 2024-05-04T03:18:34.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:34 smithi066 ceph-mon[33264]: pgmap v619: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.58k op/s 2024-05-04T03:18:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/1052594107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48108ac0 msgr2=0x7fcf4810aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/1052594107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48108ac0 0x7fcf4810aee0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009be0 tx=0x7fcf4002a6c0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/1052594107 shutdown_connections 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/1052594107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48108ac0 0x7fcf4810aee0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/1052594107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf48103dc0 0x7fcf481041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/1052594107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48102b80 0x7fcf48103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/1052594107 >> 172.21.15.39:0/1052594107 conn(0x7fcf480fdd90 msgr2=0x7fcf48100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/1052594107 shutdown_connections 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.518+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/1052594107 wait complete. 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4f8f0640 1 Processor -- start 2024-05-04T03:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4f8f0640 1 -- start start 2024-05-04T03:18:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4f8f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48102b80 0x7fcf4806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4f8f0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf48103dc0 0x7fcf4806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4f8f0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48108ac0 0x7fcf4806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4f8f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48070100 con 0x7fcf48102b80 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4f8f0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4806ff80 con 0x7fcf48108ac0 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4f8f0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48070280 con 0x7fcf48103dc0 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4ce8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf48103dc0 0x7fcf4806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4ce8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf48103dc0 0x7fcf4806d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:47234/0 (socket says 172.21.15.39:47234) 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4ce8b640 1 -- 172.21.15.39:0/3970949886 learned_addr learned my addr 172.21.15.39:0/3970949886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48102b80 0x7fcf4806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.519+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48108ac0 0x7fcf4806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.520+0000 7fcf4ce8b640 1 -- 172.21.15.39:0/3970949886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48108ac0 msgr2=0x7fcf4806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.520+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48108ac0 0x7fcf4806d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.520+0000 7fcf4ce8b640 1 -- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48102b80 msgr2=0x7fcf4806eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.520+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48102b80 0x7fcf4806eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.520+0000 7fcf4ce8b640 1 -- 172.21.15.39:0/3970949886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf481958e0 con 0x7fcf48103dc0 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.521+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48102b80 0x7fcf4806eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.521+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf48103dc0 0x7fcf4806d220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800d8d0 tx=0x7fcf3800dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.521+0000 7fcf3e7fc640 1 -- 172.21.15.39:0/3970949886 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38017070 con 0x7fcf48103dc0 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.521+0000 7fcf3e7fc640 1 -- 172.21.15.39:0/3970949886 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf38009c30 con 0x7fcf48103dc0 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.521+0000 7fcf3e7fc640 1 -- 172.21.15.39:0/3970949886 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800f620 con 0x7fcf48103dc0 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.521+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/3970949886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4806ddc0 con 0x7fcf48103dc0 2024-05-04T03:18:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.521+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/3970949886 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf481a04e0 con 0x7fcf48103dc0 2024-05-04T03:18:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.525+0000 7fcf567a0740 1 Processor -- start 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.525+0000 7fcf567a0740 1 -- start start 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf3e7fc640 1 -- 172.21.15.39:0/3970949886 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fcf3800f7c0 con 0x7fcf48103dc0 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9a41b0 0x55bfadd45a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd45fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9834d0 0x55bfadd4cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf567a0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bfadcaa5d0 con 0x55bfad9a41b0 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf567a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bfadc63890 con 0x55bfad97b2f0 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf567a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bfadca6590 con 0x55bfad9834d0 2024-05-04T03:18:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf4de8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9a41b0 0x55bfadd45a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf4de8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9a41b0 0x55bfadd45a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37798/0 (socket says 172.21.15.39:37798) 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf4de8d640 1 -- 172.21.15.39:0/491145526 learned_addr learned my addr 172.21.15.39:0/491145526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9834d0 0x55bfadd4cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.526+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd45fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.527+0000 7fcf4de8d640 1 -- 172.21.15.39:0/491145526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9834d0 msgr2=0x55bfadd4cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.527+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9834d0 0x55bfadd4cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.527+0000 7fcf4de8d640 1 -- 172.21.15.39:0/491145526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 msgr2=0x55bfadd45fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.527+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd45fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.527+0000 7fcf4de8d640 1 -- 172.21.15.39:0/491145526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfad9bc210 con 0x55bfad9a41b0 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.527+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd45fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.527+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9a41b0 0x55bfadd45a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009be0 tx=0x7fcf40030f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0f89a945f2f8b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.528+0000 7fcf337fe640 1 -- 172.21.15.39:0/491145526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40041030 con 0x55bfad9a41b0 2024-05-04T03:18:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.528+0000 7fcf337fe640 1 -- 172.21.15.39:0/491145526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf40037040 con 0x55bfad9a41b0 2024-05-04T03:18:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.528+0000 7fcf337fe640 1 -- 172.21.15.39:0/491145526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40036040 con 0x55bfad9a41b0 2024-05-04T03:18:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.528+0000 7fcf3e7fc640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf340b2970 0x7fcf340b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf340b2970 0x7fcf340b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf3e7fc640 1 -- 172.21.15.39:0/3970949886 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fcf380e1c70 con 0x7fcf48103dc0 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf567a0740 1 -- 172.21.15.39:0/491145526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9a41b0 msgr2=0x55bfadd45a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf567a0740 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9a41b0 0x55bfadd45a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009be0 tx=0x7fcf40030f60 comp rx=0 tx=0).stop 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf567a0740 1 -- 172.21.15.39:0/491145526 shutdown_connections 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf567a0740 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9834d0 0x55bfadd4cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf567a0740 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd45fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf567a0740 1 --2- 172.21.15.39:0/491145526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9a41b0 0x55bfadd45a60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.529+0000 7fcf567a0740 1 -- 172.21.15.39:0/491145526 >> 172.21.15.39:0/491145526 conn(0x55bfad998300 msgr2=0x55bfadd413a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.530+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf340b2970 0x7fcf340b4e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55bfadd46e40 tx=0x7fcf4401a270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.530+0000 7fcf567a0740 1 -- 172.21.15.39:0/491145526 shutdown_connections 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.530+0000 7fcf567a0740 1 -- 172.21.15.39:0/491145526 wait complete. 2024-05-04T03:18:35.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.531+0000 7fcf567a0740 1 -- start start 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.531+0000 7fcf567a0740 1 Processor -- start 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.531+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd4f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.531+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9834d0 0x55bfadd4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.531+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9a41b0 0x55bfadd56bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.531+0000 7fcf567a0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bfadd4d8d0 con 0x55bfad9834d0 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.531+0000 7fcf567a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bfadc67370 con 0x55bfad97b2f0 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.531+0000 7fcf567a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bfadd4da50 con 0x55bfad9a41b0 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf567a0740 10 client.0 _ll_register_callbacks cb 0x55bfad9bba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4d68c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd4f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4de8d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9a41b0 0x55bfadd56bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4d68c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd4f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:59452/0 (socket says 172.21.15.39:59452) 2024-05-04T03:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4d68c640 1 -- 172.21.15.39:0/1279095827 learned_addr learned my addr 172.21.15.39:0/1279095827 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4de8d640 1 -- 172.21.15.39:0/1279095827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 msgr2=0x55bfadd4f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9834d0 0x55bfadd4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd4f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4de8d640 1 -- 172.21.15.39:0/1279095827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9834d0 msgr2=0x55bfadd4fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9834d0 0x55bfadd4fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4de8d640 1 -- 172.21.15.39:0/1279095827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfada88620 con 0x55bfad9a41b0 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd4f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.532+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9834d0 0x55bfadd4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.533+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9a41b0 0x55bfadd56bc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcf40034040 tx=0x7fcf40005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f704a988ca23d21 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.533+0000 7fcf30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.533+0000 7fcf30ff9640 1 -- 172.21.15.39:0/1279095827 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40036050 con 0x55bfad9a41b0 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.533+0000 7fcf567a0740 1 -- 172.21.15.39:0/1279095827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf40037040 con 0x55bfad9a41b0 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.533+0000 7fcf30ff9640 1 -- 172.21.15.39:0/1279095827 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fcf40003f80 con 0x55bfad9a41b0 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf30ff9640 1 -- 172.21.15.39:0/1279095827 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf40041040 con 0x55bfad9a41b0 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf567a0740 10 client.25783 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf567a0740 1 -- 172.21.15.39:0/1279095827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bfadd4d920 con 0x55bfad9a41b0 2024-05-04T03:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf30ff9640 1 -- 172.21.15.39:0/1279095827 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf40037040 con 0x55bfad9a41b0 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf30ff9640 1 -- 172.21.15.39:0/1279095827 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcf4003c040 con 0x55bfad9a41b0 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf567a0740 10 client.25783 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf567a0740 10 client.25783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf567a0740 20 client.25783 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf567a0740 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfad962d80 0x55bfadd5d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf567a0740 4 client.25783 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.534+0000 7fcf567a0740 1 -- 172.21.15.39:0/1279095827 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfadcb81e0 con 0x55bfad962d80 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.535+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfad962d80 0x55bfadd5d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.535+0000 7fcf30ff9640 10 client.25783 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.536+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfad962d80 0x55bfadd5d320 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d203120c47230506 server_cookie=393d028b43f0dcd6 in_seq=0 out_seq=0 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.537+0000 7fcf30ff9640 1 -- 172.21.15.39:0/1279095827 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfadcb81e0 con 0x55bfad962d80 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.537+0000 7fcf30ff9640 10 client.25783 handle_command_reply: tid=0 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.537+0000 7fcf567a0740 1 client.25783 shutdown 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.538+0000 7fcf567a0740 1 -- 172.21.15.39:0/1279095827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9a41b0 msgr2=0x55bfadd56bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.538+0000 7fcf567a0740 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9a41b0 0x55bfadd56bc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcf40034040 tx=0x7fcf40005bd0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.538+0000 7fcf567a0740 1 -- 172.21.15.39:0/1279095827 shutdown_connections 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.538+0000 7fcf567a0740 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfad962d80 0x55bfadd5d320 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.538+0000 7fcf567a0740 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad9a41b0 0x55bfadd56bc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.538+0000 7fcf30ff9640 0 client.25783 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.538+0000 7fcf567a0740 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad9834d0 0x55bfadd4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.538+0000 7fcf567a0740 1 --2- 172.21.15.39:0/1279095827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad97b2f0 0x55bfadd4f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.539+0000 7fcf567a0740 1 -- 172.21.15.39:0/1279095827 >> 172.21.15.39:0/1279095827 conn(0x55bfad998300 msgr2=0x55bfadd403a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.539+0000 7fcf567a0740 1 -- 172.21.15.39:0/1279095827 shutdown_connections 2024-05-04T03:18:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.539+0000 7fcf567a0740 1 -- 172.21.15.39:0/1279095827 wait complete. 2024-05-04T03:18:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.539+0000 7fcf567a0740 20 client.25783 trim_cache size 0 max 16384 2024-05-04T03:18:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.568+0000 7fcf567a0740 1 Processor -- start 2024-05-04T03:18:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.568+0000 7fcf567a0740 1 -- start start 2024-05-04T03:18:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad962d80 0x55bfadd4e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad97b2f0 0x55bfadd4fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf567a0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481a8f90 con 0x55bfad962d80 2024-05-04T03:18:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf567a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481aaad0 con 0x55bfad9834d0 2024-05-04T03:18:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf567a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481a9660 con 0x55bfad97b2f0 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4d68c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad962d80 0x55bfadd4e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4d68c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad962d80 0x55bfadd4e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37808/0 (socket says 172.21.15.39:37808) 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4d68c640 1 -- 172.21.15.39:0/4176910565 learned_addr learned my addr 172.21.15.39:0/4176910565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad97b2f0 0x55bfadd4fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4d68c640 1 -- 172.21.15.39:0/4176910565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad97b2f0 msgr2=0x55bfadd4fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad97b2f0 0x55bfadd4fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4d68c640 1 -- 172.21.15.39:0/4176910565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 msgr2=0x55bfadd56ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd56ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.569+0000 7fcf4d68c640 1 -- 172.21.15.39:0/4176910565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfadd50830 con 0x55bfad962d80 2024-05-04T03:18:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.570+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad962d80 0x55bfadd4e460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcf48102450 tx=0x7fcf4402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cf9fdb38d346fd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.570+0000 7fcf32ffd640 1 -- 172.21.15.39:0/4176910565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44020c90 con 0x55bfad962d80 2024-05-04T03:18:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.570+0000 7fcf32ffd640 1 -- 172.21.15.39:0/4176910565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf44042030 con 0x55bfad962d80 2024-05-04T03:18:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.570+0000 7fcf32ffd640 1 -- 172.21.15.39:0/4176910565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf440232e0 con 0x55bfad962d80 2024-05-04T03:18:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.572+0000 7fcf567a0740 1 -- 172.21.15.39:0/4176910565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad962d80 msgr2=0x55bfadd4e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.572+0000 7fcf567a0740 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad962d80 0x55bfadd4e460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcf48102450 tx=0x7fcf4402c000 comp rx=0 tx=0).stop 2024-05-04T03:18:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.572+0000 7fcf567a0740 1 -- 172.21.15.39:0/4176910565 shutdown_connections 2024-05-04T03:18:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.572+0000 7fcf567a0740 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd56ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.572+0000 7fcf567a0740 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad97b2f0 0x55bfadd4fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.572+0000 7fcf567a0740 1 --2- 172.21.15.39:0/4176910565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad962d80 0x55bfadd4e460 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.572+0000 7fcf567a0740 1 -- 172.21.15.39:0/4176910565 >> 172.21.15.39:0/4176910565 conn(0x55bfad998300 msgr2=0x55bfadd4f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.572+0000 7fcf567a0740 1 -- 172.21.15.39:0/4176910565 shutdown_connections 2024-05-04T03:18:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.573+0000 7fcf567a0740 1 -- 172.21.15.39:0/4176910565 wait complete. 2024-05-04T03:18:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.573+0000 7fcf567a0740 1 -- start start 2024-05-04T03:18:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.574+0000 7fcf567a0740 1 Processor -- start 2024-05-04T03:18:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.575+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad962d80 0x55bfadd57280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.575+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad97b2f0 0x55bfadd53e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.575+0000 7fcf567a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.575+0000 7fcf567a0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481aa400 con 0x55bfad97b2f0 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.575+0000 7fcf567a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481a9d30 con 0x55bfad9834d0 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.575+0000 7fcf567a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481a6f70 con 0x55bfad962d80 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:59470/0 (socket says 172.21.15.39:59470) 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 -- 172.21.15.39:0/2943373263 learned_addr learned my addr 172.21.15.39:0/2943373263 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf567a0740 10 client.0 _ll_register_callbacks cb 0x7fcf380e5620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad962d80 0x55bfadd57280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad97b2f0 0x55bfadd53e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 -- 172.21.15.39:0/2943373263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad962d80 msgr2=0x55bfadd57280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad962d80 0x55bfadd57280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 -- 172.21.15.39:0/2943373263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad97b2f0 msgr2=0x55bfadd53e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad97b2f0 0x55bfadd53e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 -- 172.21.15.39:0/2943373263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfaddc36c0 con 0x55bfad9834d0 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4ce8b640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad97b2f0 0x55bfadd53e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad962d80 0x55bfadd57280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf4de8d640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd543a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf40002230 tx=0x7fcf40009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e256194b7bd8296 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.576+0000 7fcf30ff9640 1 -- 172.21.15.39:0/2943373263 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40036070 con 0x55bfad9834d0 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.577+0000 7fcf30ff9640 1 -- 172.21.15.39:0/2943373263 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fcf40031440 con 0x55bfad9834d0 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.577+0000 7fcf567a0740 1 -- 172.21.15.39:0/2943373263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf44042030 con 0x55bfad9834d0 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.577+0000 7fcf30ff9640 1 -- 172.21.15.39:0/2943373263 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf40031960 con 0x55bfad9834d0 2024-05-04T03:18:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.577+0000 7fcf567a0740 10 client.25781 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.577+0000 7fcf567a0740 1 -- 172.21.15.39:0/2943373263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bfaddc48c0 con 0x55bfad9834d0 2024-05-04T03:18:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.577+0000 7fcf30ff9640 1 -- 172.21.15.39:0/2943373263 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf44042030 con 0x55bfad9834d0 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.578+0000 7fcf30ff9640 1 -- 172.21.15.39:0/2943373263 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcf4003c040 con 0x55bfad9834d0 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.578+0000 7fcf567a0740 10 client.25781 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.578+0000 7fcf567a0740 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.578+0000 7fcf567a0740 20 client.25781 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.578+0000 7fcf567a0740 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfad9a41b0 0x55bfaddc7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.578+0000 7fcf567a0740 4 client.25781 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.578+0000 7fcf567a0740 1 -- 172.21.15.39:0/2943373263 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfadcb2050 con 0x55bfad9a41b0 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.578+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfad9a41b0 0x55bfaddc7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.579+0000 7fcf30ff9640 10 client.25781 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.579+0000 7fcf4d68c640 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfad9a41b0 0x55bfaddc7270 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899b924a62d6c1bb server_cookie=b01cf381b8fc69c5 in_seq=0 out_seq=0 2024-05-04T03:18:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.586+0000 7fcf30ff9640 1 -- 172.21.15.39:0/2943373263 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfadcb2050 con 0x55bfad9a41b0 2024-05-04T03:18:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf30ff9640 10 client.25781 handle_command_reply: tid=0 2024-05-04T03:18:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 client.25781 shutdown 2024-05-04T03:18:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 -- 172.21.15.39:0/2943373263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 msgr2=0x55bfadd543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd543a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf40002230 tx=0x7fcf40009ba0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 -- 172.21.15.39:0/2943373263 shutdown_connections 2024-05-04T03:18:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfad9a41b0 0x55bfaddc7270 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfad9834d0 0x55bfadd543a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfad97b2f0 0x55bfadd53e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 --2- 172.21.15.39:0/2943373263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfad962d80 0x55bfadd57280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 -- 172.21.15.39:0/2943373263 >> 172.21.15.39:0/2943373263 conn(0x55bfad998300 msgr2=0x55bfadd40d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf30ff9640 0 client.25781 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 -- 172.21.15.39:0/2943373263 shutdown_connections 2024-05-04T03:18:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 1 -- 172.21.15.39:0/2943373263 wait complete. 2024-05-04T03:18:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.587+0000 7fcf567a0740 20 client.25781 trim_cache size 0 max 16384 2024-05-04T03:18:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:18:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:18:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:18:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.588+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf340b2970 msgr2=0x7fcf340b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.588+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf340b2970 0x7fcf340b4e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55bfadd46e40 tx=0x7fcf4401a270 comp rx=0 tx=0).stop 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.588+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/3970949886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf48103dc0 msgr2=0x7fcf4806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.588+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf48103dc0 0x7fcf4806d220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800d8d0 tx=0x7fcf3800dd90 comp rx=0 tx=0).stop 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.588+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/3970949886 shutdown_connections 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.588+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fcf340b2970 0x7fcf340b4e80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.588+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf48108ac0 0x7fcf4806d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.589+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf48103dc0 0x7fcf4806d220 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.589+0000 7fcf4f8f0640 1 --2- 172.21.15.39:0/3970949886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf48102b80 0x7fcf4806eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.589+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/3970949886 >> 172.21.15.39:0/3970949886 conn(0x7fcf480fdd90 msgr2=0x7fcf48104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.589+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/3970949886 shutdown_connections 2024-05-04T03:18:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:35.589+0000 7fcf4f8f0640 1 -- 172.21.15.39:0/3970949886 wait complete. 2024-05-04T03:18:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:36 smithi167 ceph-mon[33614]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:18:36.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:36 smithi039 ceph-mon[29715]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:18:36.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 285.0 s, 1430.2 tps, lat 69.935 ms stddev 20.198 2024-05-04T03:18:36.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:36 smithi066 ceph-mon[33264]: pgmap v620: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:18:38.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:38 smithi167 ceph-mon[33614]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-04T03:18:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:38 smithi039 ceph-mon[29715]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-04T03:18:38.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:38 smithi066 ceph-mon[33264]: pgmap v621: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-04T03:18:39.195 INFO:tasks.quiescer.fs.[cephfs]:Total duration (46.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-04T03:18:39.196 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.05 2024-05-04T03:18:39.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:18:39.608 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:40 smithi167 ceph-mon[33614]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.50k op/s 2024-05-04T03:18:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:40 smithi039 ceph-mon[29715]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.50k op/s 2024-05-04T03:18:40.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:40 smithi066 ceph-mon[33264]: pgmap v622: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.50k op/s 2024-05-04T03:18:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 290.0 s, 1434.8 tps, lat 69.648 ms stddev 21.148 2024-05-04T03:18:42.645 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:42 smithi039 ceph-mon[29715]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-05-04T03:18:42.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:42 smithi167 ceph-mon[33614]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-05-04T03:18:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:42 smithi066 ceph-mon[33264]: pgmap v623: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-05-04T03:18:42.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.833+0000 7fc80c64b700 1 -- 172.21.15.39:0/3565827114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8041086a0 msgr2=0x7fc804108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:42.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.833+0000 7fc80c64b700 1 --2- 172.21.15.39:0/3565827114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8041086a0 0x7fc804108b70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009f90 tx=0x7fc7fc00b330 comp rx=0 tx=0).stop 2024-05-04T03:18:42.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.833+0000 7fc80c64b700 1 -- 172.21.15.39:0/3565827114 shutdown_connections 2024-05-04T03:18:42.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.833+0000 7fc80c64b700 1 --2- 172.21.15.39:0/3565827114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc804103250 0x7fc804103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.833+0000 7fc80c64b700 1 --2- 172.21.15.39:0/3565827114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8041086a0 0x7fc804108b70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.833+0000 7fc80c64b700 1 --2- 172.21.15.39:0/3565827114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc804103ba0 0x7fc8041080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.833+0000 7fc80c64b700 1 -- 172.21.15.39:0/3565827114 >> 172.21.15.39:0/3565827114 conn(0x7fc8040fedc0 msgr2=0x7fc8041011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.834+0000 7fc80c64b700 1 -- 172.21.15.39:0/3565827114 shutdown_connections 2024-05-04T03:18:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.834+0000 7fc80c64b700 1 -- 172.21.15.39:0/3565827114 wait complete. 2024-05-04T03:18:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.836+0000 7fc80c64b700 1 Processor -- start 2024-05-04T03:18:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.836+0000 7fc80c64b700 1 -- start start 2024-05-04T03:18:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.836+0000 7fc80c64b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc804103250 0x7fc8041a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.836+0000 7fc80c64b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc804103ba0 0x7fc8041a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.836+0000 7fc80c64b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8041086a0 0x7fc8041a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.836+0000 7fc80c64b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc80411a920 con 0x7fc804103250 2024-05-04T03:18:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.836+0000 7fc80c64b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc80411a7a0 con 0x7fc8041086a0 2024-05-04T03:18:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.836+0000 7fc80c64b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc80411aaa0 con 0x7fc804103ba0 2024-05-04T03:18:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.837+0000 7fc80abe8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8041086a0 0x7fc8041a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.837+0000 7fc80abe8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8041086a0 0x7fc8041a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43564/0 (socket says 172.21.15.39:43564) 2024-05-04T03:18:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.837+0000 7fc80abe8700 1 -- 172.21.15.39:0/2183747115 learned_addr learned my addr 172.21.15.39:0/2183747115 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.837+0000 7fc80abe8700 1 -- 172.21.15.39:0/2183747115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc804103ba0 msgr2=0x7fc8041a1910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:18:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.837+0000 7fc80abe8700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc804103ba0 0x7fc8041a1910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.837+0000 7fc80abe8700 1 -- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc804103250 msgr2=0x7fc8041a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.837+0000 7fc80abe8700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc804103250 0x7fc8041a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.837+0000 7fc80abe8700 1 -- 172.21.15.39:0/2183747115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8041a6240 con 0x7fc8041086a0 2024-05-04T03:18:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.838+0000 7fc80abe8700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8041086a0 0x7fc8041a5ba0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00b770 tx=0x7fc7fc004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:42.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.838+0000 7fc7fb7fe700 1 -- 172.21.15.39:0/2183747115 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc021070 con 0x7fc8041086a0 2024-05-04T03:18:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.838+0000 7fc7fb7fe700 1 -- 172.21.15.39:0/2183747115 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7fc007c40 con 0x7fc8041086a0 2024-05-04T03:18:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.838+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8041a64d0 con 0x7fc8041086a0 2024-05-04T03:18:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.838+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8041ade50 con 0x7fc8041086a0 2024-05-04T03:18:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.838+0000 7fc7fb7fe700 1 -- 172.21.15.39:0/2183747115 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc005330 con 0x7fc8041086a0 2024-05-04T03:18:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.841+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e8005330 con 0x7fc8041086a0 2024-05-04T03:18:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.843+0000 7fc7fb7fe700 1 -- 172.21.15.39:0/2183747115 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc7fc024070 con 0x7fc8041086a0 2024-05-04T03:18:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.844+0000 7fc7fb7fe700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7f00b2a50 0x7fc7f00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.844+0000 7fc7fb7fe700 1 -- 172.21.15.39:0/2183747115 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc7fc0e3240 con 0x7fc8041086a0 2024-05-04T03:18:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.844+0000 7fc80a3e7700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7f00b2a50 0x7fc7f00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.846+0000 7fc80a3e7700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7f00b2a50 0x7fc7f00b4f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc800005b40 tx=0x7fc800005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:42.848+0000 7fc7fb7fe700 1 -- 172.21.15.39:0/2183747115 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7fc0aafa0 con 0x7fc8041086a0 2024-05-04T03:18:43.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.327+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e8005620 con 0x7fc8041086a0 2024-05-04T03:18:43.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.328+0000 7fc7fb7fe700 1 -- 172.21.15.39:0/2183747115 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7fc7fc0aa8a0 con 0x7fc8041086a0 2024-05-04T03:18:43.330 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:18:43.330 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:18:43.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.332+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7f00b2a50 msgr2=0x7fc7f00b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.332+0000 7fc80c64b700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7f00b2a50 0x7fc7f00b4f60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc800005b40 tx=0x7fc800005a90 comp rx=0 tx=0).stop 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.332+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8041086a0 msgr2=0x7fc8041a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.332+0000 7fc80c64b700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8041086a0 0x7fc8041a5ba0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00b770 tx=0x7fc7fc004e50 comp rx=0 tx=0).stop 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.333+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 shutdown_connections 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.333+0000 7fc80c64b700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7f00b2a50 0x7fc7f00b4f60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.333+0000 7fc80c64b700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc804103250 0x7fc8041a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.333+0000 7fc80c64b700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc8041086a0 0x7fc8041a5ba0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.333+0000 7fc80c64b700 1 --2- 172.21.15.39:0/2183747115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc804103ba0 0x7fc8041a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.333+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 >> 172.21.15.39:0/2183747115 conn(0x7fc8040fedc0 msgr2=0x7fc8041060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.333+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 shutdown_connections 2024-05-04T03:18:43.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:43.333+0000 7fc80c64b700 1 -- 172.21.15.39:0/2183747115 wait complete. 2024-05-04T03:18:43.338 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:18:43.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:43 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2183747115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:43.684 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:43 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2183747115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:43.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:43 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2183747115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:44.043 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 83.05 --await-for 42.0' 2024-05-04T03:18:44.043 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 83.05 --await-for 42.0 2024-05-04T03:18:44.440 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:44 smithi167 ceph-mon[33614]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-05-04T03:18:44.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:44 smithi039 ceph-mon[29715]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-05-04T03:18:44.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:44 smithi066 ceph-mon[33264]: pgmap v624: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-05-04T03:18:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 -- 172.21.15.39:0/3511355116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4a8108740 msgr2=0x7fa4a81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/3511355116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4a8108740 0x7fa4a81100a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b0a0 tx=0x7fa4a000b3b0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 -- 172.21.15.39:0/3511355116 shutdown_connections 2024-05-04T03:18:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/3511355116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4a8108740 0x7fa4a81100a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/3511355116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a81034c0 0x7fa4a81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/3511355116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4a8107df0 0x7fa4a8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 -- 172.21.15.39:0/3511355116 >> 172.21.15.39:0/3511355116 conn(0x7fa4a80fedc0 msgr2=0x7fa4a81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 -- 172.21.15.39:0/3511355116 shutdown_connections 2024-05-04T03:18:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.357+0000 7fa4acbc0700 1 -- 172.21.15.39:0/3511355116 wait complete. 2024-05-04T03:18:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.358+0000 7fa4acbc0700 1 Processor -- start 2024-05-04T03:18:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.358+0000 7fa4acbc0700 1 -- start start 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4acbc0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4a81034c0 0x7fa4a81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4acbc0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a8107df0 0x7fa4a81a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4acbc0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4a8108740 0x7fa4a81a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4acbc0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a811a970 con 0x7fa4a81034c0 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4acbc0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a811a7f0 con 0x7fa4a8108740 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4acbc0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a811aaf0 con 0x7fa4a8107df0 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a8107df0 0x7fa4a81a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4a8108740 0x7fa4a81a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a8107df0 0x7fa4a81a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:43204/0 (socket says 172.21.15.39:43204) 2024-05-04T03:18:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/1024609584 learned_addr learned my addr 172.21.15.39:0/1024609584 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a659c700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4a81034c0 0x7fa4a81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/1024609584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4a8108740 msgr2=0x7fa4a81a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4a8108740 0x7fa4a81a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4a81034c0 msgr2=0x7fa4a81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4a81034c0 0x7fa4a81a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/1024609584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a81a6170 con 0x7fa4a8107df0 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.359+0000 7fa4a659c700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4a81034c0 0x7fa4a81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.360+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a8107df0 0x7fa4a81a1840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009a70 tx=0x7fa49c009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.360+0000 7fa4977fe700 1 -- 172.21.15.39:0/1024609584 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c01b020 con 0x7fa4a8107df0 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.360+0000 7fa4977fe700 1 -- 172.21.15.39:0/1024609584 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa49c004830 con 0x7fa4a8107df0 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.360+0000 7fa4977fe700 1 -- 172.21.15.39:0/1024609584 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c005310 con 0x7fa4a8107df0 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.360+0000 7fa4acbc0700 1 -- 172.21.15.39:0/1024609584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a81a6400 con 0x7fa4a8107df0 2024-05-04T03:18:46.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.360+0000 7fa4acbc0700 1 -- 172.21.15.39:0/1024609584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a81a6890 con 0x7fa4a8107df0 2024-05-04T03:18:46.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.364+0000 7fa4977fe700 1 -- 172.21.15.39:0/1024609584 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa49c0054b0 con 0x7fa4a8107df0 2024-05-04T03:18:46.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.366+0000 7fa4977fe700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa4900b2900 0x7fa4900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.367+0000 7fa4977fe700 1 -- 172.21.15.39:0/1024609584 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa49c0e1f70 con 0x7fa4a8107df0 2024-05-04T03:18:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.367+0000 7fa4bb5d4b80 1 Processor -- start 2024-05-04T03:18:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.367+0000 7fa4a659c700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa4900b2900 0x7fa4900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.367+0000 7fa4bb5d4b80 1 -- start start 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a7b60 0x558c009a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a84b0 0x558c009b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a659c700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa4900b2900 0x7fa4900b4e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa498009be0 tx=0x7fa498009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a84b0 0x558c009b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a6d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a7b60 0x558c009a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a7b60 0x558c009a7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:34208/0 (socket says 172.21.15.39:34208) 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a659c700 1 -- 172.21.15.39:0/647714332 learned_addr learned my addr 172.21.15.39:0/647714332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/647714332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558c008caad0 con 0x558c009b2cd0 2024-05-04T03:18:46.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/647714332 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558c0082ff30 con 0x558c009a7b60 2024-05-04T03:18:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/647714332 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558c008c6a90 con 0x558c009a84b0 2024-05-04T03:18:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a6d9d700 1 -- 172.21.15.39:0/647714332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a84b0 msgr2=0x558c009b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a6d9d700 1 --2- 172.21.15.39:0/647714332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a84b0 0x558c009b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a6d9d700 1 -- 172.21.15.39:0/647714332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a7b60 msgr2=0x558c009a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a6d9d700 1 --2- 172.21.15.39:0/647714332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a7b60 0x558c009a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.368+0000 7fa4a6d9d700 1 -- 172.21.15.39:0/647714332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c00817a60 con 0x558c009b2cd0 2024-05-04T03:18:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.369+0000 7fa4a6d9d700 1 --2- 172.21.15.39:0/647714332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b50f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b070 tx=0x7fa4a000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56f3fd77fd715cd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:46.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.369+0000 7fa494ff9700 1 -- 172.21.15.39:0/647714332 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00092e0 con 0x558c009b2cd0 2024-05-04T03:18:46.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.369+0000 7fa494ff9700 1 -- 172.21.15.39:0/647714332 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a0006000 con 0x558c009b2cd0 2024-05-04T03:18:46.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.369+0000 7fa494ff9700 1 -- 172.21.15.39:0/647714332 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00079e0 con 0x558c009b2cd0 2024-05-04T03:18:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/647714332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 msgr2=0x558c009b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/647714332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b50f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b070 tx=0x7fa4a000b700 comp rx=0 tx=0).stop 2024-05-04T03:18:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/647714332 shutdown_connections 2024-05-04T03:18:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/647714332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b50f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/647714332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a7b60 0x558c009a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/647714332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a84b0 0x558c009b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/647714332 >> 172.21.15.39:0/647714332 conn(0x558c0082afd0 msgr2=0x558c009a3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/647714332 shutdown_connections 2024-05-04T03:18:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.371+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/647714332 wait complete. 2024-05-04T03:18:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.372+0000 7fa4bb5d4b80 1 -- start start 2024-05-04T03:18:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.372+0000 7fa4bb5d4b80 1 Processor -- start 2024-05-04T03:18:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.372+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.372+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.372+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4bb5d4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558c009b5a40 con 0x558c009b2cd0 2024-05-04T03:18:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4bb5d4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558c00884060 con 0x558c009a84b0 2024-05-04T03:18:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4bb5d4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558c009b5bc0 con 0x558c009a7b60 2024-05-04T03:18:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4bb5d4b80 10 client.0 _ll_register_callbacks cb 0x558c00862ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:34210/0 (socket says 172.21.15.39:34210) 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/4268271480 learned_addr learned my addr 172.21.15.39:0/4268271480 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a6d9d700 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a659c700 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a659c700 1 -- 172.21.15.39:0/4268271480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 msgr2=0x558c009b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a659c700 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a659c700 1 -- 172.21.15.39:0/4268271480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 msgr2=0x558c009b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a659c700 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a659c700 1 -- 172.21.15.39:0/4268271480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c008818f0 con 0x558c009a7b60 2024-05-04T03:18:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.373+0000 7fa4a6d9d700 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:46.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.374+0000 7fa4a659c700 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009afd60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x558c009a4410 tx=0x7fa498004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b554e03ad97148 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:46.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.374+0000 7fa48dffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.374+0000 7fa48dffb700 1 -- 172.21.15.39:0/4268271480 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498003b30 con 0x558c009a7b60 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.374+0000 7fa48dffb700 1 -- 172.21.15.39:0/4268271480 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa498003cd0 con 0x558c009a7b60 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.374+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/4268271480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a0006000 con 0x558c009a7b60 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.374+0000 7fa48dffb700 1 -- 172.21.15.39:0/4268271480 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa498012450 con 0x558c009a7b60 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.375+0000 7fa4bb5d4b80 10 client.25804 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.375+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/4268271480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558c009c07a0 con 0x558c009a7b60 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.375+0000 7fa48dffb700 1 -- 172.21.15.39:0/4268271480 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a0006000 con 0x558c009a7b60 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.375+0000 7fa48dffb700 1 -- 172.21.15.39:0/4268271480 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa49800f040 con 0x558c009a7b60 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.376+0000 7fa4bb5d4b80 10 client.25804 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.376+0000 7fa4bb5d4b80 10 client.25804 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.376+0000 7fa4bb5d4b80 20 client.25804 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.376+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558c009c2de0 0x558c009c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.376+0000 7fa4bb5d4b80 4 client.25804 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:46.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.376+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/4268271480 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c008d84b0 con 0x558c009c2de0 2024-05-04T03:18:46.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.376+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558c009c2de0 0x558c009c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.378+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558c009c2de0 0x558c009c5250 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7748395071c0ed server_cookie=a5a90bff6403abd3 in_seq=0 out_seq=0 2024-05-04T03:18:46.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.378+0000 7fa48dffb700 10 client.25804 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:46.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.378+0000 7fa48dffb700 1 -- 172.21.15.39:0/4268271480 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c008d84b0 con 0x558c009c2de0 2024-05-04T03:18:46.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.378+0000 7fa48dffb700 10 client.25804 handle_command_reply: tid=0 2024-05-04T03:18:46.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.378+0000 7fa4bb5d4b80 1 client.25804 shutdown 2024-05-04T03:18:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/4268271480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 msgr2=0x558c009afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009afd60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x558c009a4410 tx=0x7fa498004750 comp rx=0 tx=0).stop 2024-05-04T03:18:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/4268271480 shutdown_connections 2024-05-04T03:18:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558c009c2de0 0x558c009c5250 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009b2cd0 0x558c009b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009afd60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa48dffb700 0 client.25804 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/4268271480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/4268271480 >> 172.21.15.39:0/4268271480 conn(0x558c0082afd0 msgr2=0x558c009a9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/4268271480 shutdown_connections 2024-05-04T03:18:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/4268271480 wait complete. 2024-05-04T03:18:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.379+0000 7fa4bb5d4b80 20 client.25804 trim_cache size 0 max 16384 2024-05-04T03:18:46.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.415+0000 7fa4bb5d4b80 1 Processor -- start 2024-05-04T03:18:46.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4bb5d4b80 1 -- start start 2024-05-04T03:18:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c0082afd0 0x558c009b0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4bb5d4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b5090 con 0x558c0082afd0 2024-05-04T03:18:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4bb5d4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b6bd0 con 0x558c009a84b0 2024-05-04T03:18:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4bb5d4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b5760 con 0x558c009a7b60 2024-05-04T03:18:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4a5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4a5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:43234/0 (socket says 172.21.15.39:43234) 2024-05-04T03:18:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/1141482140 learned_addr learned my addr 172.21.15.39:0/1141482140 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.417+0000 7fa4a6d9d700 1 --2- 172.21.15.39:0/1141482140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.418+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/1141482140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 msgr2=0x558c009a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.418+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/1141482140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.418+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/1141482140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c0082afd0 msgr2=0x558c009b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.418+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/1141482140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c0082afd0 0x558c009b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.418+0000 7fa4a5d9b700 1 -- 172.21.15.39:0/1141482140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c009a3d90 con 0x558c009a7b60 2024-05-04T03:18:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.419+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/1141482140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009b1e40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0a92d0 tx=0x7fa49c0ac040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb67e4d37623c0c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.419+0000 7fa48ffff700 1 -- 172.21.15.39:0/1141482140 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0ad980 con 0x558c009a7b60 2024-05-04T03:18:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.419+0000 7fa48ffff700 1 -- 172.21.15.39:0/1141482140 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa49c0b23a0 con 0x558c009a7b60 2024-05-04T03:18:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.419+0000 7fa48ffff700 1 -- 172.21.15.39:0/1141482140 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0911d0 con 0x558c009a7b60 2024-05-04T03:18:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.420+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1141482140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 msgr2=0x558c009b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.420+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1141482140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009b1e40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0a92d0 tx=0x7fa49c0ac040 comp rx=0 tx=0).stop 2024-05-04T03:18:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.420+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1141482140 shutdown_connections 2024-05-04T03:18:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.421+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1141482140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c0082afd0 0x558c009b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.421+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1141482140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.421+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1141482140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c009a7b60 0x558c009b1e40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.421+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1141482140 >> 172.21.15.39:0/1141482140 conn(0x558c0099cbf0 msgr2=0x558c009a9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.421+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1141482140 shutdown_connections 2024-05-04T03:18:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.421+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1141482140 wait complete. 2024-05-04T03:18:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.422+0000 7fa4bb5d4b80 1 -- start start 2024-05-04T03:18:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.422+0000 7fa4bb5d4b80 1 Processor -- start 2024-05-04T03:18:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.422+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c0082afd0 0x558c009b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.422+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009a7b60 0x558c009b8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c0082afd0 0x558c009b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c0082afd0 0x558c009b7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:43238/0 (socket says 172.21.15.39:43238) 2024-05-04T03:18:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4bb5d4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a659c700 1 -- 172.21.15.39:0/1376416967 learned_addr learned my addr 172.21.15.39:0/1376416967 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b6500 con 0x558c009a7b60 2024-05-04T03:18:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b5e30 con 0x558c009a84b0 2024-05-04T03:18:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a81b3030 con 0x558c0082afd0 2024-05-04T03:18:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a6d9d700 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4bb5d4b80 10 client.0 _ll_register_callbacks cb 0x558c00862ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a659c700 1 -- 172.21.15.39:0/1376416967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 msgr2=0x558c009b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a659c700 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a659c700 1 -- 172.21.15.39:0/1376416967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009a7b60 msgr2=0x558c009b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a659c700 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009a7b60 0x558c009b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.423+0000 7fa4a659c700 1 -- 172.21.15.39:0/1376416967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c00a2d620 con 0x558c0082afd0 2024-05-04T03:18:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.424+0000 7fa4a659c700 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c0082afd0 0x558c009b7e20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa4980206d0 tx=0x7fa498021fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ac82e500dcd24e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.424+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa49c0b23a0 con 0x558c0082afd0 2024-05-04T03:18:46.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.424+0000 7fa48dffb700 10 client.25810 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:18:46.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.424+0000 7fa48dffb700 1 -- 172.21.15.39:0/1376416967 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498005040 con 0x558c0082afd0 2024-05-04T03:18:46.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.424+0000 7fa48dffb700 1 -- 172.21.15.39:0/1376416967 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa4980051e0 con 0x558c0082afd0 2024-05-04T03:18:46.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.434+0000 7fa48dffb700 1 -- 172.21.15.39:0/1376416967 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa498005420 con 0x558c0082afd0 2024-05-04T03:18:46.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa4bb5d4b80 10 client.25810 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:46.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=32}) -- 0x558c00a2e770 con 0x558c0082afd0 2024-05-04T03:18:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa48dffb700 1 -- 172.21.15.39:0/1376416967 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa49803b070 con 0x558c0082afd0 2024-05-04T03:18:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa4bb5d4b80 10 client.25810 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa4bb5d4b80 10 client.25810 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:18:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa4bb5d4b80 20 client.25810 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558c009b2cd0 0x558c00a30e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa4bb5d4b80 4 client.25810 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 83.05, "await_for": 42.0}] 2024-05-04T03:18:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.435+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 83.05, "await_for": 42.0}) -- 0x7fa49c0b23a0 con 0x558c009b2cd0 2024-05-04T03:18:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.436+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558c009b2cd0 0x558c00a30e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.436+0000 7fa4a5d9b700 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558c009b2cd0 0x558c00a30e90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9565616f102c5bdf server_cookie=73aeb2edd61141b4 in_seq=0 out_seq=0 2024-05-04T03:18:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.436+0000 7fa48dffb700 10 client.25810 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:46 smithi167 ceph-mon[33614]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:18:46.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:46 smithi039 ceph-mon[29715]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:18:46.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 295.0 s, 1245.5 tps, lat 75.280 ms stddev 21.411 2024-05-04T03:18:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:46 smithi066 ceph-mon[33264]: pgmap v625: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:18:46.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa48dffb700 1 -- 172.21.15.39:0/1376416967 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7fa49c0b23a0 con 0x558c009b2cd0 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa48dffb700 10 client.25810 handle_command_reply: tid=0 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 client.25810 shutdown 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c0082afd0 msgr2=0x558c009b7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c0082afd0 0x558c009b7e20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa4980206d0 tx=0x7fa498021fb0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 shutdown_connections 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558c009b2cd0 0x558c00a30e90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558c009a84b0 0x558c009b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558c0082afd0 0x558c009b7e20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 --2- 172.21.15.39:0/1376416967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c009a7b60 0x558c009b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa48dffb700 0 client.25810 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.817+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 >> 172.21.15.39:0/1376416967 conn(0x558c0099cbf0 msgr2=0x558c009ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.818+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 shutdown_connections 2024-05-04T03:18:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.818+0000 7fa4bb5d4b80 1 -- 172.21.15.39:0/1376416967 wait complete. 2024-05-04T03:18:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.818+0000 7fa4bb5d4b80 20 client.25810 trim_cache size 0 max 16384 2024-05-04T03:18:46.821 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:18:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:18:46.821 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 18, 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "2ab0057c": { 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 18, 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 83.0, 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:18:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:18:46.823 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:18:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 -- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa4900b2900 msgr2=0x7fa4900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa4900b2900 0x7fa4900b4e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa498009be0 tx=0x7fa498009260 comp rx=0 tx=0).stop 2024-05-04T03:18:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 -- 172.21.15.39:0/1024609584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a8107df0 msgr2=0x7fa4a81a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a8107df0 0x7fa4a81a1840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009a70 tx=0x7fa49c009d80 comp rx=0 tx=0).stop 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 -- 172.21.15.39:0/1024609584 shutdown_connections 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa4900b2900 0x7fa4900b4e10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4a81034c0 0x7fa4a81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4a8108740 0x7fa4a81a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 --2- 172.21.15.39:0/1024609584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a8107df0 0x7fa4a81a1840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 -- 172.21.15.39:0/1024609584 >> 172.21.15.39:0/1024609584 conn(0x7fa4a80fedc0 msgr2=0x7fa4a81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 -- 172.21.15.39:0/1024609584 shutdown_connections 2024-05-04T03:18:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:18:46.819+0000 7fa4acbc0700 1 -- 172.21.15.39:0/1024609584 wait complete. 2024-05-04T03:18:47.507 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2ab0057c', quiesce took 8.3 seconds. Will release after: 23.05 2024-05-04T03:18:48.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:48 smithi039 ceph-mon[29715]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:18:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 -- 172.21.15.39:0/1589547015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 msgr2=0x7f717c1052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 --2- 172.21.15.39:0/1589547015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 0x7f717c1052c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f716c009900 tx=0x7f716c02a410 comp rx=0 tx=0).stop 2024-05-04T03:18:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 -- 172.21.15.39:0/1589547015 shutdown_connections 2024-05-04T03:18:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 --2- 172.21.15.39:0/1589547015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c109bb0 0x7f717c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 --2- 172.21.15.39:0/1589547015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 0x7f717c1052c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 --2- 172.21.15.39:0/1589547015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f717c103c70 0x7f717c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 -- 172.21.15.39:0/1589547015 >> 172.21.15.39:0/1589547015 conn(0x7f717c0fee40 msgr2=0x7f717c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 -- 172.21.15.39:0/1589547015 shutdown_connections 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.707+0000 7f71841f3640 1 -- 172.21.15.39:0/1589547015 wait complete. 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f71841f3640 1 Processor -- start 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f71841f3640 1 -- start start 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f71841f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c103c70 0x7f717c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f71841f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 0x7f717c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f71841f3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f717c109bb0 0x7f717c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f71841f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10f650 con 0x7f717c104eb0 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f71841f3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10f4d0 con 0x7f717c109bb0 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f71841f3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f717c10f7d0 con 0x7f717c103c70 2024-05-04T03:18:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f718178e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 0x7f717c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f718178e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 0x7f717c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46780/0 (socket says 172.21.15.39:46780) 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.708+0000 7f718178e640 1 -- 172.21.15.39:0/2365603077 learned_addr learned my addr 172.21.15.39:0/2365603077 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f7182790640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f717c109bb0 0x7f717c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f7181f8f640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c103c70 0x7f717c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f718178e640 1 -- 172.21.15.39:0/2365603077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c103c70 msgr2=0x7f717c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f718178e640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c103c70 0x7f717c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f718178e640 1 -- 172.21.15.39:0/2365603077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f717c109bb0 msgr2=0x7f717c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f718178e640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f717c109bb0 0x7f717c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f718178e640 1 -- 172.21.15.39:0/2365603077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c1a1da0 con 0x7f717c104eb0 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f7182790640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f717c109bb0 0x7f717c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f718178e640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 0x7f717c19a7a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f716c009230 tx=0x7f716c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f7172ffd640 1 -- 172.21.15.39:0/2365603077 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c03f030 con 0x7f717c104eb0 2024-05-04T03:18:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f7172ffd640 1 -- 172.21.15.39:0/2365603077 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f716c003c90 con 0x7f717c104eb0 2024-05-04T03:18:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f71841f3640 1 -- 172.21.15.39:0/2365603077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c1a2030 con 0x7f717c104eb0 2024-05-04T03:18:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f7172ffd640 1 -- 172.21.15.39:0/2365603077 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c035070 con 0x7f717c104eb0 2024-05-04T03:18:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.709+0000 7f71841f3640 1 -- 172.21.15.39:0/2365603077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1a2520 con 0x7f717c104eb0 2024-05-04T03:18:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f7172ffd640 1 -- 172.21.15.39:0/2365603077 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f716c042030 con 0x7f717c104eb0 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f718b17d740 1 Processor -- start 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f718b17d740 1 -- start start 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f718b17d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a84fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f718b17d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 0x56113a84ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f718b17d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a856ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f718b17d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56113a7b45d0 con 0x56113a48d4d0 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f718b17d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56113a76d890 con 0x56113a4ae1b0 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.712+0000 7f718b17d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56113a7b0590 con 0x56113a4852f0 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a856ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a856ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46786/0 (socket says 172.21.15.39:46786) 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 -- 172.21.15.39:0/512000307 learned_addr learned my addr 172.21.15.39:0/512000307 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f7172ffd640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f71680b27e0 0x7f71680b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f7181f8f640 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 0x56113a84ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f7182790640 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a84fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f7182790640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f71680b27e0 0x7f71680b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f7172ffd640 1 -- 172.21.15.39:0/2365603077 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f716c1028d0 con 0x7f717c104eb0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 -- 172.21.15.39:0/512000307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 msgr2=0x56113a84ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 0x56113a84ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 -- 172.21.15.39:0/512000307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 msgr2=0x56113a84fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a84fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 -- 172.21.15.39:0/512000307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113a4c6210 con 0x56113a48d4d0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f718178e640 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a856ee0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f716c0c7170 tx=0x7f716c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b53061fc2243e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f7182790640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f71680b27e0 0x7f71680b4cf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x56113a850c30 tx=0x7f71740094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.713+0000 7f7167fff640 1 -- 172.21.15.39:0/512000307 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c049070 con 0x56113a48d4d0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f7167fff640 1 -- 172.21.15.39:0/512000307 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f716c003e40 con 0x56113a48d4d0 2024-05-04T03:18:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f7167fff640 1 -- 172.21.15.39:0/512000307 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0ca3a0 con 0x56113a48d4d0 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f718b17d740 1 -- 172.21.15.39:0/512000307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 msgr2=0x56113a856ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f718b17d740 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a856ee0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f716c0c7170 tx=0x7f716c009260 comp rx=0 tx=0).stop 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f718b17d740 1 -- 172.21.15.39:0/512000307 shutdown_connections 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f718b17d740 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a856ee0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f718b17d740 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 0x56113a84ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f718b17d740 1 --2- 172.21.15.39:0/512000307 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a84fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.714+0000 7f718b17d740 1 -- 172.21.15.39:0/512000307 >> 172.21.15.39:0/512000307 conn(0x56113a4a2300 msgr2=0x56113a84b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.715+0000 7f718b17d740 1 -- 172.21.15.39:0/512000307 shutdown_connections 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.715+0000 7f718b17d740 1 -- 172.21.15.39:0/512000307 wait complete. 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.715+0000 7f718b17d740 1 -- start start 2024-05-04T03:18:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.715+0000 7f718b17d740 1 Processor -- start 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718b17d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 0x56113a859280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718b17d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a8597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718b17d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a860720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718b17d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56113a7b45d0 con 0x56113a48d4d0 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718b17d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56113a771370 con 0x56113a4ae1b0 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718b17d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56113a8576f0 con 0x56113a4852f0 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718b17d740 10 client.0 _ll_register_callbacks cb 0x56113a4c5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a8597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a8597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46790/0 (socket says 172.21.15.39:46790) 2024-05-04T03:18:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 -- 172.21.15.39:0/3581734207 learned_addr learned my addr 172.21.15.39:0/3581734207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f7181f8f640 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 0x56113a859280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f7182790640 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a860720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 -- 172.21.15.39:0/3581734207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 msgr2=0x56113a859280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 0x56113a859280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 -- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 msgr2=0x56113a860720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a860720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 -- 172.21.15.39:0/3581734207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113a592620 con 0x56113a48d4d0 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f7182790640 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a860720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718178e640 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a8597c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f716c1081b0 tx=0x7f716c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36a38b217ba2e44 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f71657fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f71657fa640 1 -- 172.21.15.39:0/3581734207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0e4020 con 0x56113a48d4d0 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f71657fa640 1 -- 172.21.15.39:0/3581734207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f716c0ca370 con 0x56113a48d4d0 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f718b17d740 1 -- 172.21.15.39:0/3581734207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56113a7c21e0 con 0x56113a48d4d0 2024-05-04T03:18:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.716+0000 7f71657fa640 1 -- 172.21.15.39:0/3581734207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f716c049070 con 0x56113a48d4d0 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f718b17d740 10 client.16191 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f718b17d740 1 -- 172.21.15.39:0/3581734207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56113a863650 con 0x56113a48d4d0 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f71657fa640 1 -- 172.21.15.39:0/3581734207 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56113a7c21e0 con 0x56113a48d4d0 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f71657fa640 1 -- 172.21.15.39:0/3581734207 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f716c0c9160 con 0x56113a48d4d0 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f718b17d740 10 client.16191 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f718b17d740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f718b17d740 20 client.16191 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f718b17d740 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56113a46cd80 0x56113a866c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f718b17d740 4 client.16191 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f718b17d740 1 -- 172.21.15.39:0/3581734207 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56113a7bc050 con 0x56113a46cd80 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.717+0000 7f7181f8f640 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56113a46cd80 0x56113a866c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.718+0000 7f7181f8f640 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56113a46cd80 0x56113a866c30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51a622bb9df599e server_cookie=e0a3ca053fbcad01 in_seq=0 out_seq=0 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.718+0000 7f71657fa640 10 client.16191 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.718+0000 7f71657fa640 1 -- 172.21.15.39:0/3581734207 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56113a7bc050 con 0x56113a46cd80 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.718+0000 7f71657fa640 10 client.16191 handle_command_reply: tid=0 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 client.16191 shutdown 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 -- 172.21.15.39:0/3581734207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 msgr2=0x56113a8597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a8597c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f716c1081b0 tx=0x7f716c009560 comp rx=0 tx=0).stop 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 -- 172.21.15.39:0/3581734207 shutdown_connections 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56113a46cd80 0x56113a866c30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4ae1b0 0x56113a860720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f71657fa640 0 client.16191 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a8597c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 --2- 172.21.15.39:0/3581734207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a4852f0 0x56113a859280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 -- 172.21.15.39:0/3581734207 >> 172.21.15.39:0/3581734207 conn(0x56113a4a2300 msgr2=0x56113a84e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 -- 172.21.15.39:0/3581734207 shutdown_connections 2024-05-04T03:18:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 1 -- 172.21.15.39:0/3581734207 wait complete. 2024-05-04T03:18:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.719+0000 7f718b17d740 20 client.16191 trim_cache size 0 max 16384 2024-05-04T03:18:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.734+0000 7f718b17d740 1 Processor -- start 2024-05-04T03:18:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.734+0000 7f718b17d740 1 -- start start 2024-05-04T03:18:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718b17d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a46cd80 0x56113a84e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718b17d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a4852f0 0x56113a84e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718b17d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a48d4d0 0x56113a860640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718b17d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1ab2e0 con 0x56113a4852f0 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718b17d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1ace20 con 0x56113a46cd80 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718b17d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1ab9b0 con 0x56113a48d4d0 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a4852f0 0x56113a84e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a4852f0 0x56113a84e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46800/0 (socket says 172.21.15.39:46800) 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 -- 172.21.15.39:0/4216745004 learned_addr learned my addr 172.21.15.39:0/4216745004 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f7181f8f640 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a46cd80 0x56113a84e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 -- 172.21.15.39:0/4216745004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a48d4d0 msgr2=0x56113a860640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a48d4d0 0x56113a860640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 -- 172.21.15.39:0/4216745004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a46cd80 msgr2=0x56113a84e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a46cd80 0x56113a84e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 -- 172.21.15.39:0/4216745004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113a846a80 con 0x56113a4852f0 2024-05-04T03:18:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f7181f8f640 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a46cd80 0x56113a84e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f718178e640 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a4852f0 0x56113a84e7e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f716c100750 tx=0x7f716c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72c0d07bd6a4984 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.735+0000 7f71677fe640 1 -- 172.21.15.39:0/4216745004 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0e4040 con 0x56113a4852f0 2024-05-04T03:18:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f71677fe640 1 -- 172.21.15.39:0/4216745004 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f716c0d2b40 con 0x56113a4852f0 2024-05-04T03:18:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f71677fe640 1 -- 172.21.15.39:0/4216745004 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0d2d60 con 0x56113a4852f0 2024-05-04T03:18:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 -- 172.21.15.39:0/4216745004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a4852f0 msgr2=0x56113a84e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a4852f0 0x56113a84e7e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f716c100750 tx=0x7f716c009260 comp rx=0 tx=0).stop 2024-05-04T03:18:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 -- 172.21.15.39:0/4216745004 shutdown_connections 2024-05-04T03:18:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a48d4d0 0x56113a860640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a4852f0 0x56113a84e7e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 --2- 172.21.15.39:0/4216745004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a46cd80 0x56113a84e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 -- 172.21.15.39:0/4216745004 >> 172.21.15.39:0/4216745004 conn(0x56113a4a2300 msgr2=0x56113a853aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 -- 172.21.15.39:0/4216745004 shutdown_connections 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.736+0000 7f718b17d740 1 -- 172.21.15.39:0/4216745004 wait complete. 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 1 -- start start 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 1 Processor -- start 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a46cd80 0x56113a85e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4852f0 0x56113a85e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a85f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1ac750 con 0x56113a48d4d0 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1ac080 con 0x56113a4852f0 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1a92c0 con 0x56113a46cd80 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f718b17d740 10 client.0 _ll_register_callbacks cb 0x7f7174009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f7182790640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a85f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f7182790640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a85f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46812/0 (socket says 172.21.15.39:46812) 2024-05-04T03:18:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.737+0000 7f7182790640 1 -- 172.21.15.39:0/2779816247 learned_addr learned my addr 172.21.15.39:0/2779816247 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f7181f8f640 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a46cd80 0x56113a85e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f718178e640 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4852f0 0x56113a85e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f7182790640 1 -- 172.21.15.39:0/2779816247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a46cd80 msgr2=0x56113a85e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f7182790640 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a46cd80 0x56113a85e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f7182790640 1 -- 172.21.15.39:0/2779816247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4852f0 msgr2=0x56113a85e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f7182790640 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4852f0 0x56113a85e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f7182790640 1 -- 172.21.15.39:0/2779816247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113a85fd10 con 0x56113a48d4d0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f7182790640 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a85f750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f717400de90 tx=0x7f7174009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11d379fa5ee060 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f71657fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f71657fa640 1 -- 172.21.15.39:0/2779816247 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174012e50 con 0x56113a48d4d0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f71657fa640 1 -- 172.21.15.39:0/2779816247 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7174004b20 con 0x56113a48d4d0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f718b17d740 1 -- 172.21.15.39:0/2779816247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56113a7375f0 con 0x56113a48d4d0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f71657fa640 1 -- 172.21.15.39:0/2779816247 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7174021600 con 0x56113a48d4d0 2024-05-04T03:18:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f718b17d740 10 client.16197 fetch_fsmap learned FSMap version 32 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f718b17d740 1 -- 172.21.15.39:0/2779816247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56113a8610a0 con 0x56113a48d4d0 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.738+0000 7f71657fa640 1 -- 172.21.15.39:0/2779816247 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56113a7375f0 con 0x56113a48d4d0 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f71657fa640 1 -- 172.21.15.39:0/2779816247 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f71740295b0 con 0x56113a48d4d0 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f718b17d740 10 client.16197 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f718b17d740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f718b17d740 20 client.16197 populate_metadata read hostname 'smithi039' 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f718b17d740 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56113a4ae1b0 0x56113a8d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f718b17d740 4 client.16197 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f718b17d740 1 -- 172.21.15.39:0/2779816247 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56113a784c80 con 0x56113a4ae1b0 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f7181f8f640 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56113a4ae1b0 0x56113a8d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f7181f8f640 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56113a4ae1b0 0x56113a8d16c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0a589272514045 server_cookie=3dd401f1dc4828ac in_seq=0 out_seq=0 2024-05-04T03:18:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.739+0000 7f71657fa640 10 client.16197 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f71657fa640 1 -- 172.21.15.39:0/2779816247 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56113a784c80 con 0x56113a4ae1b0 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f71657fa640 10 client.16197 handle_command_reply: tid=0 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 client.16197 shutdown 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 -- 172.21.15.39:0/2779816247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 msgr2=0x56113a85f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a85f750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f717400de90 tx=0x7f7174009a70 comp rx=0 tx=0).stop 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 -- 172.21.15.39:0/2779816247 shutdown_connections 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56113a4ae1b0 0x56113a8d16c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56113a48d4d0 0x56113a85f750 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f71657fa640 0 client.16197 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:18:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56113a4852f0 0x56113a85e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 --2- 172.21.15.39:0/2779816247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56113a46cd80 0x56113a85e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 -- 172.21.15.39:0/2779816247 >> 172.21.15.39:0/2779816247 conn(0x56113a4a2300 msgr2=0x56113a84e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 -- 172.21.15.39:0/2779816247 shutdown_connections 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 1 -- 172.21.15.39:0/2779816247 wait complete. 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.749+0000 7f718b17d740 20 client.16197 trim_cache size 0 max 16384 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:18:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:18:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 -- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f71680b27e0 msgr2=0x7f71680b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f71680b27e0 0x7f71680b4cf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x56113a850c30 tx=0x7f71740094a0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 -- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 msgr2=0x7f717c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 0x7f717c19a7a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f716c009230 tx=0x7f716c0042f0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 -- 172.21.15.39:0/2365603077 shutdown_connections 2024-05-04T03:18:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f71680b27e0 0x7f71680b4cf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f717c109bb0 0x7f717c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f717c104eb0 0x7f717c19a7a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 --2- 172.21.15.39:0/2365603077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c103c70 0x7f717c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 -- 172.21.15.39:0/2365603077 >> 172.21.15.39:0/2365603077 conn(0x7f717c0fee40 msgr2=0x7f717c105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 -- 172.21.15.39:0/2365603077 shutdown_connections 2024-05-04T03:18:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:18:48.750+0000 7f71841f3640 1 -- 172.21.15.39:0/2365603077 wait complete. 2024-05-04T03:18:48.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:48 smithi066 ceph-mon[33264]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:18:48.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:48 smithi167 ceph-mon[33614]: pgmap v626: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:18:50.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:50 smithi039 ceph-mon[29715]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-05-04T03:18:50.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:50 smithi066 ceph-mon[33264]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-05-04T03:18:50.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:50 smithi167 ceph-mon[33614]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-05-04T03:18:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 300.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:18:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:52 smithi039 ceph-mon[29715]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-05-04T03:18:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:52 smithi066 ceph-mon[33264]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-05-04T03:18:52.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:52 smithi167 ceph-mon[33614]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 1.17k op/s 2024-05-04T03:18:54.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:54 smithi039 ceph-mon[29715]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 1.00k op/s 2024-05-04T03:18:54.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:54 smithi167 ceph-mon[33614]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 1.00k op/s 2024-05-04T03:18:54.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:54 smithi066 ceph-mon[33264]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 1.00k op/s 2024-05-04T03:18:55.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:18:55.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:18:55.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:18:55.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:55 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:18:55.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:18:55.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:18:55.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:18:55.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:55 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:18:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:18:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:18:55.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:18:55.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:55 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:18:56.720 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:56 smithi066 ceph-mon[33264]: pgmap v630: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.6 MiB/s wr, 654 op/s 2024-05-04T03:18:56.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 305.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:18:56.889 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:56 smithi039 ceph-mon[29715]: pgmap v630: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.6 MiB/s wr, 654 op/s 2024-05-04T03:18:56.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:56 smithi167 ceph-mon[33614]: pgmap v630: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.6 MiB/s wr, 654 op/s 2024-05-04T03:18:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:18:58 smithi066 ceph-mon[33264]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-05-04T03:18:58.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:18:58 smithi167 ceph-mon[33614]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-05-04T03:18:58.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:18:58 smithi039 ceph-mon[29715]: pgmap v631: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 515 op/s 2024-05-04T03:19:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:00 smithi066 ceph-mon[33264]: pgmap v632: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-05-04T03:19:00.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:00 smithi167 ceph-mon[33614]: pgmap v632: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-05-04T03:19:00.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:00 smithi039 ceph-mon[29715]: pgmap v632: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-05-04T03:19:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 310.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 -- 172.21.15.39:0/884612737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 msgr2=0x7f0e041072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/884612737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 0x7f0e041072a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009900 tx=0x7f0df402a410 comp rx=0 tx=0).stop 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 -- 172.21.15.39:0/884612737 shutdown_connections 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/884612737 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e041077e0 0x7f0e04107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/884612737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 0x7f0e041072a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/884612737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e04102520 0x7f0e04104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 -- 172.21.15.39:0/884612737 >> 172.21.15.39:0/884612737 conn(0x7f0e040fdd30 msgr2=0x7f0e041001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 -- 172.21.15.39:0/884612737 shutdown_connections 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.862+0000 7f0e0b53f640 1 -- 172.21.15.39:0/884612737 wait complete. 2024-05-04T03:19:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e0b53f640 1 Processor -- start 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e0b53f640 1 -- start start 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e0b53f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e04102520 0x7f0e041a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e0b53f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 0x7f0e041a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e0b53f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e041077e0 0x7f0e041a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e0b53f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e04115c10 con 0x7f0e04104e80 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e0b53f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e04115a90 con 0x7f0e04102520 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e0b53f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e04115d90 con 0x7f0e041077e0 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e08ada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 0x7f0e041a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e08ada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 0x7f0e041a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56308/0 (socket says 172.21.15.39:56308) 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e08ada640 1 -- 172.21.15.39:0/3461619482 learned_addr learned my addr 172.21.15.39:0/3461619482 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e092db640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e04102520 0x7f0e041a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e09adc640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e041077e0 0x7f0e041a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e08ada640 1 -- 172.21.15.39:0/3461619482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e041077e0 msgr2=0x7f0e041a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e08ada640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e041077e0 0x7f0e041a7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e08ada640 1 -- 172.21.15.39:0/3461619482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e04102520 msgr2=0x7f0e041a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e08ada640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e04102520 0x7f0e041a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e08ada640 1 -- 172.21.15.39:0/3461619482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e041a8450 con 0x7f0e04104e80 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.863+0000 7f0e09adc640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e041077e0 0x7f0e041a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.864+0000 7f0e08ada640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 0x7f0e041a0e50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009230 tx=0x7f0df40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.864+0000 7f0dfa7fc640 1 -- 172.21.15.39:0/3461619482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df403f030 con 0x7f0e04104e80 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.864+0000 7f0dfa7fc640 1 -- 172.21.15.39:0/3461619482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0df40044c0 con 0x7f0e04104e80 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.864+0000 7f0dfa7fc640 1 -- 172.21.15.39:0/3461619482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4035070 con 0x7f0e04104e80 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.864+0000 7f0e0b53f640 1 -- 172.21.15.39:0/3461619482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e041a86e0 con 0x7f0e04104e80 2024-05-04T03:19:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.864+0000 7f0e0b53f640 1 -- 172.21.15.39:0/3461619482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e041a8bd0 con 0x7f0e04104e80 2024-05-04T03:19:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e123f9740 1 Processor -- start 2024-05-04T03:19:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e123f9740 1 -- start start 2024-05-04T03:19:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0dfa7fc640 1 -- 172.21.15.39:0/3461619482 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0df4042030 con 0x7f0e04104e80 2024-05-04T03:19:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d4131b0 0x55f89d7b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7bbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e123f9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f89d7195d0 con 0x55f89d4131b0 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e123f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f89d6d2890 con 0x55f89d3ea2f0 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e123f9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f89d715590 con 0x55f89d3f24d0 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e09adc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d4131b0 0x55f89d7b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e09adc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d4131b0 0x55f89d7b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56312/0 (socket says 172.21.15.39:56312) 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e09adc640 1 -- 172.21.15.39:0/2076977157 learned_addr learned my addr 172.21.15.39:0/2076977157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e092db640 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e08ada640 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7bbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e09adc640 1 -- 172.21.15.39:0/2076977157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 msgr2=0x55f89d7bbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e09adc640 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7bbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e09adc640 1 -- 172.21.15.39:0/2076977157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 msgr2=0x55f89d7b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e09adc640 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.867+0000 7f0e09adc640 1 -- 172.21.15.39:0/2076977157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89d42b210 con 0x55f89d4131b0 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.868+0000 7f0e09adc640 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d4131b0 0x55f89d7b4a60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc010480 tx=0x7f0dfc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721ecb96bf16dbf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.868+0000 7f0dfa7fc640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0df00b27e0 0x7f0df00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.868+0000 7f0def7fe640 1 -- 172.21.15.39:0/2076977157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc011760 con 0x55f89d4131b0 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.868+0000 7f0e092db640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0df00b27e0 0x7f0df00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.868+0000 7f0def7fe640 1 -- 172.21.15.39:0/2076977157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0dfc011d90 con 0x55f89d4131b0 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.868+0000 7f0def7fe640 1 -- 172.21.15.39:0/2076977157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc01ad60 con 0x55f89d4131b0 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.868+0000 7f0dfa7fc640 1 -- 172.21.15.39:0/3461619482 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0df4102780 con 0x7f0e04104e80 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 -- 172.21.15.39:0/2076977157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d4131b0 msgr2=0x55f89d7b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d4131b0 0x55f89d7b4a60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc010480 tx=0x7f0dfc009f90 comp rx=0 tx=0).stop 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e092db640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0df00b27e0 0x7f0df00b4cf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f89d7b5e40 tx=0x7f0e0000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 -- 172.21.15.39:0/2076977157 shutdown_connections 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7bbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 --2- 172.21.15.39:0/2076977157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d4131b0 0x55f89d7b4a60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 -- 172.21.15.39:0/2076977157 >> 172.21.15.39:0/2076977157 conn(0x55f89d407300 msgr2=0x55f89d7b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 -- 172.21.15.39:0/2076977157 shutdown_connections 2024-05-04T03:19:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 -- 172.21.15.39:0/2076977157 wait complete. 2024-05-04T03:19:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 -- start start 2024-05-04T03:19:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.869+0000 7f0e123f9740 1 Processor -- start 2024-05-04T03:19:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3f24d0 0x55f89d7beca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d4131b0 0x55f89d7c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e123f9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f89d7bc930 con 0x55f89d3f24d0 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e123f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f89d6d6370 con 0x55f89d3ea2f0 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e123f9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f89d7bcab0 con 0x55f89d4131b0 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e123f9740 10 client.0 _ll_register_callbacks cb 0x55f89d42aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e08ada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3f24d0 0x55f89d7beca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e092db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e08ada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3f24d0 0x55f89d7beca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56326/0 (socket says 172.21.15.39:56326) 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e08ada640 1 -- 172.21.15.39:0/1374160066 learned_addr learned my addr 172.21.15.39:0/1374160066 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e09adc640 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d4131b0 0x55f89d7c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e08ada640 1 -- 172.21.15.39:0/1374160066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d4131b0 msgr2=0x55f89d7c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e08ada640 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d4131b0 0x55f89d7c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e08ada640 1 -- 172.21.15.39:0/1374160066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 msgr2=0x55f89d7be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e08ada640 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.870+0000 7f0e08ada640 1 -- 172.21.15.39:0/1374160066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89d4f7620 con 0x55f89d3f24d0 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0e09adc640 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d4131b0 0x55f89d7c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0e092db640 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0e08ada640 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3f24d0 0x55f89d7beca0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0df40c7020 tx=0x7f0df40c62c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7cbdb1af21afe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0decff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0decff9640 1 -- 172.21.15.39:0/1374160066 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4049040 con 0x55f89d3f24d0 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0decff9640 1 -- 172.21.15.39:0/1374160066 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0df40ca220 con 0x55f89d3f24d0 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0e123f9740 1 -- 172.21.15.39:0/1374160066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dfc011d90 con 0x55f89d3f24d0 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0decff9640 1 -- 172.21.15.39:0/1374160066 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0df40c98c0 con 0x55f89d3f24d0 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0e123f9740 10 client.16209 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0e123f9740 1 -- 172.21.15.39:0/1374160066 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f89d7c8b30 con 0x55f89d3f24d0 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0decff9640 1 -- 172.21.15.39:0/1374160066 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0dfc011d90 con 0x55f89d3f24d0 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.871+0000 7f0decff9640 1 -- 172.21.15.39:0/1374160066 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0df40e2070 con 0x55f89d3f24d0 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.872+0000 7f0e123f9740 10 client.16209 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.872+0000 7f0e123f9740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.872+0000 7f0e123f9740 20 client.16209 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.872+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f89d3d1d80 0x55f89d7cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.872+0000 7f0e123f9740 4 client.16209 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.872+0000 7f0e123f9740 1 -- 172.21.15.39:0/1374160066 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f89d7271e0 con 0x55f89d3d1d80 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.872+0000 7f0e092db640 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f89d3d1d80 0x55f89d7cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.873+0000 7f0e092db640 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f89d3d1d80 0x55f89d7cc230 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e720b3378b2cf3c2 server_cookie=4d71a5078a4de0b4 in_seq=0 out_seq=0 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.873+0000 7f0decff9640 10 client.16209 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.873+0000 7f0decff9640 1 -- 172.21.15.39:0/1374160066 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f89d7271e0 con 0x55f89d3d1d80 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.873+0000 7f0decff9640 10 client.16209 handle_command_reply: tid=0 2024-05-04T03:19:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.873+0000 7f0e123f9740 1 client.16209 shutdown 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 -- 172.21.15.39:0/1374160066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3f24d0 msgr2=0x55f89d7beca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3f24d0 0x55f89d7beca0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0df40c7020 tx=0x7f0df40c62c0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 -- 172.21.15.39:0/1374160066 shutdown_connections 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f89d3d1d80 0x55f89d7cc230 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d4131b0 0x55f89d7c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0decff9640 0 client.16209 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3f24d0 0x55f89d7beca0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1374160066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3ea2f0 0x55f89d7be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 -- 172.21.15.39:0/1374160066 >> 172.21.15.39:0/1374160066 conn(0x55f89d407300 msgr2=0x55f89d7af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 -- 172.21.15.39:0/1374160066 shutdown_connections 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 1 -- 172.21.15.39:0/1374160066 wait complete. 2024-05-04T03:19:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.874+0000 7f0e123f9740 20 client.16209 trim_cache size 0 max 16384 2024-05-04T03:19:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.889+0000 7f0e123f9740 1 Processor -- start 2024-05-04T03:19:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.889+0000 7f0e123f9740 1 -- start start 2024-05-04T03:19:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e123f9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041b18b0 con 0x55f89d3ea2f0 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e123f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041b33f0 con 0x55f89d3d1d80 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e123f9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041b1f80 con 0x55f89d3f24d0 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56336/0 (socket says 172.21.15.39:56336) 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 -- 172.21.15.39:0/3568844674 learned_addr learned my addr 172.21.15.39:0/3568844674 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e09adc640 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e092db640 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 -- 172.21.15.39:0/3568844674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 msgr2=0x55f89d7afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 -- 172.21.15.39:0/3568844674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 msgr2=0x55f89d7b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7b8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 -- 172.21.15.39:0/3568844674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89d7b0d00 con 0x55f89d3ea2f0 2024-05-04T03:19:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0e08ada640 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b9100 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0df40c9a60 tx=0x7f0df4009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71751aebb8ca086 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0deeffd640 1 -- 172.21.15.39:0/3568844674 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4049070 con 0x55f89d3ea2f0 2024-05-04T03:19:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0deeffd640 1 -- 172.21.15.39:0/3568844674 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0df40cd020 con 0x55f89d3ea2f0 2024-05-04T03:19:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.890+0000 7f0deeffd640 1 -- 172.21.15.39:0/3568844674 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df40d2780 con 0x55f89d3ea2f0 2024-05-04T03:19:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 -- 172.21.15.39:0/3568844674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 msgr2=0x55f89d7b9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b9100 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0df40c9a60 tx=0x7f0df4009630 comp rx=0 tx=0).stop 2024-05-04T03:19:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 -- 172.21.15.39:0/3568844674 shutdown_connections 2024-05-04T03:19:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7b9100 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 --2- 172.21.15.39:0/3568844674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 -- 172.21.15.39:0/3568844674 >> 172.21.15.39:0/3568844674 conn(0x55f89d407300 msgr2=0x55f89d7bce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 -- 172.21.15.39:0/3568844674 shutdown_connections 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.891+0000 7f0e123f9740 1 -- 172.21.15.39:0/3568844674 wait complete. 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 1 -- start start 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 1 Processor -- start 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7c3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041b2d20 con 0x55f89d3ea2f0 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041b2650 con 0x55f89d3d1d80 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041af890 con 0x55f89d3f24d0 2024-05-04T03:19:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e123f9740 10 client.0 _ll_register_callbacks cb 0x7f0df4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e08ada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e08ada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7c2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56352/0 (socket says 172.21.15.39:56352) 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e08ada640 1 -- 172.21.15.39:0/1539650743 learned_addr learned my addr 172.21.15.39:0/1539650743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.892+0000 7f0e092db640 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e09adc640 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e08ada640 1 -- 172.21.15.39:0/1539650743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 msgr2=0x55f89d7c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e08ada640 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7c3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e08ada640 1 -- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 msgr2=0x55f89d7c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e08ada640 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e08ada640 1 -- 172.21.15.39:0/1539650743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f89d7c3ed0 con 0x55f89d3ea2f0 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e092db640 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e08ada640 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7c2a30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0df4000c00 tx=0x7f0df4100330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8ee0fc8a7211a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0decff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0decff9640 1 -- 172.21.15.39:0/1539650743 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4049070 con 0x55f89d3ea2f0 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0decff9640 1 -- 172.21.15.39:0/1539650743 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0df40c9220 con 0x55f89d3ea2f0 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e123f9740 1 -- 172.21.15.39:0/1539650743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df40cd020 con 0x55f89d3ea2f0 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0decff9640 1 -- 172.21.15.39:0/1539650743 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0df40ca4b0 con 0x55f89d3ea2f0 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e123f9740 10 client.16215 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0e123f9740 1 -- 172.21.15.39:0/1539650743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f89d7c5190 con 0x55f89d3ea2f0 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.893+0000 7f0decff9640 1 -- 172.21.15.39:0/1539650743 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0df40cd020 con 0x55f89d3ea2f0 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0decff9640 1 -- 172.21.15.39:0/1539650743 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0df40e2070 con 0x55f89d3ea2f0 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0e123f9740 10 client.16215 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0e123f9740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0e123f9740 20 client.16215 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f89d4131b0 0x55f89d8367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0e123f9740 4 client.16215 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0e123f9740 1 -- 172.21.15.39:0/1539650743 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f89d721050 con 0x55f89d4131b0 2024-05-04T03:19:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0e092db640 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f89d4131b0 0x55f89d8367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0decff9640 10 client.16215 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.894+0000 7f0e092db640 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f89d4131b0 0x55f89d8367b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126419ff8a3d96df server_cookie=16c38d87a5f3da51 in_seq=0 out_seq=0 2024-05-04T03:19:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0decff9640 1 -- 172.21.15.39:0/1539650743 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f89d721050 con 0x55f89d4131b0 2024-05-04T03:19:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0decff9640 10 client.16215 handle_command_reply: tid=0 2024-05-04T03:19:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 client.16215 shutdown 2024-05-04T03:19:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 -- 172.21.15.39:0/1539650743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 msgr2=0x55f89d7c2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7c2a30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0df4000c00 tx=0x7f0df4100330 comp rx=0 tx=0).stop 2024-05-04T03:19:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 -- 172.21.15.39:0/1539650743 shutdown_connections 2024-05-04T03:19:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f89d4131b0 0x55f89d8367b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f89d3f24d0 0x55f89d7c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f89d3ea2f0 0x55f89d7c2a30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 --2- 172.21.15.39:0/1539650743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f89d3d1d80 0x55f89d7c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0decff9640 0 client.16215 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 -- 172.21.15.39:0/1539650743 >> 172.21.15.39:0/1539650743 conn(0x55f89d407300 msgr2=0x55f89d7be720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 -- 172.21.15.39:0/1539650743 shutdown_connections 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 1 -- 172.21.15.39:0/1539650743 wait complete. 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.903+0000 7f0e123f9740 20 client.16215 trim_cache size 0 max 16384 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:19:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:19:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:19:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 -- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0df00b27e0 msgr2=0x7f0df00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0df00b27e0 0x7f0df00b4cf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f89d7b5e40 tx=0x7f0e0000b390 comp rx=0 tx=0).stop 2024-05-04T03:19:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 -- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 msgr2=0x7f0e041a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 0x7f0e041a0e50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009230 tx=0x7f0df40040c0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 -- 172.21.15.39:0/3461619482 shutdown_connections 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0df00b27e0 0x7f0df00b4cf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e041077e0 0x7f0e041a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e04104e80 0x7f0e041a0e50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 --2- 172.21.15.39:0/3461619482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e04102520 0x7f0e041a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 -- 172.21.15.39:0/3461619482 >> 172.21.15.39:0/3461619482 conn(0x7f0e040fdd30 msgr2=0x7f0e04103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 -- 172.21.15.39:0/3461619482 shutdown_connections 2024-05-04T03:19:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:01.904+0000 7f0e0b53f640 1 -- 172.21.15.39:0/3461619482 wait complete. 2024-05-04T03:19:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:02 smithi066 ceph-mon[33264]: pgmap v633: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 387 KiB/s wr, 58 op/s 2024-05-04T03:19:02.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:02 smithi167 ceph-mon[33614]: pgmap v633: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 387 KiB/s wr, 58 op/s 2024-05-04T03:19:02.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:02 smithi039 ceph-mon[29715]: pgmap v633: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 387 KiB/s wr, 58 op/s 2024-05-04T03:19:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:04 smithi066 ceph-mon[33264]: pgmap v634: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 166 KiB/s wr, 30 op/s 2024-05-04T03:19:04.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:04 smithi167 ceph-mon[33614]: pgmap v634: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 166 KiB/s wr, 30 op/s 2024-05-04T03:19:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:04 smithi039 ceph-mon[29715]: pgmap v634: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 166 KiB/s wr, 30 op/s 2024-05-04T03:19:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 315.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:19:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:06 smithi066 ceph-mon[33264]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 185 KiB/s wr, 32 op/s 2024-05-04T03:19:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:06 smithi167 ceph-mon[33614]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 185 KiB/s wr, 32 op/s 2024-05-04T03:19:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:06 smithi039 ceph-mon[29715]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 185 KiB/s wr, 32 op/s 2024-05-04T03:19:08.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:08 smithi066 ceph-mon[33264]: pgmap v636: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 178 KiB/s wr, 34 op/s 2024-05-04T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:08 smithi167 ceph-mon[33614]: pgmap v636: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 178 KiB/s wr, 34 op/s 2024-05-04T03:19:08.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:08 smithi039 ceph-mon[29715]: pgmap v636: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 178 KiB/s wr, 34 op/s 2024-05-04T03:19:10.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:19:10.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:10 smithi066 ceph-mon[33264]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 185 KiB/s wr, 35 op/s 2024-05-04T03:19:10.822 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:10 smithi167 ceph-mon[33614]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 185 KiB/s wr, 35 op/s 2024-05-04T03:19:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:10 smithi039 ceph-mon[29715]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 185 KiB/s wr, 35 op/s 2024-05-04T03:19:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 320.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:19:12.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 -- 172.21.15.39:0/1079598989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 msgr2=0x7f03581100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 --2- 172.21.15.39:0/1079598989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 0x7f03581100a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f035000b0d0 tx=0x7f035000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 -- 172.21.15.39:0/1079598989 shutdown_connections 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 --2- 172.21.15.39:0/1079598989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 0x7f03581100a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 --2- 172.21.15.39:0/1079598989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03581034c0 0x7f03581038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 --2- 172.21.15.39:0/1079598989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0358107df0 0x7f0358108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 -- 172.21.15.39:0/1079598989 >> 172.21.15.39:0/1079598989 conn(0x7f03580fedc0 msgr2=0x7f03581011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 -- 172.21.15.39:0/1079598989 shutdown_connections 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.554+0000 7f036049b700 1 -- 172.21.15.39:0/1079598989 wait complete. 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f036049b700 1 Processor -- start 2024-05-04T03:19:12.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f036049b700 1 -- start start 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f036049b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03581034c0 0x7f03581a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f036049b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0358107df0 0x7f03581a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f036049b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 0x7f035819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f036049b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0358113220 con 0x7f0358108740 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f036049b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f03581130a0 con 0x7f0358107df0 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f036049b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03581133a0 con 0x7f03581034c0 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f035ea38700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 0x7f035819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f035da36700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0358107df0 0x7f03581a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f035ea38700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 0x7f035819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48878/0 (socket says 172.21.15.39:48878) 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f035ea38700 1 -- 172.21.15.39:0/1472953547 learned_addr learned my addr 172.21.15.39:0/1472953547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.555+0000 7f035e237700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03581034c0 0x7f03581a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f035ea38700 1 -- 172.21.15.39:0/1472953547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03581034c0 msgr2=0x7f03581a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f035ea38700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03581034c0 0x7f03581a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f035ea38700 1 -- 172.21.15.39:0/1472953547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0358107df0 msgr2=0x7f03581a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f035ea38700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0358107df0 0x7f03581a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f035ea38700 1 -- 172.21.15.39:0/1472953547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035819b1d0 con 0x7f0358108740 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f035ea38700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 0x7f035819ab30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0350007d90 tx=0x7f0350007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f034f7fe700 1 -- 172.21.15.39:0/1472953547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035001f030 con 0x7f0358108740 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035819b460 con 0x7f0358108740 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f034f7fe700 1 -- 172.21.15.39:0/1472953547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f035001bad0 con 0x7f0358108740 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f034f7fe700 1 -- 172.21.15.39:0/1472953547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350012b70 con 0x7f0358108740 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.556+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035819b9a0 con 0x7f0358108740 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.557+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0358104970 con 0x7f0358108740 2024-05-04T03:19:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.559+0000 7f034f7fe700 1 -- 172.21.15.39:0/1472953547 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0350012d10 con 0x7f0358108740 2024-05-04T03:19:12.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.559+0000 7f034f7fe700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f03440b2880 0x7f03440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.560+0000 7f035e237700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f03440b2880 0x7f03440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.560+0000 7f034f7fe700 1 -- 172.21.15.39:0/1472953547 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f035000f020 con 0x7f0358108740 2024-05-04T03:19:12.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.560+0000 7f035e237700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f03440b2880 0x7f03440b4d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0354005950 tx=0x7f035400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.563+0000 7f034f7fe700 1 -- 172.21.15.39:0/1472953547 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0350029020 con 0x7f0358108740 2024-05-04T03:19:12.565 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:12 smithi039 ceph-mon[29715]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-05-04T03:19:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:12 smithi066 ceph-mon[33264]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-05-04T03:19:12.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.829+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03580512c0 con 0x7f0358108740 2024-05-04T03:19:12.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.830+0000 7f034f7fe700 1 -- 172.21.15.39:0/1472953547 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f03580512c0 con 0x7f0358108740 2024-05-04T03:19:12.832 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:19:12.832 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f03440b2880 msgr2=0x7f03440b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f03440b2880 0x7f03440b4d90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0354005950 tx=0x7f035400a300 comp rx=0 tx=0).stop 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 msgr2=0x7f035819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 0x7f035819ab30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0350007d90 tx=0x7f0350007dc0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 shutdown_connections 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f03440b2880 0x7f03440b4d90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0358108740 0x7f035819ab30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03581034c0 0x7f03581a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 --2- 172.21.15.39:0/1472953547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0358107df0 0x7f03581a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 >> 172.21.15.39:0/1472953547 conn(0x7f03580fedc0 msgr2=0x7f03581075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 shutdown_connections 2024-05-04T03:19:12.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:12.832+0000 7f036049b700 1 -- 172.21.15.39:0/1472953547 wait complete. 2024-05-04T03:19:12.835 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:19:12.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:12 smithi167 ceph-mon[33614]: pgmap v638: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-05-04T03:19:13.509 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 2ab0057c --release --await' 2024-05-04T03:19:13.509 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id 2ab0057c --release --await 2024-05-04T03:19:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:13.809 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:13 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1472953547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:13.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:13 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1472953547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:13.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:13 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1472953547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:14.788 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:14 smithi039 ceph-mon[29715]: pgmap v639: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 222 KiB/s wr, 38 op/s 2024-05-04T03:19:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:14 smithi066 ceph-mon[33264]: pgmap v639: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 222 KiB/s wr, 38 op/s 2024-05-04T03:19:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:14 smithi167 ceph-mon[33614]: pgmap v639: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 222 KiB/s wr, 38 op/s 2024-05-04T03:19:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.004+0000 7f833e15e640 1 -- 172.21.15.39:0/2083775238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338102b50 msgr2=0x7f8338103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.004+0000 7f833e15e640 1 --2- 172.21.15.39:0/2083775238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338102b50 0x7f8338103850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8328009900 tx=0x7f832802a410 comp rx=0 tx=0).stop 2024-05-04T03:19:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 -- 172.21.15.39:0/2083775238 shutdown_connections 2024-05-04T03:19:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 --2- 172.21.15.39:0/2083775238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338108a90 0x7f833810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 --2- 172.21.15.39:0/2083775238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8338103d90 0x7f83381041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 --2- 172.21.15.39:0/2083775238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338102b50 0x7f8338103850 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 -- 172.21.15.39:0/2083775238 >> 172.21.15.39:0/2083775238 conn(0x7f83380fdd40 msgr2=0x7f83381001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 -- 172.21.15.39:0/2083775238 shutdown_connections 2024-05-04T03:19:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 -- 172.21.15.39:0/2083775238 wait complete. 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 Processor -- start 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.005+0000 7f833e15e640 1 -- start start 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f833e15e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8338102b50 0x7f833819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f833e15e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338103d90 0x7f833819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f833e15e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338108a90 0x7f83381a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f833e15e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f833810f630 con 0x7f8338103d90 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f833e15e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f833810f4b0 con 0x7f8338108a90 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f833e15e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f833810f7b0 con 0x7f8338102b50 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338103d90 0x7f833819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338103d90 0x7f833819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48890/0 (socket says 172.21.15.39:48890) 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 -- 172.21.15.39:0/1673670377 learned_addr learned my addr 172.21.15.39:0/1673670377 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f83377fe640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8338102b50 0x7f833819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 -- 172.21.15.39:0/1673670377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8338102b50 msgr2=0x7f833819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8338102b50 0x7f833819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 -- 172.21.15.39:0/1673670377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338108a90 msgr2=0x7f83381a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338108a90 0x7f83381a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 -- 172.21.15.39:0/1673670377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83381a1d70 con 0x7f8338103d90 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8336ffd640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338103d90 0x7f833819a820 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f832c00d950 tx=0x7f832c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8334ff9640 1 -- 172.21.15.39:0/1673670377 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c017070 con 0x7f8338103d90 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f833e15e640 1 -- 172.21.15.39:0/1673670377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83381a2060 con 0x7f8338103d90 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8334ff9640 1 -- 172.21.15.39:0/1673670377 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f832c009c30 con 0x7f8338103d90 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.006+0000 7f8334ff9640 1 -- 172.21.15.39:0/1673670377 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00f620 con 0x7f8338103d90 2024-05-04T03:19:15.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.007+0000 7f833e15e640 1 -- 172.21.15.39:0/1673670377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381a2600 con 0x7f8338103d90 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.009+0000 7f8345010740 1 Processor -- start 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8345010740 1 -- start start 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8345010740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb401b0 0x55c78fee1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8345010740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb172f0 0x55c78fee1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8345010740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb1f4d0 0x55c78fee8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8345010740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c78fe465d0 con 0x55c78fb172f0 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8345010740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c78fdff890 con 0x55c78fb1f4d0 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8345010740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c78fe42590 con 0x55c78fb401b0 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f83377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb172f0 0x55c78fee1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8337fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb401b0 0x55c78fee1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f83377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb172f0 0x55c78fee1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48902/0 (socket says 172.21.15.39:48902) 2024-05-04T03:19:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f83377fe640 1 -- 172.21.15.39:0/2629422588 learned_addr learned my addr 172.21.15.39:0/2629422588 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f8334ff9640 1 -- 172.21.15.39:0/1673670377 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f832c010460 con 0x7f8338103d90 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f83377fe640 1 -- 172.21.15.39:0/2629422588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb401b0 msgr2=0x55c78fee1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f83377fe640 1 --2- 172.21.15.39:0/2629422588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb401b0 0x55c78fee1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f83377fe640 1 -- 172.21.15.39:0/2629422588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb1f4d0 msgr2=0x55c78fee8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f83377fe640 1 --2- 172.21.15.39:0/2629422588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb1f4d0 0x55c78fee8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.010+0000 7f83377fe640 1 -- 172.21.15.39:0/2629422588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78fb58210 con 0x55c78fb172f0 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.011+0000 7f83377fe640 1 --2- 172.21.15.39:0/2629422588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb172f0 0x55c78fee1fe0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8328005760 tx=0x7f8328031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13eb26a196be2544 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.011+0000 7f831dffb640 1 -- 172.21.15.39:0/2629422588 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328035030 con 0x55c78fb172f0 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.011+0000 7f8334ff9640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f83200b27e0 0x7f83200b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.011+0000 7f831dffb640 1 -- 172.21.15.39:0/2629422588 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83280044c0 con 0x55c78fb172f0 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.011+0000 7f831dffb640 1 -- 172.21.15.39:0/2629422588 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328003b50 con 0x55c78fb172f0 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.011+0000 7f8337fff640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f83200b27e0 0x7f83200b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.011+0000 7f8334ff9640 1 -- 172.21.15.39:0/1673670377 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f832c0e1ee0 con 0x7f8338103d90 2024-05-04T03:19:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 -- 172.21.15.39:0/2629422588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb172f0 msgr2=0x55c78fee1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8337fff640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f83200b27e0 0x7f83200b4cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55c78fee2c30 tx=0x7f8330007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 --2- 172.21.15.39:0/2629422588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb172f0 0x55c78fee1fe0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8328005760 tx=0x7f8328031650 comp rx=0 tx=0).stop 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 -- 172.21.15.39:0/2629422588 shutdown_connections 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 --2- 172.21.15.39:0/2629422588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb1f4d0 0x55c78fee8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 --2- 172.21.15.39:0/2629422588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb172f0 0x55c78fee1fe0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 --2- 172.21.15.39:0/2629422588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb401b0 0x55c78fee1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 -- 172.21.15.39:0/2629422588 >> 172.21.15.39:0/2629422588 conn(0x55c78fb34300 msgr2=0x55c78fedd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 -- 172.21.15.39:0/2629422588 shutdown_connections 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 -- 172.21.15.39:0/2629422588 wait complete. 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.012+0000 7f8345010740 1 -- start start 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8345010740 1 Processor -- start 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8345010740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8345010740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78feebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8345010740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 0x55c78fef2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8345010740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c78fee98d0 con 0x55c78fb1f4d0 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8345010740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c78fe03370 con 0x55c78fb401b0 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8345010740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c78fee9a50 con 0x55c78fb172f0 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8345010740 10 client.0 _ll_register_callbacks cb 0x55c78fb57a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8336ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78feebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8336ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78feebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48906/0 (socket says 172.21.15.39:48906) 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8336ffd640 1 -- 172.21.15.39:0/1129194465 learned_addr learned my addr 172.21.15.39:0/1129194465 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8337fff640 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 0x55c78fef2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f83377fe640 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8336ffd640 1 -- 172.21.15.39:0/1129194465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 msgr2=0x55c78feeb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8336ffd640 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8336ffd640 1 -- 172.21.15.39:0/1129194465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 msgr2=0x55c78fef2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8336ffd640 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 0x55c78fef2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f8336ffd640 1 -- 172.21.15.39:0/1129194465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78fc24620 con 0x55c78fb1f4d0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.013+0000 7f83377fe640 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8336ffd640 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78feebb80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f832c0a9210 tx=0x7f832c0a8d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abce72504e4756d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8316ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8316ffd640 1 -- 172.21.15.39:0/1129194465 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c020070 con 0x55c78fb1f4d0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8316ffd640 1 -- 172.21.15.39:0/1129194465 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f832c0a8770 con 0x55c78fb1f4d0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 1 -- 172.21.15.39:0/1129194465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c78fe541e0 con 0x55c78fb1f4d0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8316ffd640 1 -- 172.21.15.39:0/1129194465 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f832c009de0 con 0x55c78fb1f4d0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 10 client.16233 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 1 -- 172.21.15.39:0/1129194465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c78fef5a80 con 0x55c78fb1f4d0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8316ffd640 1 -- 172.21.15.39:0/1129194465 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c78fe541e0 con 0x55c78fb1f4d0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8316ffd640 1 -- 172.21.15.39:0/1129194465 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f832c010870 con 0x55c78fb1f4d0 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 10 client.16233 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 20 client.16233 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c78fafed80 0x55c78fef90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 4 client.16233 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.014+0000 7f8345010740 1 -- 172.21.15.39:0/1129194465 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c78fe4e050 con 0x55c78fafed80 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.015+0000 7f83377fe640 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c78fafed80 0x55c78fef90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.015+0000 7f83377fe640 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c78fafed80 0x55c78fef90e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da6155a2ef0f2bb server_cookie=5ca18329eccfcdc1 in_seq=0 out_seq=0 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.015+0000 7f8316ffd640 10 client.16233 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8316ffd640 1 -- 172.21.15.39:0/1129194465 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c78fe4e050 con 0x55c78fafed80 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8316ffd640 10 client.16233 handle_command_reply: tid=0 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 client.16233 shutdown 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 -- 172.21.15.39:0/1129194465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 msgr2=0x55c78feebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78feebb80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f832c0a9210 tx=0x7f832c0a8d70 comp rx=0 tx=0).stop 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 -- 172.21.15.39:0/1129194465 shutdown_connections 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c78fafed80 0x55c78fef90e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 0x55c78fef2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78feebb80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8316ffd640 0 client.16233 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 --2- 172.21.15.39:0/1129194465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 -- 172.21.15.39:0/1129194465 >> 172.21.15.39:0/1129194465 conn(0x55c78fb34300 msgr2=0x55c78fee0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 -- 172.21.15.39:0/1129194465 shutdown_connections 2024-05-04T03:19:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 1 -- 172.21.15.39:0/1129194465 wait complete. 2024-05-04T03:19:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.016+0000 7f8345010740 20 client.16233 trim_cache size 0 max 16384 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8345010740 1 Processor -- start 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8345010740 1 -- start start 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8345010740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb172f0 0x55c78fedd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8345010740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb1f4d0 0x55c78fef2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8345010740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb401b0 0x55c78fedb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8345010740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83381ab210 con 0x55c78fb401b0 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8345010740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83381acd50 con 0x55c78fb172f0 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8345010740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83381ab8e0 con 0x55c78fb1f4d0 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8337fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb401b0 0x55c78fedb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8337fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb401b0 0x55c78fedb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48910/0 (socket says 172.21.15.39:48910) 2024-05-04T03:19:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8337fff640 1 -- 172.21.15.39:0/3702914615 learned_addr learned my addr 172.21.15.39:0/3702914615 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f83377fe640 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb172f0 0x55c78fedd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8336ffd640 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb1f4d0 0x55c78fef2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8337fff640 1 -- 172.21.15.39:0/3702914615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb1f4d0 msgr2=0x55c78fef2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8337fff640 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb1f4d0 0x55c78fef2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8337fff640 1 -- 172.21.15.39:0/3702914615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb172f0 msgr2=0x55c78fedd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8337fff640 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb172f0 0x55c78fedd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.032+0000 7f8337fff640 1 -- 172.21.15.39:0/3702914615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78feea6c0 con 0x55c78fb401b0 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.033+0000 7f8337fff640 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb401b0 0x55c78fedb970 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f833001e770 tx=0x7f8330012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c2d317d0581007 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.033+0000 7f831d7fa640 1 -- 172.21.15.39:0/3702914615 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833002b070 con 0x55c78fb401b0 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.033+0000 7f831d7fa640 1 -- 172.21.15.39:0/3702914615 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8330010e00 con 0x55c78fb401b0 2024-05-04T03:19:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.033+0000 7f831d7fa640 1 -- 172.21.15.39:0/3702914615 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330026e00 con 0x55c78fb401b0 2024-05-04T03:19:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 -- 172.21.15.39:0/3702914615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb401b0 msgr2=0x55c78fedb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb401b0 0x55c78fedb970 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f833001e770 tx=0x7f8330012000 comp rx=0 tx=0).stop 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 -- 172.21.15.39:0/3702914615 shutdown_connections 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb401b0 0x55c78fedb970 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb1f4d0 0x55c78fef2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 --2- 172.21.15.39:0/3702914615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb172f0 0x55c78fedd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 -- 172.21.15.39:0/3702914615 >> 172.21.15.39:0/3702914615 conn(0x55c78fb34300 msgr2=0x55c78fedcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 -- 172.21.15.39:0/3702914615 shutdown_connections 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 -- 172.21.15.39:0/3702914615 wait complete. 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 -- start start 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.034+0000 7f8345010740 1 Processor -- start 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8345010740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8345010740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78fef0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8345010740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 0x55c78fef13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8345010740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83381ac680 con 0x55c78fb1f4d0 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8345010740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83381abfb0 con 0x55c78fb401b0 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8345010740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83381a91f0 con 0x55c78fb172f0 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8345010740 10 client.0 _ll_register_callbacks cb 0x7f832c0a8f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78fef0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78fef0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48918/0 (socket says 172.21.15.39:48918) 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 -- 172.21.15.39:0/2032070525 learned_addr learned my addr 172.21.15.39:0/2032070525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f83377fe640 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8337fff640 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 0x55c78fef13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 -- 172.21.15.39:0/2032070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 msgr2=0x55c78feeffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 -- 172.21.15.39:0/2032070525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 msgr2=0x55c78fef13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 0x55c78fef13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 -- 172.21.15.39:0/2032070525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78fef19a0 con 0x55c78fb1f4d0 2024-05-04T03:19:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8336ffd640 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78fef0500 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f832c0a5590 tx=0x7f832c0a8ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e0f528f07d3e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.035+0000 7f8316ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8316ffd640 1 -- 172.21.15.39:0/2032070525 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c020070 con 0x55c78fb1f4d0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8316ffd640 1 -- 172.21.15.39:0/2032070525 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f832c0a5090 con 0x55c78fb1f4d0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 1 -- 172.21.15.39:0/2032070525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8330010e00 con 0x55c78fb1f4d0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8316ffd640 1 -- 172.21.15.39:0/2032070525 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f832c0b2370 con 0x55c78fb1f4d0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 10 client.16239 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 1 -- 172.21.15.39:0/2032070525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c78fef2cd0 con 0x55c78fb1f4d0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8316ffd640 1 -- 172.21.15.39:0/2032070525 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8330010e00 con 0x55c78fb1f4d0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8316ffd640 1 -- 172.21.15.39:0/2032070525 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f832c08fa80 con 0x55c78fb1f4d0 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 10 client.16239 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 20 client.16239 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c78fad41c0 0x55c78ff63b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 4 client.16239 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.036+0000 7f8345010740 1 -- 172.21.15.39:0/2032070525 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c78fdc95f0 con 0x55c78fad41c0 2024-05-04T03:19:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.037+0000 7f83377fe640 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c78fad41c0 0x55c78ff63b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.037+0000 7f83377fe640 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c78fad41c0 0x55c78ff63b90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94acfad47e5da2f9 server_cookie=bb8ed8a350291757 in_seq=0 out_seq=0 2024-05-04T03:19:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.037+0000 7f8316ffd640 10 client.16239 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8316ffd640 1 -- 172.21.15.39:0/2032070525 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c78fdc95f0 con 0x55c78fad41c0 2024-05-04T03:19:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8316ffd640 10 client.16239 handle_command_reply: tid=0 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 client.16239 shutdown 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 -- 172.21.15.39:0/2032070525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 msgr2=0x55c78fef0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78fef0500 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f832c0a5590 tx=0x7f832c0a8ba0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 -- 172.21.15.39:0/2032070525 shutdown_connections 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c78fad41c0 0x55c78ff63b90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c78fb401b0 0x55c78fef13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8316ffd640 0 client.16239 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c78fb1f4d0 0x55c78fef0500 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 --2- 172.21.15.39:0/2032070525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c78fb172f0 0x55c78feeffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 -- 172.21.15.39:0/2032070525 >> 172.21.15.39:0/2032070525 conn(0x55c78fb34300 msgr2=0x55c78fefd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 -- 172.21.15.39:0/2032070525 shutdown_connections 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.042+0000 7f8345010740 1 -- 172.21.15.39:0/2032070525 wait complete. 2024-05-04T03:19:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f8345010740 20 client.16239 trim_cache size 0 max 16384 2024-05-04T03:19:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:19:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:19:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:19:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 -- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f83200b27e0 msgr2=0x7f83200b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f83200b27e0 0x7f83200b4cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55c78fee2c30 tx=0x7f8330007520 comp rx=0 tx=0).stop 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 -- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338103d90 msgr2=0x7f833819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338103d90 0x7f833819a820 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f832c00d950 tx=0x7f832c00de10 comp rx=0 tx=0).stop 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 -- 172.21.15.39:0/1673670377 shutdown_connections 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f83200b27e0 0x7f83200b4cf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8338108a90 0x7f83381a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8338103d90 0x7f833819a820 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 --2- 172.21.15.39:0/1673670377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8338102b50 0x7f833819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 -- 172.21.15.39:0/1673670377 >> 172.21.15.39:0/1673670377 conn(0x7f83380fdd40 msgr2=0x7f8338104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 -- 172.21.15.39:0/1673670377 shutdown_connections 2024-05-04T03:19:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:15.043+0000 7f833e15e640 1 -- 172.21.15.39:0/1673670377 wait complete. 2024-05-04T03:19:15.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 -- 172.21.15.39:0/3941954557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4108750 msgr2=0x7efed41100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 --2- 172.21.15.39:0/3941954557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4108750 0x7efed41100b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efecc00b0d0 tx=0x7efecc00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 -- 172.21.15.39:0/3941954557 shutdown_connections 2024-05-04T03:19:15.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 --2- 172.21.15.39:0/3941954557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4108750 0x7efed41100b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 --2- 172.21.15.39:0/3941954557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed4103520 0x7efed4103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 --2- 172.21.15.39:0/3941954557 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed4107e00 0x7efed4108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 -- 172.21.15.39:0/3941954557 >> 172.21.15.39:0/3941954557 conn(0x7efed40fef90 msgr2=0x7efed41013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 -- 172.21.15.39:0/3941954557 shutdown_connections 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.538+0000 7efedabdb700 1 -- 172.21.15.39:0/3941954557 wait complete. 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efedabdb700 1 Processor -- start 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efedabdb700 1 -- start start 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efedabdb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4103520 0x7efed419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efedabdb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed4107e00 0x7efed419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efedabdb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed4108750 0x7efed419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efedabdb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efed4113320 con 0x7efed4103520 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efedabdb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efed41131a0 con 0x7efed4108750 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efedabdb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efed41134a0 con 0x7efed4107e00 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efed8977700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4103520 0x7efed419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efed8977700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4103520 0x7efed419aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48938/0 (socket says 172.21.15.39:48938) 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efed8977700 1 -- 172.21.15.39:0/3171321680 learned_addr learned my addr 172.21.15.39:0/3171321680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efed9178700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed4108750 0x7efed419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efed3fff700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed4107e00 0x7efed419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.539+0000 7efed8977700 1 -- 172.21.15.39:0/3171321680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed4107e00 msgr2=0x7efed419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efed8977700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed4107e00 0x7efed419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efed8977700 1 -- 172.21.15.39:0/3171321680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed4108750 msgr2=0x7efed419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efed8977700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed4108750 0x7efed419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efed8977700 1 -- 172.21.15.39:0/3171321680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed419fdb0 con 0x7efed4103520 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efed8977700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4103520 0x7efed419aee0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efec800b530 tx=0x7efec800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efed1ffb700 1 -- 172.21.15.39:0/3171321680 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8015020 con 0x7efed4103520 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efed1ffb700 1 -- 172.21.15.39:0/3171321680 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efec8011cc0 con 0x7efed4103520 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efed1ffb700 1 -- 172.21.15.39:0/3171321680 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8010c80 con 0x7efed4103520 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efedabdb700 1 -- 172.21.15.39:0/3171321680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed41a00a0 con 0x7efed4103520 2024-05-04T03:19:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.540+0000 7efedabdb700 1 -- 172.21.15.39:0/3171321680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed41a79c0 con 0x7efed4103520 2024-05-04T03:19:15.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.543+0000 7efed1ffb700 1 -- 172.21.15.39:0/3171321680 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efec8018020 con 0x7efed4103520 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.543+0000 7efed1ffb700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efec40b2880 0x7efec40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efed3fff700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efec40b2880 0x7efec40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efed1ffb700 1 -- 172.21.15.39:0/3171321680 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efec80e18e0 con 0x7efed4103520 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efee95efb80 1 Processor -- start 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efee95efb80 1 -- start start 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efed3fff700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efec40b2880 0x7efec40b4d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efed41098a0 tx=0x7efec0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efee95efb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efee95efb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efee95efb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efee95efb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e6eb2f7960 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efee95efb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e6eb2968c0 con 0x55e6eb3d4670 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efee95efb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e6eb2f3920 con 0x55e6eb3d4fc0 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.544+0000 7efed9178700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed9178700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48950/0 (socket says 172.21.15.39:48950) 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed9178700 1 -- 172.21.15.39:0/1745432466 learned_addr learned my addr 172.21.15.39:0/1745432466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed8977700 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed3fff700 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed9178700 1 -- 172.21.15.39:0/1745432466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 msgr2=0x55e6eb3e37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed9178700 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed9178700 1 -- 172.21.15.39:0/1745432466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 msgr2=0x55e6eb3d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed9178700 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed9178700 1 -- 172.21.15.39:0/1745432466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6eb2449d0 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed3fff700 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed8977700 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:15.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efed9178700 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e6110 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7efecc009e40 tx=0x7efecc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4900635e50cabe server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efebeffd700 1 -- 172.21.15.39:0/1745432466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc009450 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efebeffd700 1 -- 172.21.15.39:0/1745432466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efecc020070 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.545+0000 7efebeffd700 1 -- 172.21.15.39:0/1745432466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc005070 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 -- 172.21.15.39:0/1745432466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 msgr2=0x55e6eb3e6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e6110 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7efecc009e40 tx=0x7efecc005b00 comp rx=0 tx=0).stop 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 -- 172.21.15.39:0/1745432466 shutdown_connections 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e6110 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 --2- 172.21.15.39:0/1745432466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 -- 172.21.15.39:0/1745432466 >> 172.21.15.39:0/1745432466 conn(0x55e6eb257f40 msgr2=0x55e6eb3d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 -- 172.21.15.39:0/1745432466 shutdown_connections 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.546+0000 7efee95efb80 1 -- 172.21.15.39:0/1745432466 wait complete. 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.547+0000 7efee95efb80 1 -- start start 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.547+0000 7efee95efb80 1 Processor -- start 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.547+0000 7efee95efb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.547+0000 7efee95efb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.547+0000 7efee95efb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.547+0000 7efee95efb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e6eb3e6a40 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.547+0000 7efee95efb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e6eb25dd30 con 0x55e6eb3d4fc0 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.547+0000 7efee95efb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e6eb3e6bc0 con 0x55e6eb3d4670 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efee95efb80 10 client.0 _ll_register_callbacks cb 0x55e6eb28fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48958/0 (socket says 172.21.15.39:48958) 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 -- 172.21.15.39:0/2760693658 learned_addr learned my addr 172.21.15.39:0/2760693658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed3fff700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed8977700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 -- 172.21.15.39:0/2760693658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 msgr2=0x55e6eb3e0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3e0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 -- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 msgr2=0x55e6eb3e11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 -- 172.21.15.39:0/2760693658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6eb293790 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed3fff700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed8977700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efed9178700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e1710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efecc00b820 tx=0x7efecc007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5c4dadd8713d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efeb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efeb3fff700 1 -- 172.21.15.39:0/2760693658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc003cd0 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efeb3fff700 1 -- 172.21.15.39:0/2760693658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efecc005040 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efee95efb80 1 -- 172.21.15.39:0/2760693658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efecc020070 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efeb3fff700 1 -- 172.21.15.39:0/2760693658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efecc035050 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.548+0000 7efee95efb80 10 client.16251 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efee95efb80 1 -- 172.21.15.39:0/2760693658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6eb3f1640 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efeb3fff700 1 -- 172.21.15.39:0/2760693658 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efecc020070 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efeb3fff700 1 -- 172.21.15.39:0/2760693658 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7efecc011070 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efee95efb80 10 client.16251 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:15.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efee95efb80 10 client.16251 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:19:15.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efee95efb80 20 client.16251 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:15.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efee95efb80 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e6eb3f3b10 0x55e6eb3f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efee95efb80 4 client.16251 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:15.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efee95efb80 1 -- 172.21.15.39:0/2760693658 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6eb305340 con 0x55e6eb3f3b10 2024-05-04T03:19:15.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.549+0000 7efed8977700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e6eb3f3b10 0x55e6eb3f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.550+0000 7efed8977700 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e6eb3f3b10 0x55e6eb3f5f30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42383c466b1a88c1 server_cookie=b31af5f491a9fb15 in_seq=0 out_seq=0 2024-05-04T03:19:15.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.550+0000 7efeb3fff700 10 client.16251 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:15.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.550+0000 7efeb3fff700 1 -- 172.21.15.39:0/2760693658 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6eb305340 con 0x55e6eb3f3b10 2024-05-04T03:19:15.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efeb3fff700 10 client.16251 handle_command_reply: tid=0 2024-05-04T03:19:15.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 client.16251 shutdown 2024-05-04T03:19:15.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 -- 172.21.15.39:0/2760693658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 msgr2=0x55e6eb3e1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e1710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efecc00b820 tx=0x7efecc007bf0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 -- 172.21.15.39:0/2760693658 shutdown_connections 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e6eb3f3b10 0x55e6eb3f5f30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3e1710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efeb3fff700 0 client.16251 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 --2- 172.21.15.39:0/2760693658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 -- 172.21.15.39:0/2760693658 >> 172.21.15.39:0/2760693658 conn(0x55e6eb257f40 msgr2=0x55e6eb3d6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 -- 172.21.15.39:0/2760693658 shutdown_connections 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 1 -- 172.21.15.39:0/2760693658 wait complete. 2024-05-04T03:19:15.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.551+0000 7efee95efb80 20 client.16251 trim_cache size 0 max 16384 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.571+0000 7efee95efb80 1 Processor -- start 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.571+0000 7efee95efb80 1 -- start start 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efee95efb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efee95efb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efee95efb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efee95efb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efed41ac910 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efee95efb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efed41ae450 con 0x55e6eb3d4fc0 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efee95efb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efed41acfe0 con 0x55e6eb3d4670 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3dfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48968/0 (socket says 172.21.15.39:48968) 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 -- 172.21.15.39:0/118572404 learned_addr learned my addr 172.21.15.39:0/118572404 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed3fff700 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed8977700 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 -- 172.21.15.39:0/118572404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 msgr2=0x55e6eb3dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 -- 172.21.15.39:0/118572404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 msgr2=0x55e6eb3e1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 -- 172.21.15.39:0/118572404 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6eb3cfff0 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.572+0000 7efed9178700 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3dfc30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efecc009a20 tx=0x7efecc00b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a67c0b179994704 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efebe7fc700 1 -- 172.21.15.39:0/118572404 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc025470 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efebe7fc700 1 -- 172.21.15.39:0/118572404 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efecc020040 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efebe7fc700 1 -- 172.21.15.39:0/118572404 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc005040 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 -- 172.21.15.39:0/118572404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 msgr2=0x55e6eb3dfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3dfc30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efecc009a20 tx=0x7efecc00b090 comp rx=0 tx=0).stop 2024-05-04T03:19:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 -- 172.21.15.39:0/118572404 shutdown_connections 2024-05-04T03:19:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3dfc30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4670 0x55e6eb3dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 --2- 172.21.15.39:0/118572404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 -- 172.21.15.39:0/118572404 >> 172.21.15.39:0/118572404 conn(0x55e6eb257f40 msgr2=0x55e6eb3e2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 -- 172.21.15.39:0/118572404 shutdown_connections 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.573+0000 7efee95efb80 1 -- 172.21.15.39:0/118572404 wait complete. 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 1 -- start start 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 1 Processor -- start 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efed41add80 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efed41ad6b0 con 0x55e6eb3d4670 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efed41aa8d0 con 0x55e6eb3d4fc0 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.574+0000 7efee95efb80 10 client.0 _ll_register_callbacks cb 0x55e6eb28fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48980/0 (socket says 172.21.15.39:48980) 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 -- 172.21.15.39:0/4282571410 learned_addr learned my addr 172.21.15.39:0/4282571410 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed8977700 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed3fff700 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 -- 172.21.15.39:0/4282571410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 msgr2=0x55e6eb3f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 -- 172.21.15.39:0/4282571410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 msgr2=0x55e6eb3f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3f7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 -- 172.21.15.39:0/4282571410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6eb45e120 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efed9178700 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3f8710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efecc031040 tx=0x7efecc025700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170a2f405f20036e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efeb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efeb3fff700 1 -- 172.21.15.39:0/4282571410 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc025b70 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efeb3fff700 1 -- 172.21.15.39:0/4282571410 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efecc02fb10 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efee95efb80 1 -- 172.21.15.39:0/4282571410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efecc020040 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efeb3fff700 1 -- 172.21.15.39:0/4282571410 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efecc02fd10 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efee95efb80 10 client.16257 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.575+0000 7efee95efb80 1 -- 172.21.15.39:0/4282571410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6eb45f3c0 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efeb3fff700 1 -- 172.21.15.39:0/4282571410 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efecc020040 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efeb3fff700 1 -- 172.21.15.39:0/4282571410 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7efecc011070 con 0x55e6eb3e3cf0 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efee95efb80 10 client.16257 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efee95efb80 10 client.16257 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efee95efb80 20 client.16257 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efee95efb80 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e6eb3f3b10 0x55e6eb4619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efee95efb80 4 client.16257 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2ab0057c", "release": true, "await": true}] 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efee95efb80 1 -- 172.21.15.39:0/4282571410 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2ab0057c", "release": true, "await": true}) -- 0x55e6eb2ff3e0 con 0x55e6eb3f3b10 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.576+0000 7efed8977700 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e6eb3f3b10 0x55e6eb4619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.577+0000 7efeb3fff700 10 client.16257 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.577+0000 7efed8977700 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e6eb3f3b10 0x55e6eb4619d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b774413c84ff9a5b server_cookie=ef81ef6f3918cae5 in_seq=0 out_seq=0 2024-05-04T03:19:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efeb3fff700 1 -- 172.21.15.39:0/4282571410 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e6eb2ff3e0 con 0x55e6eb3f3b10 2024-05-04T03:19:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efeb3fff700 10 client.16257 handle_command_reply: tid=0 2024-05-04T03:19:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 client.16257 shutdown 2024-05-04T03:19:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 -- 172.21.15.39:0/4282571410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 msgr2=0x55e6eb3f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3f8710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efecc031040 tx=0x7efecc025700 comp rx=0 tx=0).stop 2024-05-04T03:19:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 -- 172.21.15.39:0/4282571410 shutdown_connections 2024-05-04T03:19:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e6eb3f3b10 0x55e6eb4619d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e6eb3e3cf0 0x55e6eb3f8710 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e6eb3d4670 0x55e6eb3f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 --2- 172.21.15.39:0/4282571410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6eb3d4fc0 0x55e6eb3f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 -- 172.21.15.39:0/4282571410 >> 172.21.15.39:0/4282571410 conn(0x55e6eb257f40 msgr2=0x55e6eb3e2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efeb3fff700 0 client.16257 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 -- 172.21.15.39:0/4282571410 shutdown_connections 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 1 -- 172.21.15.39:0/4282571410 wait complete. 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.582+0000 7efee95efb80 20 client.16257 trim_cache size 0 max 16384 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:19:15.586 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 20, 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "2ab0057c": { 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 20, 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 83.0, 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:19:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:15.588 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 -- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efec40b2880 msgr2=0x7efec40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efec40b2880 0x7efec40b4d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efed41098a0 tx=0x7efec0009190 comp rx=0 tx=0).stop 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 -- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4103520 msgr2=0x7efed419aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4103520 0x7efed419aee0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efec800b530 tx=0x7efec800b840 comp rx=0 tx=0).stop 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 -- 172.21.15.39:0/3171321680 shutdown_connections 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efec40b2880 0x7efec40b4d90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efed4103520 0x7efed419aee0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efed4108750 0x7efed419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 --2- 172.21.15.39:0/3171321680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efed4107e00 0x7efed419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 -- 172.21.15.39:0/3171321680 >> 172.21.15.39:0/3171321680 conn(0x7efed40fef90 msgr2=0x7efed406c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 -- 172.21.15.39:0/3171321680 shutdown_connections 2024-05-04T03:19:15.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:15.583+0000 7efedabdb700 1 -- 172.21.15.39:0/3171321680 wait complete. 2024-05-04T03:19:16.383 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2ab0057c', seconds elapsed: 37.2 2024-05-04T03:19:16.383 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.05 2024-05-04T03:19:16.383 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:19:16.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:16 smithi039 ceph-mon[29715]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 195 KiB/s wr, 34 op/s 2024-05-04T03:19:16.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 325.0 s, 322.3 tps, lat 1882.058 ms stddev 7038.529 2024-05-04T03:19:16.809 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:16 smithi066 ceph-mon[33264]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 195 KiB/s wr, 34 op/s 2024-05-04T03:19:16.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:16 smithi167 ceph-mon[33614]: pgmap v640: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 195 KiB/s wr, 34 op/s 2024-05-04T03:19:18.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:18 smithi039 ceph-mon[29715]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 673 KiB/s wr, 74 op/s 2024-05-04T03:19:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.783+0000 7ff85b92f700 1 -- 172.21.15.39:0/2296787381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854108740 msgr2=0x7ff8541100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.783+0000 7ff85b92f700 1 --2- 172.21.15.39:0/2296787381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854108740 0x7ff8541100a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009fc0 tx=0x7ff84c00b330 comp rx=0 tx=0).stop 2024-05-04T03:19:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.784+0000 7ff85b92f700 1 -- 172.21.15.39:0/2296787381 shutdown_connections 2024-05-04T03:19:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.784+0000 7ff85b92f700 1 --2- 172.21.15.39:0/2296787381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854108740 0x7ff8541100a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.784+0000 7ff85b92f700 1 --2- 172.21.15.39:0/2296787381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8541034c0 0x7ff8541038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.784+0000 7ff85b92f700 1 --2- 172.21.15.39:0/2296787381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff854107df0 0x7ff854108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.784+0000 7ff85b92f700 1 -- 172.21.15.39:0/2296787381 >> 172.21.15.39:0/2296787381 conn(0x7ff8540fedc0 msgr2=0x7ff8541011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.784+0000 7ff85b92f700 1 -- 172.21.15.39:0/2296787381 shutdown_connections 2024-05-04T03:19:18.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.784+0000 7ff85b92f700 1 -- 172.21.15.39:0/2296787381 wait complete. 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.786+0000 7ff85b92f700 1 Processor -- start 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.786+0000 7ff85b92f700 1 -- start start 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff85b92f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8541034c0 0x7ff85419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff85b92f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854107df0 0x7ff85419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff85b92f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff854108740 0x7ff854199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff85b92f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff854113270 con 0x7ff854107df0 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff85b92f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541130f0 con 0x7ff8541034c0 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff85b92f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541133f0 con 0x7ff854108740 2024-05-04T03:19:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff858eca700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854107df0 0x7ff85419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff8596cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8541034c0 0x7ff85419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff8596cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8541034c0 0x7ff85419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:54990/0 (socket says 172.21.15.39:54990) 2024-05-04T03:19:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff8596cb700 1 -- 172.21.15.39:0/1599347651 learned_addr learned my addr 172.21.15.39:0/1599347651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.787+0000 7ff859ecc700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff854108740 0x7ff854199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.788+0000 7ff858eca700 1 -- 172.21.15.39:0/1599347651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff854108740 msgr2=0x7ff854199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.788+0000 7ff858eca700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff854108740 0x7ff854199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.788+0000 7ff858eca700 1 -- 172.21.15.39:0/1599347651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8541034c0 msgr2=0x7ff85419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.788+0000 7ff858eca700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8541034c0 0x7ff85419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.788+0000 7ff858eca700 1 -- 172.21.15.39:0/1599347651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85419a100 con 0x7ff854107df0 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.788+0000 7ff8596cb700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8541034c0 0x7ff85419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.788+0000 7ff858eca700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854107df0 0x7ff85419fd70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff854076ab0 tx=0x7ff84400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.789+0000 7ff84a7fc700 1 -- 172.21.15.39:0/1599347651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844016070 con 0x7ff854107df0 2024-05-04T03:19:18.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.789+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85419a3f0 con 0x7ff854107df0 2024-05-04T03:19:18.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.789+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85406b610 con 0x7ff854107df0 2024-05-04T03:19:18.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.789+0000 7ff84a7fc700 1 -- 172.21.15.39:0/1599347651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff84400fd20 con 0x7ff854107df0 2024-05-04T03:19:18.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.790+0000 7ff84a7fc700 1 -- 172.21.15.39:0/1599347651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844010df0 con 0x7ff854107df0 2024-05-04T03:19:18.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.791+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff838005330 con 0x7ff854107df0 2024-05-04T03:19:18.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.793+0000 7ff84a7fc700 1 -- 172.21.15.39:0/1599347651 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff844019070 con 0x7ff854107df0 2024-05-04T03:19:18.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.794+0000 7ff84a7fc700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8400b29c0 0x7ff8400b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:18.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.795+0000 7ff84a7fc700 1 -- 172.21.15.39:0/1599347651 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff8440e1f20 con 0x7ff854107df0 2024-05-04T03:19:18.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.795+0000 7ff8596cb700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8400b29c0 0x7ff8400b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.796+0000 7ff8596cb700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8400b29c0 0x7ff8400b4ed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff85419ab90 tx=0x7ff850009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:18.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:18.800+0000 7ff84a7fc700 1 -- 172.21.15.39:0/1599347651 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8440a9bf0 con 0x7ff854107df0 2024-05-04T03:19:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:18 smithi066 ceph-mon[33264]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 673 KiB/s wr, 74 op/s 2024-05-04T03:19:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:18 smithi167 ceph-mon[33614]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 673 KiB/s wr, 74 op/s 2024-05-04T03:19:19.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.261+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff838005620 con 0x7ff854107df0 2024-05-04T03:19:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.262+0000 7ff84a7fc700 1 -- 172.21.15.39:0/1599347651 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7ff8440a94f0 con 0x7ff854107df0 2024-05-04T03:19:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:19:19.267 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:19:19.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8400b29c0 msgr2=0x7ff8400b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:19.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8400b29c0 0x7ff8400b4ed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff85419ab90 tx=0x7ff850009310 comp rx=0 tx=0).stop 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854107df0 msgr2=0x7ff85419fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854107df0 0x7ff85419fd70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff854076ab0 tx=0x7ff84400dc30 comp rx=0 tx=0).stop 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 shutdown_connections 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff8400b29c0 0x7ff8400b4ed0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff854108740 0x7ff854199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8541034c0 0x7ff85419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 --2- 172.21.15.39:0/1599347651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff854107df0 0x7ff85419fd70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 >> 172.21.15.39:0/1599347651 conn(0x7ff8540fedc0 msgr2=0x7ff8541096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 shutdown_connections 2024-05-04T03:19:19.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:19.266+0000 7ff85b92f700 1 -- 172.21.15.39:0/1599347651 wait complete. 2024-05-04T03:19:19.272 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:19:19.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:19 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1599347651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:19.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:19 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1599347651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:19.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:19 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1599347651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:19.958 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 83.05 --await-for 42.0' 2024-05-04T03:19:19.958 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 83.05 --await-for 42.0 2024-05-04T03:19:20.374 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:20.772 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:20 smithi039 ceph-mon[29715]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-04T03:19:20.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:20 smithi066 ceph-mon[33264]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-04T03:19:20.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:20 smithi167 ceph-mon[33614]: pgmap v642: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-05-04T03:19:21.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 330.0 s, 1296.4 tps, lat 77.151 ms stddev 23.358 2024-05-04T03:19:21.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:21 smithi167 ceph-mon[33614]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 282 op/s 2024-05-04T03:19:21.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:21 smithi039 ceph-mon[29715]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 282 op/s 2024-05-04T03:19:22.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:21 smithi066 ceph-mon[33264]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 282 op/s 2024-05-04T03:19:22.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.509+0000 7f4c36ffd700 1 -- 172.21.15.39:0/4245942283 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28003b80 con 0x7f4c38107df0 2024-05-04T03:19:22.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 -- 172.21.15.39:0/4245942283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c38107df0 msgr2=0x7f4c38108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 --2- 172.21.15.39:0/4245942283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c38107df0 0x7f4c38108200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c280099c0 tx=0x7f4c28009cd0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 -- 172.21.15.39:0/4245942283 shutdown_connections 2024-05-04T03:19:22.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 --2- 172.21.15.39:0/4245942283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c381034c0 0x7f4c381038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 --2- 172.21.15.39:0/4245942283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c38108740 0x7f4c381100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 --2- 172.21.15.39:0/4245942283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c38107df0 0x7f4c38108200 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 -- 172.21.15.39:0/4245942283 >> 172.21.15.39:0/4245942283 conn(0x7f4c380fedc0 msgr2=0x7f4c381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:22.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 -- 172.21.15.39:0/4245942283 shutdown_connections 2024-05-04T03:19:22.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.510+0000 7f4c3e434700 1 -- 172.21.15.39:0/4245942283 wait complete. 2024-05-04T03:19:22.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.511+0000 7f4c3e434700 1 Processor -- start 2024-05-04T03:19:22.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.511+0000 7f4c3e434700 1 -- start start 2024-05-04T03:19:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c3e434700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c381034c0 0x7f4c3810c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c3e434700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c38107df0 0x7f4c3810ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c3e434700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c38108740 0x7f4c3810cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c381034c0 0x7f4c3810c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c381034c0 0x7f4c3810c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:55022/0 (socket says 172.21.15.39:55022) 2024-05-04T03:19:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c37fff700 1 -- 172.21.15.39:0/3078740200 learned_addr learned my addr 172.21.15.39:0/3078740200 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38112180 con 0x7f4c38107df0 2024-05-04T03:19:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38112000 con 0x7f4c381034c0 2024-05-04T03:19:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38112300 con 0x7f4c38108740 2024-05-04T03:19:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.512+0000 7f4c3c9d1700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c38108740 0x7f4c3810cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.513+0000 7f4c377fe700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c38107df0 0x7f4c3810ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.513+0000 7f4c37fff700 1 -- 172.21.15.39:0/3078740200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c38108740 msgr2=0x7f4c3810cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.513+0000 7f4c37fff700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c38108740 0x7f4c3810cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.513+0000 7f4c37fff700 1 -- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c38107df0 msgr2=0x7f4c3810ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.513+0000 7f4c37fff700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c38107df0 0x7f4c3810ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.513+0000 7f4c37fff700 1 -- 172.21.15.39:0/3078740200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381ad490 con 0x7f4c381034c0 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.513+0000 7f4c37fff700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c381034c0 0x7f4c3810c510 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000ef10 tx=0x7f4c3000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.514+0000 7f4c357fa700 1 -- 172.21.15.39:0/3078740200 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3000c9c0 con 0x7f4c381034c0 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.514+0000 7f4c357fa700 1 -- 172.21.15.39:0/3078740200 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c30019070 con 0x7f4c381034c0 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.514+0000 7f4c357fa700 1 -- 172.21.15.39:0/3078740200 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30014480 con 0x7f4c381034c0 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.514+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c381ad720 con 0x7f4c381034c0 2024-05-04T03:19:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.514+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381adbe0 con 0x7f4c381034c0 2024-05-04T03:19:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.517+0000 7f4c357fa700 1 -- 172.21.15.39:0/3078740200 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4c30012050 con 0x7f4c381034c0 2024-05-04T03:19:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.518+0000 7f4c357fa700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c200b2900 0x7f4c200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.518+0000 7f4c357fa700 1 -- 172.21.15.39:0/3078740200 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4c300e20d0 con 0x7f4c381034c0 2024-05-04T03:19:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.518+0000 7f4c377fe700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c200b2900 0x7f4c200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.519+0000 7f4c377fe700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c200b2900 0x7f4c200b4e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4c28000c00 tx=0x7f4c2801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:22.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.520+0000 7f4c4ce48b80 1 Processor -- start 2024-05-04T03:19:22.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.520+0000 7f4c4ce48b80 1 -- start start 2024-05-04T03:19:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba13cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13d4b0 0x559eba147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba147cd0 0x559eba14a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c4ce48b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559eba05fad0 con 0x559eba147cd0 2024-05-04T03:19:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba13cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13d4b0 0x559eba147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba13cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34486/0 (socket says 172.21.15.39:34486) 2024-05-04T03:19:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c37fff700 1 -- 172.21.15.39:0/3328937275 learned_addr learned my addr 172.21.15.39:0/3328937275 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c3c9d1700 1 --2- 172.21.15.39:0/3328937275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba147cd0 0x559eba14a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/3328937275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559eb9fc4f30 con 0x559eba13d4b0 2024-05-04T03:19:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.521+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/3328937275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559eba05ba90 con 0x559eba13cb60 2024-05-04T03:19:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.522+0000 7f4c37fff700 1 -- 172.21.15.39:0/3328937275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13d4b0 msgr2=0x559eba147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.522+0000 7f4c37fff700 1 --2- 172.21.15.39:0/3328937275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13d4b0 0x559eba147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.522+0000 7f4c37fff700 1 -- 172.21.15.39:0/3328937275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba147cd0 msgr2=0x559eba14a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.522+0000 7f4c37fff700 1 --2- 172.21.15.39:0/3328937275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba147cd0 0x559eba14a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.522+0000 7f4c37fff700 1 -- 172.21.15.39:0/3328937275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb9faca60 con 0x559eba13cb60 2024-05-04T03:19:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.522+0000 7f4c37fff700 1 --2- 172.21.15.39:0/3328937275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba13cf70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4c300090a0 tx=0x7f4c300a9210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fb186c02145e7b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.522+0000 7f4c1e7fc700 1 -- 172.21.15.39:0/3328937275 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c300a9700 con 0x559eba13cb60 2024-05-04T03:19:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.523+0000 7f4c1e7fc700 1 -- 172.21.15.39:0/3328937275 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c300adab0 con 0x559eba13cb60 2024-05-04T03:19:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.523+0000 7f4c1e7fc700 1 -- 172.21.15.39:0/3328937275 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c300a9a50 con 0x559eba13cb60 2024-05-04T03:19:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/3328937275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 msgr2=0x559eba13cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/3328937275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba13cf70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4c300090a0 tx=0x7f4c300a9210 comp rx=0 tx=0).stop 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/3328937275 shutdown_connections 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/3328937275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba147cd0 0x559eba14a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/3328937275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba13cf70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/3328937275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13d4b0 0x559eba147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/3328937275 >> 172.21.15.39:0/3328937275 conn(0x559eb9fbffd0 msgr2=0x559eba138a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/3328937275 shutdown_connections 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.524+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/3328937275 wait complete. 2024-05-04T03:19:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.525+0000 7f4c4ce48b80 1 -- start start 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.525+0000 7f4c4ce48b80 1 Processor -- start 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.526+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba144da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.526+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba1452e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.526+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 0x559eba145820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.526+0000 7f4c4ce48b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559eba14aaa0 con 0x559eba13d4b0 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.526+0000 7f4c4ce48b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559eba019060 con 0x559eba147cd0 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.526+0000 7f4c4ce48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559eba14ac20 con 0x559eba13cb60 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.526+0000 7f4c4ce48b80 10 client.0 _ll_register_callbacks cb 0x559eb9ff7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba1452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba1452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46766/0 (socket says 172.21.15.39:46766) 2024-05-04T03:19:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c377fe700 1 -- 172.21.15.39:0/4165982702 learned_addr learned my addr 172.21.15.39:0/4165982702 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c3c9d1700 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 0x559eba145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c37fff700 1 -- 172.21.15.39:0/4165982702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 msgr2=0x559eba145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c37fff700 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 0x559eba145820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c37fff700 1 -- 172.21.15.39:0/4165982702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 msgr2=0x559eba1452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c37fff700 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba1452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.527+0000 7f4c37fff700 1 -- 172.21.15.39:0/4165982702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eba0168f0 con 0x559eba13cb60 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.528+0000 7f4c37fff700 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba144da0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c300dff50 tx=0x7f4c300a9c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e690417c7743a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.528+0000 7f4c177fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.528+0000 7f4c177fe700 1 -- 172.21.15.39:0/4165982702 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c300a9700 con 0x559eba13cb60 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.528+0000 7f4c177fe700 1 -- 172.21.15.39:0/4165982702 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4c30007500 con 0x559eba13cb60 2024-05-04T03:19:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.528+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/4165982702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c300adab0 con 0x559eba13cb60 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.528+0000 7f4c177fe700 1 -- 172.21.15.39:0/4165982702 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c300b22e0 con 0x559eba13cb60 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c4ce48b80 10 client.25885 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/4165982702 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559eba14aaa0 con 0x559eba13cb60 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c177fe700 1 -- 172.21.15.39:0/4165982702 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c300beb50 con 0x559eba13cb60 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c177fe700 1 -- 172.21.15.39:0/4165982702 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c300a5070 con 0x559eba13cb60 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c4ce48b80 10 client.25885 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c4ce48b80 10 client.25885 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c4ce48b80 20 client.25885 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559eba157ad0 0x559eba159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c4ce48b80 4 client.25885 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/4165982702 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c300adab0 con 0x559eba157ad0 2024-05-04T03:19:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.529+0000 7f4c377fe700 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559eba157ad0 0x559eba159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.531+0000 7f4c377fe700 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559eba157ad0 0x559eba159ef0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5681f1357fff794c server_cookie=90e6efc00f753c9 in_seq=0 out_seq=0 2024-05-04T03:19:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.531+0000 7f4c177fe700 10 client.25885 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.532+0000 7f4c177fe700 1 -- 172.21.15.39:0/4165982702 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4c300adab0 con 0x559eba157ad0 2024-05-04T03:19:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.532+0000 7f4c177fe700 10 client.25885 handle_command_reply: tid=0 2024-05-04T03:19:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.532+0000 7f4c4ce48b80 1 client.25885 shutdown 2024-05-04T03:19:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/4165982702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 msgr2=0x559eba144da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba144da0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4c300dff50 tx=0x7f4c300a9c10 comp rx=0 tx=0).stop 2024-05-04T03:19:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/4165982702 shutdown_connections 2024-05-04T03:19:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559eba157ad0 0x559eba159ef0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 0x559eba145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13cb60 0x559eba144da0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/4165982702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba1452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c177fe700 0 client.25885 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/4165982702 >> 172.21.15.39:0/4165982702 conn(0x559eb9fbffd0 msgr2=0x559eba13e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.533+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/4165982702 shutdown_connections 2024-05-04T03:19:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.534+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/4165982702 wait complete. 2024-05-04T03:19:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.534+0000 7f4c4ce48b80 20 client.25885 trim_cache size 0 max 16384 2024-05-04T03:19:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c4ce48b80 1 Processor -- start 2024-05-04T03:19:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c4ce48b80 1 -- start start 2024-05-04T03:19:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13cb60 0x559eba1406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba143f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba147cd0 0x559eba15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c4ce48b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b5220 con 0x559eba13d4b0 2024-05-04T03:19:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c4ce48b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b6d60 con 0x559eba13cb60 2024-05-04T03:19:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c4ce48b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b58f0 con 0x559eba147cd0 2024-05-04T03:19:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.569+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13cb60 0x559eba1406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13cb60 0x559eba1406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:55048/0 (socket says 172.21.15.39:55048) 2024-05-04T03:19:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c37fff700 1 -- 172.21.15.39:0/1629516004 learned_addr learned my addr 172.21.15.39:0/1629516004 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c3c9d1700 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba147cd0 0x559eba15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c377fe700 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c3c9d1700 1 -- 172.21.15.39:0/1629516004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13cb60 msgr2=0x559eba1406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c3c9d1700 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13cb60 0x559eba1406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c3c9d1700 1 -- 172.21.15.39:0/1629516004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 msgr2=0x559eba143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c3c9d1700 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba143f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c3c9d1700 1 -- 172.21.15.39:0/1629516004 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eba13d8c0 con 0x559eba147cd0 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.570+0000 7f4c3c9d1700 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba147cd0 0x559eba15cb00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009f50 tx=0x7f4c2c004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c11e62e4633387 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.571+0000 7f4c1dffb700 1 -- 172.21.15.39:0/1629516004 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c004a30 con 0x559eba147cd0 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.571+0000 7f4c1dffb700 1 -- 172.21.15.39:0/1629516004 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c2c003cb0 con 0x559eba147cd0 2024-05-04T03:19:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.571+0000 7f4c1dffb700 1 -- 172.21.15.39:0/1629516004 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0055e0 con 0x559eba147cd0 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.572+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/1629516004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba147cd0 msgr2=0x559eba15cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.572+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba147cd0 0x559eba15cb00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009f50 tx=0x7f4c2c004320 comp rx=0 tx=0).stop 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.573+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/1629516004 shutdown_connections 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.573+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba147cd0 0x559eba15cb00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.573+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba13cb60 0x559eba1406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.573+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/1629516004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13d4b0 0x559eba143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.573+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/1629516004 >> 172.21.15.39:0/1629516004 conn(0x559eb9fbffd0 msgr2=0x559eba13ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.573+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/1629516004 shutdown_connections 2024-05-04T03:19:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.573+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/1629516004 wait complete. 2024-05-04T03:19:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.574+0000 7f4c4ce48b80 1 -- start start 2024-05-04T03:19:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.574+0000 7f4c4ce48b80 1 Processor -- start 2024-05-04T03:19:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.575+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13cb60 0x559eba15af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.575+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13d4b0 0x559eba15b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.575+0000 7f4c4ce48b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 0x559eba1bb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.575+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13d4b0 0x559eba15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.575+0000 7f4c377fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13d4b0 0x559eba15b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34520/0 (socket says 172.21.15.39:34520) 2024-05-04T03:19:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.575+0000 7f4c377fe700 1 -- 172.21.15.39:0/2234728675 learned_addr learned my addr 172.21.15.39:0/2234728675 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.575+0000 7f4c3c9d1700 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 0x559eba1bb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b6690 con 0x559eba13cb60 2024-05-04T03:19:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b5fc0 con 0x559eba147cd0 2024-05-04T03:19:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b31a0 con 0x559eba13d4b0 2024-05-04T03:19:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c4ce48b80 10 client.0 _ll_register_callbacks cb 0x559eb9ff7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c37fff700 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13cb60 0x559eba15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c377fe700 1 -- 172.21.15.39:0/2234728675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 msgr2=0x559eba1bb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c377fe700 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 0x559eba1bb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c377fe700 1 -- 172.21.15.39:0/2234728675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13cb60 msgr2=0x559eba15af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c377fe700 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13cb60 0x559eba15af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c377fe700 1 -- 172.21.15.39:0/2234728675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eba1bb5d0 con 0x559eba13d4b0 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.576+0000 7f4c37fff700 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13cb60 0x559eba15af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.577+0000 7f4c377fe700 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13d4b0 0x559eba15b490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800b580 tx=0x7f4c28005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16bd7ab888923bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.577+0000 7f4c177fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.577+0000 7f4c177fe700 1 -- 172.21.15.39:0/2234728675 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28037070 con 0x559eba13d4b0 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.577+0000 7f4c177fe700 1 -- 172.21.15.39:0/2234728675 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4c2800b990 con 0x559eba13d4b0 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.577+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2c003cb0 con 0x559eba13d4b0 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.578+0000 7f4c177fe700 1 -- 172.21.15.39:0/2234728675 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c28040050 con 0x559eba13d4b0 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.578+0000 7f4c4ce48b80 10 client.25891 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.578+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559eba1c28f0 con 0x559eba13d4b0 2024-05-04T03:19:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c177fe700 1 -- 172.21.15.39:0/2234728675 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4c2c003cb0 con 0x559eba13d4b0 2024-05-04T03:19:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c177fe700 1 -- 172.21.15.39:0/2234728675 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c2800fa70 con 0x559eba13d4b0 2024-05-04T03:19:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c4ce48b80 10 client.25891 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c4ce48b80 10 client.25891 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:19:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c4ce48b80 20 client.25891 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559eba1c3020 0x559eba1c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c4ce48b80 4 client.25891 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 83.05, "await_for": 42.0}] 2024-05-04T03:19:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 83.05, "await_for": 42.0}) -- 0x559eba06d4b0 con 0x559eba1c3020 2024-05-04T03:19:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.579+0000 7f4c37fff700 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559eba1c3020 0x559eba1c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.580+0000 7f4c177fe700 10 client.25891 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.580+0000 7f4c37fff700 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559eba1c3020 0x559eba1c5440 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c8ce036578b200 server_cookie=ad250c1e01eb9977 in_seq=0 out_seq=0 2024-05-04T03:19:22.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c177fe700 1 -- 172.21.15.39:0/2234728675 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559eba06d4b0 con 0x559eba1c3020 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c177fe700 10 client.25891 handle_command_reply: tid=0 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 client.25891 shutdown 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13d4b0 msgr2=0x559eba15b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13d4b0 0x559eba15b490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800b580 tx=0x7f4c28005040 comp rx=0 tx=0).stop 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 shutdown_connections 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559eba1c3020 0x559eba1c5440 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eba13cb60 0x559eba15af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559eba147cd0 0x559eba1bb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 --2- 172.21.15.39:0/2234728675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559eba13d4b0 0x559eba15b490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c177fe700 0 client.25891 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 >> 172.21.15.39:0/2234728675 conn(0x559eb9fbffd0 msgr2=0x559eba137ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 shutdown_connections 2024-05-04T03:19:22.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.955+0000 7f4c4ce48b80 1 -- 172.21.15.39:0/2234728675 wait complete. 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c4ce48b80 20 client.25891 trim_cache size 0 max 16384 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 22, 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "5b1a1bbc": { 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 22, 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:19:22.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 83.0, 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:22.960 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c200b2900 msgr2=0x7f4c200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c200b2900 0x7f4c200b4e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4c28000c00 tx=0x7f4c2801a040 comp rx=0 tx=0).stop 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c381034c0 msgr2=0x7f4c3810c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c381034c0 0x7f4c3810c510 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000ef10 tx=0x7f4c3000c3a0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 shutdown_connections 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4c200b2900 0x7f4c200b4e10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c38108740 0x7f4c3810cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4c381034c0 0x7f4c3810c510 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 --2- 172.21.15.39:0/3078740200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c38107df0 0x7f4c3810ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 >> 172.21.15.39:0/3078740200 conn(0x7f4c380fedc0 msgr2=0x7f4c381011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:22.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 shutdown_connections 2024-05-04T03:19:22.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:22.956+0000 7f4c3e434700 1 -- 172.21.15.39:0/3078740200 wait complete. 2024-05-04T03:19:23.593 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5b1a1bbc', quiesce took 7.2 seconds. Will release after: 23.05 2024-05-04T03:19:24.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:24 smithi066 ceph-mon[33264]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-05-04T03:19:24.631 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:24 smithi167 ceph-mon[33614]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-05-04T03:19:24.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:24 smithi039 ceph-mon[29715]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 571 op/s 2024-05-04T03:19:26.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:26 smithi066 ceph-mon[33264]: pgmap v645: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 677 op/s 2024-05-04T03:19:26.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:26 smithi167 ceph-mon[33614]: pgmap v645: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 677 op/s 2024-05-04T03:19:26.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:26 smithi039 ceph-mon[29715]: pgmap v645: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 677 op/s 2024-05-04T03:19:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 335.0 s, 222.7 tps, lat 76.934 ms stddev 19.768 2024-05-04T03:19:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 -- 172.21.15.39:0/2719062921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e0104740 msgr2=0x7f55e0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 --2- 172.21.15.39:0/2719062921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e0104740 0x7f55e0106b60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02a410 comp rx=0 tx=0).stop 2024-05-04T03:19:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 -- 172.21.15.39:0/2719062921 shutdown_connections 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 --2- 172.21.15.39:0/2719062921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e0109a00 0x7f55e010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 --2- 172.21.15.39:0/2719062921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e01070a0 0x7f55e01094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 --2- 172.21.15.39:0/2719062921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e0104740 0x7f55e0106b60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 -- 172.21.15.39:0/2719062921 >> 172.21.15.39:0/2719062921 conn(0x7f55e00fdd40 msgr2=0x7f55e01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 -- 172.21.15.39:0/2719062921 shutdown_connections 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.159+0000 7f55e8784640 1 -- 172.21.15.39:0/2719062921 wait complete. 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e8784640 1 Processor -- start 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e8784640 1 -- start start 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e8784640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e0104740 0x7f55e019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e8784640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e01070a0 0x7f55e019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e8784640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e0109a00 0x7f55e01a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e8784640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55e010f5e0 con 0x7f55e01070a0 2024-05-04T03:19:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e8784640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55e010f460 con 0x7f55e0109a00 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e8784640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55e010f760 con 0x7f55e0104740 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e5d1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e01070a0 0x7f55e019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e5d1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e01070a0 0x7f55e019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41726/0 (socket says 172.21.15.39:41726) 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e5d1f640 1 -- 172.21.15.39:0/3153261685 learned_addr learned my addr 172.21.15.39:0/3153261685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.160+0000 7f55e6d21640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e0109a00 0x7f55e01a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e6520640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e0104740 0x7f55e019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e5d1f640 1 -- 172.21.15.39:0/3153261685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e0104740 msgr2=0x7f55e019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e0104740 0x7f55e019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e5d1f640 1 -- 172.21.15.39:0/3153261685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e0109a00 msgr2=0x7f55e01a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e0109a00 0x7f55e01a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e5d1f640 1 -- 172.21.15.39:0/3153261685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e01a1de0 con 0x7f55e01070a0 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e01070a0 0x7f55e019a7e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f55d000d950 tx=0x7f55d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55d77fe640 1 -- 172.21.15.39:0/3153261685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0017070 con 0x7f55e01070a0 2024-05-04T03:19:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55d77fe640 1 -- 172.21.15.39:0/3153261685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55d0009c30 con 0x7f55e01070a0 2024-05-04T03:19:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e8784640 1 -- 172.21.15.39:0/3153261685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e01a20d0 con 0x7f55e01070a0 2024-05-04T03:19:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55d77fe640 1 -- 172.21.15.39:0/3153261685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d000f620 con 0x7f55e01070a0 2024-05-04T03:19:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.161+0000 7f55e8784640 1 -- 172.21.15.39:0/3153261685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e01a2620 con 0x7f55e01070a0 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.164+0000 7f55d77fe640 1 -- 172.21.15.39:0/3153261685 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f55d0010460 con 0x7f55e01070a0 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.164+0000 7f55ef63b740 1 Processor -- start 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.164+0000 7f55ef63b740 1 -- start start 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e388a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfbe2f0 0x55b47e388fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e38fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55ef63b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b47e2ed5d0 con 0x55b47dfc64d0 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55ef63b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b47e2a6890 con 0x55b47dfbe2f0 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55ef63b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b47e2e9590 con 0x55b47dfe71b0 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e5d1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e38fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e5d1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e38fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41728/0 (socket says 172.21.15.39:41728) 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e5d1f640 1 -- 172.21.15.39:0/3308807352 learned_addr learned my addr 172.21.15.39:0/3308807352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e6d21640 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e388a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e6520640 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfbe2f0 0x55b47e388fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55d77fe640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55cc0b27e0 0x7f55cc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e6d21640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55cc0b27e0 0x7f55cc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e5d1f640 1 -- 172.21.15.39:0/3308807352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 msgr2=0x55b47e388a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e388a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e5d1f640 1 -- 172.21.15.39:0/3308807352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfbe2f0 msgr2=0x55b47e388fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.165+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfbe2f0 0x55b47e388fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.166+0000 7f55d77fe640 1 -- 172.21.15.39:0/3153261685 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f55d00e1d80 con 0x7f55e01070a0 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.166+0000 7f55e5d1f640 1 -- 172.21.15.39:0/3308807352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47dfff210 con 0x55b47dfc64d0 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.166+0000 7f55e6d21640 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e388a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.166+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e38fee0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f55d00df7d0 tx=0x7f55d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec08e6dac8e46a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.166+0000 7f55d4ff9640 1 -- 172.21.15.39:0/3308807352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0020020 con 0x55b47dfc64d0 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.166+0000 7f55e6d21640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55cc0b27e0 0x7f55cc0b4cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55b47e389c30 tx=0x7f55d8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.166+0000 7f55d4ff9640 1 -- 172.21.15.39:0/3308807352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55d00a8f70 con 0x55b47dfc64d0 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.166+0000 7f55d4ff9640 1 -- 172.21.15.39:0/3308807352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00b2210 con 0x55b47dfc64d0 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 -- 172.21.15.39:0/3308807352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 msgr2=0x55b47e38fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e38fee0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f55d00df7d0 tx=0x7f55d0005880 comp rx=0 tx=0).stop 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 -- 172.21.15.39:0/3308807352 shutdown_connections 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e38fee0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfbe2f0 0x55b47e388fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 --2- 172.21.15.39:0/3308807352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e388a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 -- 172.21.15.39:0/3308807352 >> 172.21.15.39:0/3308807352 conn(0x55b47dfdb300 msgr2=0x55b47e3843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 -- 172.21.15.39:0/3308807352 shutdown_connections 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 -- 172.21.15.39:0/3308807352 wait complete. 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 -- start start 2024-05-04T03:19:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.167+0000 7f55ef63b740 1 Processor -- start 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfbe2f0 0x55b47e3925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e392b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e399a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55ef63b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b47e390870 con 0x55b47dfc64d0 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55ef63b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b47e2aa370 con 0x55b47dfe71b0 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55ef63b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b47e3909f0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55ef63b740 10 client.0 _ll_register_callbacks cb 0x55b47dffea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e392b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e392b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41740/0 (socket says 172.21.15.39:41740) 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 -- 172.21.15.39:0/1622687588 learned_addr learned my addr 172.21.15.39:0/1622687588 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e6520640 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfbe2f0 0x55b47e3925c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e6d21640 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e399a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 -- 172.21.15.39:0/1622687588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfbe2f0 msgr2=0x55b47e3925c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfbe2f0 0x55b47e3925c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 -- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 msgr2=0x55b47e399a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e399a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 -- 172.21.15.39:0/1622687588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47e0cb620 con 0x55b47dfc64d0 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e6d21640 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e399a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e392b00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f55d00a66d0 tx=0x7f55d00df800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9e71b8e40260c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.168+0000 7f55c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1622687588 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0020070 con 0x55b47dfc64d0 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1622687588 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f55d00a3d80 con 0x55b47dfc64d0 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 1 -- 172.21.15.39:0/1622687588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d00a8f70 con 0x55b47dfc64d0 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1622687588 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55d00a8db0 con 0x55b47dfc64d0 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 10 client.16290 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 1 -- 172.21.15.39:0/1622687588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b47e39c990 con 0x55b47dfc64d0 2024-05-04T03:19:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1622687588 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55d00a8f70 con 0x55b47dfc64d0 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1622687588 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f55d00e5050 con 0x55b47dfc64d0 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 10 client.16290 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 20 client.16290 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b47dfa5d80 0x55b47e39fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 4 client.16290 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.169+0000 7f55ef63b740 1 -- 172.21.15.39:0/1622687588 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b47e2fb1e0 con 0x55b47dfa5d80 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.170+0000 7f55e6520640 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b47dfa5d80 0x55b47e39fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.170+0000 7f55e6520640 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b47dfa5d80 0x55b47e39fcf0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5b9390c2fc4c1e server_cookie=43d0434eaca5ec59 in_seq=0 out_seq=0 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.170+0000 7f55c9ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1622687588 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b47e2fb1e0 con 0x55b47dfa5d80 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55c9ffb640 10 client.16290 handle_command_reply: tid=0 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 client.16290 shutdown 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 -- 172.21.15.39:0/1622687588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 msgr2=0x55b47e392b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e392b00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f55d00a66d0 tx=0x7f55d00df800 comp rx=0 tx=0).stop 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 -- 172.21.15.39:0/1622687588 shutdown_connections 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b47dfa5d80 0x55b47e39fcf0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e399a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfc64d0 0x55b47e392b00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55c9ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1622687588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfbe2f0 0x55b47e3925c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 -- 172.21.15.39:0/1622687588 >> 172.21.15.39:0/1622687588 conn(0x55b47dfdb300 msgr2=0x55b47e387300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 -- 172.21.15.39:0/1622687588 shutdown_connections 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 1 -- 172.21.15.39:0/1622687588 wait complete. 2024-05-04T03:19:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.171+0000 7f55ef63b740 20 client.16290 trim_cache size 0 max 16384 2024-05-04T03:19:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.186+0000 7f55ef63b740 1 Processor -- start 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55ef63b740 1 -- start start 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e3889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfc64d0 0x55b47e388f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e38cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55ef63b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01ab2b0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55ef63b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01acdf0 con 0x55b47dfc64d0 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55ef63b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01ab980 con 0x55b47dfe71b0 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e3889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e3889e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41754/0 (socket says 172.21.15.39:41754) 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 -- 172.21.15.39:0/422126815 learned_addr learned my addr 172.21.15.39:0/422126815 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6d21640 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e38cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 -- 172.21.15.39:0/422126815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 msgr2=0x55b47e38cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfc64d0 0x55b47e388f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e38cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 -- 172.21.15.39:0/422126815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfc64d0 msgr2=0x55b47e388f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfc64d0 0x55b47e388f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 -- 172.21.15.39:0/422126815 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47e38fec0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.187+0000 7f55e6520640 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e3889e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0057c0 tx=0x7f55dc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e432d7a3fbb465 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55cbfff640 1 -- 172.21.15.39:0/422126815 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc035040 con 0x55b47dfbe2f0 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55cbfff640 1 -- 172.21.15.39:0/422126815 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55dc031df0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55cbfff640 1 -- 172.21.15.39:0/422126815 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc046ac0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55ef63b740 1 -- 172.21.15.39:0/422126815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 msgr2=0x55b47e3889e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55ef63b740 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e3889e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0057c0 tx=0x7f55dc005fd0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55ef63b740 1 -- 172.21.15.39:0/422126815 shutdown_connections 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55ef63b740 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfe71b0 0x55b47e38cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55ef63b740 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfc64d0 0x55b47e388f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55ef63b740 1 --2- 172.21.15.39:0/422126815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e3889e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55ef63b740 1 -- 172.21.15.39:0/422126815 >> 172.21.15.39:0/422126815 conn(0x55b47dfdb300 msgr2=0x55b47e38bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.188+0000 7f55ef63b740 1 -- 172.21.15.39:0/422126815 shutdown_connections 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 -- 172.21.15.39:0/422126815 wait complete. 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 -- start start 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 Processor -- start 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e396410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfc64d0 0x55b47e396950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e397830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01ac720 con 0x55b47dfbe2f0 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01ac050 con 0x55b47dfe71b0 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01a9290 con 0x55b47dfc64d0 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55ef63b740 10 client.0 _ll_register_callbacks cb 0x7f55dc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55e6520640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e396410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55e6520640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e396410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41756/0 (socket says 172.21.15.39:41756) 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.189+0000 7f55e6520640 1 -- 172.21.15.39:0/1596825121 learned_addr learned my addr 172.21.15.39:0/1596825121 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfc64d0 0x55b47e396950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55e6d21640 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e397830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55e6520640 1 -- 172.21.15.39:0/1596825121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfc64d0 msgr2=0x55b47e396950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55e6520640 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfc64d0 0x55b47e396950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55e6520640 1 -- 172.21.15.39:0/1596825121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 msgr2=0x55b47e397830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55e6520640 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e397830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55e6520640 1 -- 172.21.15.39:0/1596825121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47e397df0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55e6520640 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e396410 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f55dc02a930 tx=0x7f55dc0465a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0abf1c36e6d5d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1596825121 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc054070 con 0x55b47dfbe2f0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55ef63b740 1 -- 172.21.15.39:0/1596825121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55dc031df0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1596825121 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f55dc0043f0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1596825121 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55dc060050 con 0x55b47dfbe2f0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55ef63b740 10 client.16296 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55ef63b740 1 -- 172.21.15.39:0/1596825121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b47e3990f0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.190+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1596825121 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55dc031df0 con 0x55b47dfbe2f0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1596825121 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f55dc050440 con 0x55b47dfbe2f0 2024-05-04T03:19:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55ef63b740 10 client.16296 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55ef63b740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55ef63b740 20 client.16296 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b47df7b1c0 0x55b47e40a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55ef63b740 4 client.16296 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55ef63b740 1 -- 172.21.15.39:0/1596825121 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b47e2f5050 con 0x55b47df7b1c0 2024-05-04T03:19:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b47df7b1c0 0x55b47e40a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55e5d1f640 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b47df7b1c0 0x55b47e40a4c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8aaccaea8286b8 server_cookie=aa8c2707047e9927 in_seq=0 out_seq=0 2024-05-04T03:19:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.191+0000 7f55c9ffb640 10 client.16296 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55c9ffb640 1 -- 172.21.15.39:0/1596825121 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b47e2f5050 con 0x55b47df7b1c0 2024-05-04T03:19:28.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55c9ffb640 10 client.16296 handle_command_reply: tid=0 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 client.16296 shutdown 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 -- 172.21.15.39:0/1596825121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 msgr2=0x55b47e396410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e396410 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f55dc02a930 tx=0x7f55dc0465a0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 -- 172.21.15.39:0/1596825121 shutdown_connections 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b47df7b1c0 0x55b47e40a4c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b47dfe71b0 0x55b47e397830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b47dfc64d0 0x55b47e396950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 --2- 172.21.15.39:0/1596825121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b47dfbe2f0 0x55b47e396410 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55c9ffb640 0 client.16296 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 -- 172.21.15.39:0/1596825121 >> 172.21.15.39:0/1596825121 conn(0x55b47dfdb300 msgr2=0x55b47e392540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 -- 172.21.15.39:0/1596825121 shutdown_connections 2024-05-04T03:19:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 1 -- 172.21.15.39:0/1596825121 wait complete. 2024-05-04T03:19:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.200+0000 7f55ef63b740 20 client.16296 trim_cache size 0 max 16384 2024-05-04T03:19:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:19:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:19:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:19:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 -- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55cc0b27e0 msgr2=0x7f55cc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55cc0b27e0 0x7f55cc0b4cf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55b47e389c30 tx=0x7f55d8004ab0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 -- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e01070a0 msgr2=0x7f55e019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e01070a0 0x7f55e019a7e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f55d000d950 tx=0x7f55d000de10 comp rx=0 tx=0).stop 2024-05-04T03:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 -- 172.21.15.39:0/3153261685 shutdown_connections 2024-05-04T03:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f55cc0b27e0 0x7f55cc0b4cf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55e0109a00 0x7f55e01a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55e01070a0 0x7f55e019a7e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 --2- 172.21.15.39:0/3153261685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55e0104740 0x7f55e019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 -- 172.21.15.39:0/3153261685 >> 172.21.15.39:0/3153261685 conn(0x7f55e00fdd40 msgr2=0x7f55e01067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 -- 172.21.15.39:0/3153261685 shutdown_connections 2024-05-04T03:19:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:28.201+0000 7f55e8784640 1 -- 172.21.15.39:0/3153261685 wait complete. 2024-05-04T03:19:28.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:28 smithi066 ceph-mon[33264]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 742 op/s 2024-05-04T03:19:28.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:28 smithi167 ceph-mon[33614]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 742 op/s 2024-05-04T03:19:28.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:28 smithi039 ceph-mon[29715]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 742 op/s 2024-05-04T03:19:30.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:30 smithi066 ceph-mon[33264]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 709 op/s 2024-05-04T03:19:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:30 smithi167 ceph-mon[33614]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 709 op/s 2024-05-04T03:19:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:30 smithi039 ceph-mon[29715]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 709 op/s 2024-05-04T03:19:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 340.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:19:31.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:31 smithi167 ceph-mon[33614]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 619 op/s 2024-05-04T03:19:31.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:31 smithi039 ceph-mon[29715]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 619 op/s 2024-05-04T03:19:32.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:31 smithi066 ceph-mon[33264]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 619 op/s 2024-05-04T03:19:34.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:34 smithi066 ceph-mon[33264]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-05-04T03:19:34.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:34 smithi167 ceph-mon[33614]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-05-04T03:19:34.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:34 smithi039 ceph-mon[29715]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-05-04T03:19:36.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:36 smithi066 ceph-mon[33264]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-05-04T03:19:36.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:36 smithi167 ceph-mon[33614]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-05-04T03:19:36.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:36 smithi039 ceph-mon[29715]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-05-04T03:19:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 345.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:19:38.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:38 smithi066 ceph-mon[33264]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 727 KiB/s wr, 102 op/s 2024-05-04T03:19:38.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:38 smithi167 ceph-mon[33614]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 727 KiB/s wr, 102 op/s 2024-05-04T03:19:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:38 smithi039 ceph-mon[29715]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 727 KiB/s wr, 102 op/s 2024-05-04T03:19:40.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:40 smithi066 ceph-mon[33264]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 217 KiB/s wr, 39 op/s 2024-05-04T03:19:40.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:40 smithi167 ceph-mon[33614]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 217 KiB/s wr, 39 op/s 2024-05-04T03:19:40.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:40 smithi039 ceph-mon[29715]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 217 KiB/s wr, 39 op/s 2024-05-04T03:19:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.314+0000 7f3ef3b47640 1 -- 172.21.15.39:0/4250065278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec103c80 msgr2=0x7f3eec104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/4250065278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec103c80 0x7f3eec104980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009900 tx=0x7f3ee802a410 comp rx=0 tx=0).stop 2024-05-04T03:19:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 -- 172.21.15.39:0/4250065278 shutdown_connections 2024-05-04T03:19:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/4250065278 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3eec109bc0 0x7f3eec10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/4250065278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eec104ec0 0x7f3eec1052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/4250065278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec103c80 0x7f3eec104980 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 -- 172.21.15.39:0/4250065278 >> 172.21.15.39:0/4250065278 conn(0x7f3eec0fee50 msgr2=0x7f3eec101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 -- 172.21.15.39:0/4250065278 shutdown_connections 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 -- 172.21.15.39:0/4250065278 wait complete. 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 Processor -- start 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.315+0000 7f3ef3b47640 1 -- start start 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef3b47640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eec103c80 0x7f3eec19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef3b47640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec104ec0 0x7f3eec19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef3b47640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3eec109bc0 0x7f3eec1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef3b47640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10f630 con 0x7f3eec104ec0 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef3b47640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10f4b0 con 0x7f3eec109bc0 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef3b47640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10f7b0 con 0x7f3eec103c80 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec104ec0 0x7f3eec19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec104ec0 0x7f3eec19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40070/0 (socket says 172.21.15.39:40070) 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 -- 172.21.15.39:0/981759045 learned_addr learned my addr 172.21.15.39:0/981759045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eec103c80 0x7f3eec19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef20e4640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3eec109bc0 0x7f3eec1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 -- 172.21.15.39:0/981759045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eec103c80 msgr2=0x7f3eec19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eec103c80 0x7f3eec19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 -- 172.21.15.39:0/981759045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3eec109bc0 msgr2=0x7f3eec1a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3eec109bc0 0x7f3eec1a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 -- 172.21.15.39:0/981759045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec1a1e10 con 0x7f3eec104ec0 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec104ec0 0x7f3eec19a810 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00e9f0 tx=0x7f3edc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.316+0000 7f3ee2ffd640 1 -- 172.21.15.39:0/981759045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00cd80 con 0x7f3eec104ec0 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.317+0000 7f3ee2ffd640 1 -- 172.21.15.39:0/981759045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3edc0054b0 con 0x7f3eec104ec0 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.317+0000 7f3ef3b47640 1 -- 172.21.15.39:0/981759045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec1a2100 con 0x7f3eec104ec0 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.317+0000 7f3ee2ffd640 1 -- 172.21.15.39:0/981759045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc010430 con 0x7f3eec104ec0 2024-05-04T03:19:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.317+0000 7f3ef3b47640 1 -- 172.21.15.39:0/981759045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec1a2600 con 0x7f3eec104ec0 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.320+0000 7f3efa9fd740 1 Processor -- start 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.320+0000 7f3efa9fd740 1 -- start start 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.320+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24821b0 0x564ae2823a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24592f0 0x564ae2823fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ee2ffd640 1 -- 172.21.15.39:0/981759045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3edc0046c0 con 0x7f3eec104ec0 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24614d0 0x564ae282aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564ae27885d0 con 0x564ae24821b0 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564ae2741890 con 0x564ae24592f0 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564ae2784590 con 0x564ae24614d0 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24821b0 0x564ae2823a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24821b0 0x564ae2823a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40074/0 (socket says 172.21.15.39:40074) 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 -- 172.21.15.39:0/3837416390 learned_addr learned my addr 172.21.15.39:0/3837416390 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24592f0 0x564ae2823fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24614d0 0x564ae282aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 -- 172.21.15.39:0/3837416390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24614d0 msgr2=0x564ae282aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24614d0 0x564ae282aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 -- 172.21.15.39:0/3837416390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24592f0 msgr2=0x564ae2823fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24592f0 0x564ae2823fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 -- 172.21.15.39:0/3837416390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae249a210 con 0x564ae24821b0 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ee2ffd640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3ed80b27e0 0x7f3ed80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.321+0000 7f3ef20e4640 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24821b0 0x564ae2823a60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009df0 tx=0x7f3ee400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e411ed9d1d2517 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.322+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3ed80b27e0 0x7f3ed80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.322+0000 7f3ee2ffd640 1 -- 172.21.15.39:0/981759045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3edc0e26d0 con 0x7f3eec104ec0 2024-05-04T03:19:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.322+0000 7f3ed7fff640 1 -- 172.21.15.39:0/3837416390 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4003970 con 0x564ae24821b0 2024-05-04T03:19:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.322+0000 7f3ed7fff640 1 -- 172.21.15.39:0/3837416390 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ee4004030 con 0x564ae24821b0 2024-05-04T03:19:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.322+0000 7f3ed7fff640 1 -- 172.21.15.39:0/3837416390 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4011660 con 0x564ae24821b0 2024-05-04T03:19:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3837416390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24821b0 msgr2=0x564ae2823a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3ed80b27e0 0x7f3ed80b4cf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80057c0 tx=0x7f3ee8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24821b0 0x564ae2823a60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009df0 tx=0x7f3ee400f730 comp rx=0 tx=0).stop 2024-05-04T03:19:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3837416390 shutdown_connections 2024-05-04T03:19:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24614d0 0x564ae282aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24592f0 0x564ae2823fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3837416390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24821b0 0x564ae2823a60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3837416390 >> 172.21.15.39:0/3837416390 conn(0x564ae2476300 msgr2=0x564ae281f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3837416390 shutdown_connections 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3837416390 wait complete. 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 -- start start 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.323+0000 7f3efa9fd740 1 Processor -- start 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae282d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae282dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 0x564ae2834bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564ae282b930 con 0x564ae24592f0 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564ae2745370 con 0x564ae24614d0 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564ae282bab0 con 0x564ae24821b0 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3efa9fd740 10 client.0 _ll_register_callbacks cb 0x564ae2499a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef18e3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae282d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef18e3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae282d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40082/0 (socket says 172.21.15.39:40082) 2024-05-04T03:19:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef18e3640 1 -- 172.21.15.39:0/3736685969 learned_addr learned my addr 172.21.15.39:0/3736685969 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae282dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef20e4640 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 0x564ae2834bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef18e3640 1 -- 172.21.15.39:0/3736685969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 msgr2=0x564ae2834bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 0x564ae2834bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef18e3640 1 -- 172.21.15.39:0/3736685969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 msgr2=0x564ae282dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae282dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.324+0000 7f3ef18e3640 1 -- 172.21.15.39:0/3736685969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae2566620 con 0x564ae24592f0 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae282d730 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8006010 tx=0x7f3ee8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4177c9183888b2b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3ed57fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3ed57fa640 1 -- 172.21.15.39:0/3736685969 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee804d030 con 0x564ae24592f0 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3ed57fa640 1 -- 172.21.15.39:0/3736685969 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3ee8004250 con 0x564ae24592f0 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3736685969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee4004030 con 0x564ae24592f0 2024-05-04T03:19:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3ed57fa640 1 -- 172.21.15.39:0/3736685969 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ee8035040 con 0x564ae24592f0 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 10 client.16308 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3736685969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564ae2837b70 con 0x564ae24592f0 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3ed57fa640 1 -- 172.21.15.39:0/3736685969 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ee4004030 con 0x564ae24592f0 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3ed57fa640 1 -- 172.21.15.39:0/3736685969 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3ee803d450 con 0x564ae24592f0 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 10 client.16308 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 20 client.16308 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564ae2440d80 0x564ae283b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 4 client.16308 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.325+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3736685969 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ae27961e0 con 0x564ae2440d80 2024-05-04T03:19:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.326+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564ae2440d80 0x564ae283b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.326+0000 7f3ed57fa640 10 client.16308 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.326+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564ae2440d80 0x564ae283b2c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701e37165eee2b41 server_cookie=b12604e42980d8ce in_seq=0 out_seq=0 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3ed57fa640 1 -- 172.21.15.39:0/3736685969 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ae27961e0 con 0x564ae2440d80 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3ed57fa640 10 client.16308 handle_command_reply: tid=0 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 client.16308 shutdown 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3736685969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 msgr2=0x564ae282d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae282d730 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8006010 tx=0x7f3ee8009260 comp rx=0 tx=0).stop 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3736685969 shutdown_connections 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564ae2440d80 0x564ae283b2c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 0x564ae2834bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae282dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3ed57fa640 0 client.16308 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3736685969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae282d730 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3736685969 >> 172.21.15.39:0/3736685969 conn(0x564ae2476300 msgr2=0x564ae281e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.327+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3736685969 shutdown_connections 2024-05-04T03:19:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.328+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3736685969 wait complete. 2024-05-04T03:19:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.328+0000 7f3efa9fd740 20 client.16308 trim_cache size 0 max 16384 2024-05-04T03:19:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.345+0000 7f3efa9fd740 1 Processor -- start 2024-05-04T03:19:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.345+0000 7f3efa9fd740 1 -- start start 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24592f0 0x564ae282da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24614d0 0x564ae2834b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24821b0 0x564ae28350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec1ab2e0 con 0x564ae24614d0 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec1ace20 con 0x564ae24821b0 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec1ab9b0 con 0x564ae24592f0 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef10e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24614d0 0x564ae2834b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef10e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24614d0 0x564ae2834b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40084/0 (socket says 172.21.15.39:40084) 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef10e2640 1 -- 172.21.15.39:0/3179652500 learned_addr learned my addr 172.21.15.39:0/3179652500 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24592f0 0x564ae282da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef20e4640 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24821b0 0x564ae28350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef10e2640 1 -- 172.21.15.39:0/3179652500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24592f0 msgr2=0x564ae282da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24592f0 0x564ae282da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef10e2640 1 -- 172.21.15.39:0/3179652500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24821b0 msgr2=0x564ae28350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24821b0 0x564ae28350a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef10e2640 1 -- 172.21.15.39:0/3179652500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae281fd00 con 0x564ae24614d0 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24592f0 0x564ae282da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.346+0000 7f3ef20e4640 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24821b0 0x564ae28350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.347+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24614d0 0x564ae2834b60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0e01e0 tx=0x7f3edc0e8550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96477a22042c61d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.347+0000 7f3ed77fe640 1 -- 172.21.15.39:0/3179652500 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0a9650 con 0x564ae24614d0 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.347+0000 7f3ed77fe640 1 -- 172.21.15.39:0/3179652500 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3edc0a9810 con 0x564ae24614d0 2024-05-04T03:19:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.347+0000 7f3ed77fe640 1 -- 172.21.15.39:0/3179652500 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0a9470 con 0x564ae24614d0 2024-05-04T03:19:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3179652500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24614d0 msgr2=0x564ae2834b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24614d0 0x564ae2834b60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0e01e0 tx=0x7f3edc0e8550 comp rx=0 tx=0).stop 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3179652500 shutdown_connections 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24821b0 0x564ae28350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24614d0 0x564ae2834b60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/3179652500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24592f0 0x564ae282da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3179652500 >> 172.21.15.39:0/3179652500 conn(0x564ae2476300 msgr2=0x564ae282d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3179652500 shutdown_connections 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 -- 172.21.15.39:0/3179652500 wait complete. 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 -- start start 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.348+0000 7f3efa9fd740 1 Processor -- start 2024-05-04T03:19:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae28323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae28328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3efa9fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 0x564ae28a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec1ac750 con 0x564ae24592f0 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec1ac080 con 0x564ae24614d0 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3efa9fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec1a92c0 con 0x564ae24821b0 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3efa9fd740 10 client.0 _ll_register_callbacks cb 0x7f3ee8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef18e3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae28323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef18e3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae28323a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40086/0 (socket says 172.21.15.39:40086) 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef18e3640 1 -- 172.21.15.39:0/1521109054 learned_addr learned my addr 172.21.15.39:0/1521109054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef20e4640 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 0x564ae28a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae28328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef18e3640 1 -- 172.21.15.39:0/1521109054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 msgr2=0x564ae28a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 0x564ae28a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef18e3640 1 -- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 msgr2=0x564ae28328e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae28328e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef18e3640 1 -- 172.21.15.39:0/1521109054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae28a1b30 con 0x564ae24592f0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.349+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae28328e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3ef18e3640 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae28323a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3ee803d8c0 tx=0x7f3ee8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427e9b44020df876 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3ed57fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3ed57fa640 1 -- 172.21.15.39:0/1521109054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee804d040 con 0x564ae24592f0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3ed57fa640 1 -- 172.21.15.39:0/1521109054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3ee803d650 con 0x564ae24592f0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3efa9fd740 1 -- 172.21.15.39:0/1521109054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3edc0a9810 con 0x564ae24592f0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3ed57fa640 1 -- 172.21.15.39:0/1521109054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ee8004250 con 0x564ae24592f0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3efa9fd740 10 client.16314 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3efa9fd740 1 -- 172.21.15.39:0/1521109054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564ae28a2e10 con 0x564ae24592f0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.350+0000 7f3ed57fa640 1 -- 172.21.15.39:0/1521109054 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3edc0a9810 con 0x564ae24592f0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.351+0000 7f3ed57fa640 1 -- 172.21.15.39:0/1521109054 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3ee8066470 con 0x564ae24592f0 2024-05-04T03:19:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.351+0000 7f3efa9fd740 10 client.16314 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.351+0000 7f3efa9fd740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.351+0000 7f3efa9fd740 20 client.16314 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.351+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564ae24161c0 0x564ae28a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.351+0000 7f3efa9fd740 4 client.16314 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.351+0000 7f3efa9fd740 1 -- 172.21.15.39:0/1521109054 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ae2790050 con 0x564ae24161c0 2024-05-04T03:19:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.351+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564ae24161c0 0x564ae28a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.352+0000 7f3ed57fa640 10 client.16314 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.352+0000 7f3ef10e2640 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564ae24161c0 0x564ae28a57d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32cceac90b617082 server_cookie=7c38bfe5f29bf226 in_seq=0 out_seq=0 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.360+0000 7f3ed57fa640 1 -- 172.21.15.39:0/1521109054 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ae2790050 con 0x564ae24161c0 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.360+0000 7f3ed57fa640 10 client.16314 handle_command_reply: tid=0 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 client.16314 shutdown 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 -- 172.21.15.39:0/1521109054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 msgr2=0x564ae28323a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae28323a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3ee803d8c0 tx=0x7f3ee8000bc0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 -- 172.21.15.39:0/1521109054 shutdown_connections 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564ae24161c0 0x564ae28a57d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ae24821b0 0x564ae28a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564ae24614d0 0x564ae28328e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3ed57fa640 0 client.16314 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 --2- 172.21.15.39:0/1521109054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564ae24592f0 0x564ae28323a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 -- 172.21.15.39:0/1521109054 >> 172.21.15.39:0/1521109054 conn(0x564ae2476300 msgr2=0x564ae28221f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 -- 172.21.15.39:0/1521109054 shutdown_connections 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 1 -- 172.21.15.39:0/1521109054 wait complete. 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.361+0000 7f3efa9fd740 20 client.16314 trim_cache size 0 max 16384 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:19:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:19:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 -- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3ed80b27e0 msgr2=0x7f3ed80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3ed80b27e0 0x7f3ed80b4cf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80057c0 tx=0x7f3ee8031400 comp rx=0 tx=0).stop 2024-05-04T03:19:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 -- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec104ec0 msgr2=0x7f3eec19a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec104ec0 0x7f3eec19a810 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00e9f0 tx=0x7f3edc00edf0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 -- 172.21.15.39:0/981759045 shutdown_connections 2024-05-04T03:19:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3ed80b27e0 0x7f3ed80b4cf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3eec109bc0 0x7f3eec1a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eec104ec0 0x7f3eec19a810 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 --2- 172.21.15.39:0/981759045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eec103c80 0x7f3eec19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 -- 172.21.15.39:0/981759045 >> 172.21.15.39:0/981759045 conn(0x7f3eec0fee50 msgr2=0x7f3eec105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 -- 172.21.15.39:0/981759045 shutdown_connections 2024-05-04T03:19:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:41.362+0000 7f3ef3b47640 1 -- 172.21.15.39:0/981759045 wait complete. 2024-05-04T03:19:41.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 350.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:19:42.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:42 smithi066 ceph-mon[33264]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 163 KiB/s wr, 33 op/s 2024-05-04T03:19:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:42 smithi167 ceph-mon[33614]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 163 KiB/s wr, 33 op/s 2024-05-04T03:19:42.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:42 smithi039 ceph-mon[29715]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 163 KiB/s wr, 33 op/s 2024-05-04T03:19:44.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:44 smithi066 ceph-mon[33264]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 197 KiB/s wr, 36 op/s 2024-05-04T03:19:44.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:44 smithi167 ceph-mon[33614]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 197 KiB/s wr, 36 op/s 2024-05-04T03:19:44.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:44 smithi039 ceph-mon[29715]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 197 KiB/s wr, 36 op/s 2024-05-04T03:19:46.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:46 smithi066 ceph-mon[33264]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 174 KiB/s wr, 31 op/s 2024-05-04T03:19:46.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:19:46.667 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:46 smithi039 ceph-mon[29715]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 174 KiB/s wr, 31 op/s 2024-05-04T03:19:46.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:46 smithi167 ceph-mon[33614]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 174 KiB/s wr, 31 op/s 2024-05-04T03:19:46.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 355.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:19:46.902 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:48.403 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:48 smithi039 ceph-mon[29715]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 204 KiB/s wr, 38 op/s 2024-05-04T03:19:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/1958203668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b48107df0 msgr2=0x7f3b48108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/1958203668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b48107df0 0x7f3b48108200 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0099c0 tx=0x7f3b3c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/1958203668 shutdown_connections 2024-05-04T03:19:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/1958203668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b48108740 0x7f3b481100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/1958203668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b481034c0 0x7f3b481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/1958203668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b48107df0 0x7f3b48108200 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/1958203668 >> 172.21.15.39:0/1958203668 conn(0x7f3b480fedc0 msgr2=0x7f3b481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:48.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/1958203668 shutdown_connections 2024-05-04T03:19:48.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.516+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/1958203668 wait complete. 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.517+0000 7f3b4ca3b700 1 Processor -- start 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.517+0000 7f3b4ca3b700 1 -- start start 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4ca3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b481034c0 0x7f3b4819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4ca3b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b48107df0 0x7f3b4819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4ca3b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b48108740 0x7f3b48199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4ca3b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48113200 con 0x7f3b481034c0 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4ca3b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48113080 con 0x7f3b48107df0 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4ca3b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48113380 con 0x7f3b48108740 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b481034c0 0x7f3b4819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b481034c0 0x7f3b4819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39864/0 (socket says 172.21.15.39:39864) 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 -- 172.21.15.39:0/319021677 learned_addr learned my addr 172.21.15.39:0/319021677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:48.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b46d9d700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b48108740 0x7f3b48199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b45d9b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b48107df0 0x7f3b4819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 -- 172.21.15.39:0/319021677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b48108740 msgr2=0x7f3b48199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b48108740 0x7f3b48199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 -- 172.21.15.39:0/319021677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b48107df0 msgr2=0x7f3b4819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b48107df0 0x7f3b4819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 -- 172.21.15.39:0/319021677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4819a0f0 con 0x7f3b481034c0 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b45d9b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b48107df0 0x7f3b4819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b4659c700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b481034c0 0x7f3b4819f820 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800d980 tx=0x7f3b3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.518+0000 7f3b377fe700 1 -- 172.21.15.39:0/319021677 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38016070 con 0x7f3b481034c0 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.519+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4819a3e0 con 0x7f3b481034c0 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.519+0000 7f3b377fe700 1 -- 172.21.15.39:0/319021677 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b38004830 con 0x7f3b481034c0 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.519+0000 7f3b377fe700 1 -- 172.21.15.39:0/319021677 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38005440 con 0x7f3b481034c0 2024-05-04T03:19:48.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.519+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b481a66b0 con 0x7f3b481034c0 2024-05-04T03:19:48.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.520+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b480512c0 con 0x7f3b481034c0 2024-05-04T03:19:48.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.521+0000 7f3b377fe700 1 -- 172.21.15.39:0/319021677 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3b380055e0 con 0x7f3b481034c0 2024-05-04T03:19:48.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.522+0000 7f3b377fe700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b300b2880 0x7f3b300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:48.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.522+0000 7f3b45d9b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b300b2880 0x7f3b300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:48.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.522+0000 7f3b377fe700 1 -- 172.21.15.39:0/319021677 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3b380e24a0 con 0x7f3b481034c0 2024-05-04T03:19:48.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.523+0000 7f3b45d9b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b300b2880 0x7f3b300b4d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c006010 tx=0x7f3b3c005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:48.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.526+0000 7f3b377fe700 1 -- 172.21.15.39:0/319021677 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b3801f020 con 0x7f3b481034c0 2024-05-04T03:19:48.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:48 smithi066 ceph-mon[33264]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 204 KiB/s wr, 38 op/s 2024-05-04T03:19:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:48 smithi167 ceph-mon[33614]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 204 KiB/s wr, 38 op/s 2024-05-04T03:19:48.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.794+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b4819af70 con 0x7f3b481034c0 2024-05-04T03:19:48.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.795+0000 7f3b377fe700 1 -- 172.21.15.39:0/319021677 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f3b380aa1d0 con 0x7f3b481034c0 2024-05-04T03:19:48.797 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:19:48.797 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b300b2880 msgr2=0x7f3b300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b300b2880 0x7f3b300b4d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c006010 tx=0x7f3b3c005040 comp rx=0 tx=0).stop 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b481034c0 msgr2=0x7f3b4819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b481034c0 0x7f3b4819f820 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800d980 tx=0x7f3b3800dc90 comp rx=0 tx=0).stop 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 shutdown_connections 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3b300b2880 0x7f3b300b4d90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b481034c0 0x7f3b4819f820 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b48108740 0x7f3b48199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 --2- 172.21.15.39:0/319021677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3b48107df0 0x7f3b4819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 >> 172.21.15.39:0/319021677 conn(0x7f3b480fedc0 msgr2=0x7f3b481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 shutdown_connections 2024-05-04T03:19:48.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:48.798+0000 7f3b4ca3b700 1 -- 172.21.15.39:0/319021677 wait complete. 2024-05-04T03:19:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:19:49.434 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:49 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/319021677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:49.463 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 5b1a1bbc --release --await' 2024-05-04T03:19:49.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id 5b1a1bbc --release --await 2024-05-04T03:19:49.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:49 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/319021677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:49.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:49 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/319021677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:49.756 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:50.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:50 smithi039 ceph-mon[29715]: pgmap v657: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 177 KiB/s wr, 33 op/s 2024-05-04T03:19:50.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:50 smithi066 ceph-mon[33264]: pgmap v657: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 177 KiB/s wr, 33 op/s 2024-05-04T03:19:50.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:50 smithi167 ceph-mon[33614]: pgmap v657: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 177 KiB/s wr, 33 op/s 2024-05-04T03:19:51.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 -- 172.21.15.39:0/2337709865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c107e00 msgr2=0x7f6a2c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 --2- 172.21.15.39:0/2337709865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c107e00 0x7f6a2c108210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6a280099c0 tx=0x7f6a28009cd0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 -- 172.21.15.39:0/2337709865 shutdown_connections 2024-05-04T03:19:51.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 --2- 172.21.15.39:0/2337709865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a2c108750 0x7f6a2c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 --2- 172.21.15.39:0/2337709865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a2c1034d0 0x7f6a2c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 --2- 172.21.15.39:0/2337709865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c107e00 0x7f6a2c108210 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 -- 172.21.15.39:0/2337709865 >> 172.21.15.39:0/2337709865 conn(0x7f6a2c0fef20 msgr2=0x7f6a2c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 -- 172.21.15.39:0/2337709865 shutdown_connections 2024-05-04T03:19:51.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.482+0000 7f6a33db3700 1 -- 172.21.15.39:0/2337709865 wait complete. 2024-05-04T03:19:51.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a33db3700 1 Processor -- start 2024-05-04T03:19:51.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a33db3700 1 -- start start 2024-05-04T03:19:51.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a33db3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a2c1034d0 0x7f6a2c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a33db3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a2c107e00 0x7f6a2c19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a33db3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c108750 0x7f6a2c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a33db3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c113210 con 0x7f6a2c108750 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a33db3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c113090 con 0x7f6a2c107e00 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a33db3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c113390 con 0x7f6a2c1034d0 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a32350700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c108750 0x7f6a2c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a32350700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c108750 0x7f6a2c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39882/0 (socket says 172.21.15.39:39882) 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.483+0000 7f6a32350700 1 -- 172.21.15.39:0/875642463 learned_addr learned my addr 172.21.15.39:0/875642463 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a2c1034d0 0x7f6a2c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a3134e700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a2c107e00 0x7f6a2c19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a32350700 1 -- 172.21.15.39:0/875642463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a2c1034d0 msgr2=0x7f6a2c19bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a32350700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a2c1034d0 0x7f6a2c19bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a32350700 1 -- 172.21.15.39:0/875642463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a2c107e00 msgr2=0x7f6a2c19c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a32350700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a2c107e00 0x7f6a2c19c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a32350700 1 -- 172.21.15.39:0/875642463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c1a6ee0 con 0x7f6a2c108750 2024-05-04T03:19:51.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a32350700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c108750 0x7f6a2c19c630 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6a24007ee0 tx=0x7f6a2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a22ffd700 1 -- 172.21.15.39:0/875642463 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400ddd0 con 0x7f6a2c108750 2024-05-04T03:19:51.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a22ffd700 1 -- 172.21.15.39:0/875642463 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a24004020 con 0x7f6a2c108750 2024-05-04T03:19:51.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a33db3700 1 -- 172.21.15.39:0/875642463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c1a71d0 con 0x7f6a2c108750 2024-05-04T03:19:51.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a22ffd700 1 -- 172.21.15.39:0/875642463 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24015660 con 0x7f6a2c108750 2024-05-04T03:19:51.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.484+0000 7f6a33db3700 1 -- 172.21.15.39:0/875642463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c1a7690 con 0x7f6a2c108750 2024-05-04T03:19:51.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.487+0000 7f6a22ffd700 1 -- 172.21.15.39:0/875642463 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6a24015800 con 0x7f6a2c108750 2024-05-04T03:19:51.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.488+0000 7f6a22ffd700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a180b2880 0x7f6a180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.488+0000 7f6a427c7b80 1 Processor -- start 2024-05-04T03:19:51.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.488+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a180b2880 0x7f6a180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.488+0000 7f6a22ffd700 1 -- 172.21.15.39:0/875642463 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a240e1c90 con 0x7f6a2c108750 2024-05-04T03:19:51.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a427c7b80 1 -- start start 2024-05-04T03:19:51.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a180b2880 0x7f6a180b4d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009c60 tx=0x7f6a1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 0x563f6786aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786b420 0x563f67875700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f67878060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563f6778da40 con 0x563f6786b420 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f676f2ea0 con 0x563f67875c40 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f67789a00 con 0x563f6786aad0 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a3134e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786b420 0x563f67875700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a3134e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786b420 0x563f67875700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39898/0 (socket says 172.21.15.39:39898) 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a3134e700 1 -- 172.21.15.39:0/2088945103 learned_addr learned my addr 172.21.15.39:0/2088945103 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:51.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 0x563f6786aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a32350700 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f67878060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a3134e700 1 -- 172.21.15.39:0/2088945103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 msgr2=0x563f6786aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a3134e700 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 0x563f6786aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a3134e700 1 -- 172.21.15.39:0/2088945103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 msgr2=0x563f67878060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a3134e700 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f67878060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.489+0000 7f6a3134e700 1 -- 172.21.15.39:0/2088945103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f676da9d0 con 0x563f6786b420 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.490+0000 7f6a3134e700 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786b420 0x563f67875700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c076ab0 tx=0x7f6a28005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb1d822cb041def server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.490+0000 7f6a17fff700 1 -- 172.21.15.39:0/2088945103 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28027070 con 0x563f6786b420 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.490+0000 7f6a17fff700 1 -- 172.21.15.39:0/2088945103 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a28009590 con 0x563f6786b420 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.490+0000 7f6a17fff700 1 -- 172.21.15.39:0/2088945103 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800b9f0 con 0x563f6786b420 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2088945103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786b420 msgr2=0x563f67875700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786b420 0x563f67875700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c076ab0 tx=0x7f6a28005040 comp rx=0 tx=0).stop 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2088945103 shutdown_connections 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f67878060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 0x563f6786aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2088945103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786b420 0x563f67875700 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2088945103 >> 172.21.15.39:0/2088945103 conn(0x563f676edf40 msgr2=0x563f678669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2088945103 shutdown_connections 2024-05-04T03:19:51.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2088945103 wait complete. 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 -- start start 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.491+0000 7f6a427c7b80 1 Processor -- start 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 0x563f67872c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f678731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f67875c40 0x563f67873700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563f678789b0 con 0x563f67875c40 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f67746fd0 con 0x563f6786b420 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f67878b30 con 0x563f6786aad0 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a427c7b80 10 client.0 _ll_register_callbacks cb 0x563f67725a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a32350700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f67875c40 0x563f67873700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a32350700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f67875c40 0x563f67873700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39904/0 (socket says 172.21.15.39:39904) 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a32350700 1 -- 172.21.15.39:0/2718654257 learned_addr learned my addr 172.21.15.39:0/2718654257 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 0x563f67872c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a3134e700 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f678731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a32350700 1 -- 172.21.15.39:0/2718654257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 msgr2=0x563f67872c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a32350700 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 0x563f67872c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a32350700 1 -- 172.21.15.39:0/2718654257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 msgr2=0x563f678731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a32350700 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f678731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.492+0000 7f6a32350700 1 -- 172.21.15.39:0/2718654257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f67744860 con 0x563f67875c40 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a32350700 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f67875c40 0x563f67873700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6a24015000 tx=0x7f6a240a8e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d33fc25cb101da8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a157fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a157fa700 1 -- 172.21.15.39:0/2718654257 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240ad610 con 0x563f67875c40 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a157fa700 1 -- 172.21.15.39:0/2718654257 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a240044f0 con 0x563f67875c40 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2718654257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a28009590 con 0x563f67875c40 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a157fa700 1 -- 172.21.15.39:0/2718654257 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a240b20c0 con 0x563f67875c40 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a427c7b80 10 client.16332 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2718654257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f678836b0 con 0x563f67875c40 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a157fa700 1 -- 172.21.15.39:0/2718654257 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a28009590 con 0x563f67875c40 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.493+0000 7f6a157fa700 1 -- 172.21.15.39:0/2718654257 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6a240a95c0 con 0x563f67875c40 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a427c7b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:51.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a427c7b80 10 client.16332 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a427c7b80 20 client.16332 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f67885c40 0x563f67888060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a427c7b80 4 client.16332 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2718654257 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f6779b420 con 0x563f67885c40 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f67885c40 0x563f67888060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a157fa700 10 client.16332 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.494+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f67885c40 0x563f67888060 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43193927cd8b0757 server_cookie=6a1b32e196e7cec4 in_seq=0 out_seq=0 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.495+0000 7f6a157fa700 1 -- 172.21.15.39:0/2718654257 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f6779b420 con 0x563f67885c40 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.495+0000 7f6a157fa700 10 client.16332 handle_command_reply: tid=0 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.495+0000 7f6a427c7b80 1 client.16332 shutdown 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.495+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2718654257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f67875c40 msgr2=0x563f67873700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f67875c40 0x563f67873700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6a24015000 tx=0x7f6a240a8e60 comp rx=0 tx=0).stop 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2718654257 shutdown_connections 2024-05-04T03:19:51.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f67885c40 0x563f67888060 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f67875c40 0x563f67873700 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786aad0 0x563f67872c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a157fa700 0 client.16332 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:51.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2718654257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f678731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2718654257 >> 172.21.15.39:0/2718654257 conn(0x563f676edf40 msgr2=0x563f6786c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2718654257 shutdown_connections 2024-05-04T03:19:51.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2718654257 wait complete. 2024-05-04T03:19:51.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.496+0000 7f6a427c7b80 20 client.16332 trim_cache size 0 max 16384 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.516+0000 7f6a427c7b80 1 Processor -- start 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.516+0000 7f6a427c7b80 1 -- start start 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f67874a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786b420 0x563f678731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f6788d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1ae990 con 0x563f6786aad0 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1b04d0 con 0x563f67875c40 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1af060 con 0x563f6786b420 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f67874a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f67874a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39908/0 (socket says 172.21.15.39:39908) 2024-05-04T03:19:51.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 -- 172.21.15.39:0/1616069690 learned_addr learned my addr 172.21.15.39:0/1616069690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a3134e700 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786b420 0x563f678731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a32350700 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f6788d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 -- 172.21.15.39:0/1616069690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786b420 msgr2=0x563f678731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786b420 0x563f678731f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 -- 172.21.15.39:0/1616069690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 msgr2=0x563f6788d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f6788d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 -- 172.21.15.39:0/1616069690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f678711f0 con 0x563f6786aad0 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a32350700 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f6788d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a3134e700 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786b420 0x563f678731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:51.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f67874a40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x563f67867380 tx=0x7f6a1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b48452f1e4d14ef server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.517+0000 7f6a177fe700 1 -- 172.21.15.39:0/1616069690 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c031070 con 0x563f6786aad0 2024-05-04T03:19:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.518+0000 7f6a177fe700 1 -- 172.21.15.39:0/1616069690 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a1c02c5f0 con 0x563f6786aad0 2024-05-04T03:19:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.518+0000 7f6a177fe700 1 -- 172.21.15.39:0/1616069690 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c01d780 con 0x563f6786aad0 2024-05-04T03:19:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 -- 172.21.15.39:0/1616069690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 msgr2=0x563f67874a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f67874a40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x563f67867380 tx=0x7f6a1c014000 comp rx=0 tx=0).stop 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 -- 172.21.15.39:0/1616069690 shutdown_connections 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f67874a40 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f67875c40 0x563f6788d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/1616069690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f6786b420 0x563f678731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 -- 172.21.15.39:0/1616069690 >> 172.21.15.39:0/1616069690 conn(0x563f676edf40 msgr2=0x563f67874330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 -- 172.21.15.39:0/1616069690 shutdown_connections 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 -- 172.21.15.39:0/1616069690 wait complete. 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 -- start start 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.519+0000 7f6a427c7b80 1 Processor -- start 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f6787e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f6787ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a427c7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f67875c40 0x563f6787b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1afe00 con 0x563f6786aad0 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1af730 con 0x563f6786b420 2024-05-04T03:19:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a427c7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c1ac950 con 0x563f67875c40 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a427c7b80 10 client.0 _ll_register_callbacks cb 0x563f67725a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f6787e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f6787e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39922/0 (socket says 172.21.15.39:39922) 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 -- 172.21.15.39:0/2723966057 learned_addr learned my addr 172.21.15.39:0/2723966057 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a32350700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f67875c40 0x563f6787b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a3134e700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f6787ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 -- 172.21.15.39:0/2723966057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f67875c40 msgr2=0x563f6787b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f67875c40 0x563f6787b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 -- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 msgr2=0x563f6787ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f6787ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 -- 172.21.15.39:0/2723966057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6787b810 con 0x563f6786aad0 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a32350700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f67875c40 0x563f6787b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a3134e700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f6787ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a31b4f700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f6787e0a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c003eb0 tx=0x7f6a1c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18e90fea76bde91 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a157fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.520+0000 7f6a157fa700 1 -- 172.21.15.39:0/2723966057 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c031040 con 0x563f6786aad0 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a157fa700 1 -- 172.21.15.39:0/2723966057 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a1c02c360 con 0x563f6786aad0 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2723966057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a1c02c5f0 con 0x563f6786aad0 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a157fa700 1 -- 172.21.15.39:0/2723966057 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c005030 con 0x563f6786aad0 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 10 client.16338 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2723966057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f678f10f0 con 0x563f6786aad0 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a157fa700 1 -- 172.21.15.39:0/2723966057 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a1c02c5f0 con 0x563f6786aad0 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a157fa700 1 -- 172.21.15.39:0/2723966057 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6a1c01d410 con 0x563f6786aad0 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 10 client.16338 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 10 client.16338 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 20 client.16338 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f678f1850 0x563f678f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 4 client.16338 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5b1a1bbc", "release": true, "await": true}] 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.521+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2723966057 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5b1a1bbc", "release": true, "await": true}) -- 0x563f677954c0 con 0x563f678f1850 2024-05-04T03:19:51.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.522+0000 7f6a3134e700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f678f1850 0x563f678f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.522+0000 7f6a3134e700 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f678f1850 0x563f678f3c70 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f856aecc2484fd7c server_cookie=a867fb9563cef3a6 in_seq=0 out_seq=0 2024-05-04T03:19:51.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.522+0000 7f6a157fa700 10 client.16338 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a157fa700 1 -- 172.21.15.39:0/2723966057 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563f677954c0 con 0x563f678f1850 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a157fa700 10 client.16338 handle_command_reply: tid=0 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 client.16338 shutdown 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2723966057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 msgr2=0x563f6787e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f6787e0a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c003eb0 tx=0x7f6a1c003f90 comp rx=0 tx=0).stop 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2723966057 shutdown_connections 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f678f1850 0x563f678f3c70 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f6786aad0 0x563f6787e0a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f67875c40 0x563f6787b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 --2- 172.21.15.39:0/2723966057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f6786b420 0x563f6787ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a157fa700 0 client.16338 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2723966057 >> 172.21.15.39:0/2723966057 conn(0x563f676edf40 msgr2=0x563f6786e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2723966057 shutdown_connections 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 1 -- 172.21.15.39:0/2723966057 wait complete. 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.527+0000 7f6a427c7b80 20 client.16338 trim_cache size 0 max 16384 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 24, 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:19:51.530 INFO:teuthology.orchestra.run.smithi039.stdout: "5b1a1bbc": { 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 24, 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 83.0, 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:19:51.531 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:51.532 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:51.532 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:51.532 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:51.532 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:19:51.532 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:19:51.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 -- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a180b2880 msgr2=0x7f6a180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a180b2880 0x7f6a180b4d90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009c60 tx=0x7f6a1c009260 comp rx=0 tx=0).stop 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 -- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c108750 msgr2=0x7f6a2c19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c108750 0x7f6a2c19c630 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6a24007ee0 tx=0x7f6a2400d3b0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 -- 172.21.15.39:0/875642463 shutdown_connections 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a180b2880 0x7f6a180b4d90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a2c108750 0x7f6a2c19c630 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a2c1034d0 0x7f6a2c19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 --2- 172.21.15.39:0/875642463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a2c107e00 0x7f6a2c19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 -- 172.21.15.39:0/875642463 >> 172.21.15.39:0/875642463 conn(0x7f6a2c0fef20 msgr2=0x7f6a2c1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 -- 172.21.15.39:0/875642463 shutdown_connections 2024-05-04T03:19:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:19:51.528+0000 7f6a33db3700 1 -- 172.21.15.39:0/875642463 wait complete. 2024-05-04T03:19:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 360.0 s, 66.9 tps, lat 8727.017 ms stddev 13271.179 2024-05-04T03:19:52.216 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5b1a1bbc', seconds elapsed: 35.8 2024-05-04T03:19:52.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:52 smithi039 ceph-mon[29715]: pgmap v658: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 171 KiB/s wr, 33 op/s 2024-05-04T03:19:52.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:52 smithi066 ceph-mon[33264]: pgmap v658: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 171 KiB/s wr, 33 op/s 2024-05-04T03:19:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:52 smithi167 ceph-mon[33614]: pgmap v658: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 171 KiB/s wr, 33 op/s 2024-05-04T03:19:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.517+0000 7efdb1d5e640 1 -- 172.21.15.39:0/3205441432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac102b50 msgr2=0x7efdac103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.517+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/3205441432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac102b50 0x7efdac103850 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02a3f0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.517+0000 7efdb1d5e640 1 -- 172.21.15.39:0/3205441432 shutdown_connections 2024-05-04T03:19:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.517+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/3205441432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdac108a90 0x7efdac10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.517+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/3205441432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac103d90 0x7efdac1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.517+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/3205441432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac102b50 0x7efdac103850 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.517+0000 7efdb1d5e640 1 -- 172.21.15.39:0/3205441432 >> 172.21.15.39:0/3205441432 conn(0x7efdac0fdd20 msgr2=0x7efdac1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.517+0000 7efdb1d5e640 1 -- 172.21.15.39:0/3205441432 shutdown_connections 2024-05-04T03:19:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.518+0000 7efdb1d5e640 1 -- 172.21.15.39:0/3205441432 wait complete. 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.518+0000 7efdb1d5e640 1 Processor -- start 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdb1d5e640 1 -- start start 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdb1d5e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac102b50 0x7efdac1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdb1d5e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdac103d90 0x7efdac1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdb1d5e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac108a90 0x7efdac1a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdb1d5e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10d3d0 con 0x7efdac102b50 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdb1d5e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10d250 con 0x7efdac103d90 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdb1d5e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdac10d550 con 0x7efdac108a90 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efda3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdac103d90 0x7efdac1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efda3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdac103d90 0x7efdac1a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:50414/0 (socket says 172.21.15.39:50414) 2024-05-04T03:19:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efda3fff640 1 -- 172.21.15.39:0/2010914063 learned_addr learned my addr 172.21.15.39:0/2010914063 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdab7fe640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac102b50 0x7efdac1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.519+0000 7efdabfff640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac108a90 0x7efdac1a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.520+0000 7efda3fff640 1 -- 172.21.15.39:0/2010914063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac108a90 msgr2=0x7efdac1a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.520+0000 7efda3fff640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac108a90 0x7efdac1a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.520+0000 7efda3fff640 1 -- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac102b50 msgr2=0x7efdac1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.520+0000 7efda3fff640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac102b50 0x7efdac1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.520+0000 7efda3fff640 1 -- 172.21.15.39:0/2010914063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac1a83e0 con 0x7efdac103d90 2024-05-04T03:19:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.520+0000 7efdab7fe640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac102b50 0x7efdac1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.520+0000 7efdabfff640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac108a90 0x7efdac1a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:54.523 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.521+0000 7efda3fff640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdac103d90 0x7efdac1a0de0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e9f0 tx=0x7efd9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.521+0000 7efda97fa640 1 -- 172.21.15.39:0/2010914063 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9400cd80 con 0x7efdac103d90 2024-05-04T03:19:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.521+0000 7efdb1d5e640 1 -- 172.21.15.39:0/2010914063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac1a86d0 con 0x7efdac103d90 2024-05-04T03:19:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.521+0000 7efda97fa640 1 -- 172.21.15.39:0/2010914063 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd940108c0 con 0x7efdac103d90 2024-05-04T03:19:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.522+0000 7efda97fa640 1 -- 172.21.15.39:0/2010914063 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940187c0 con 0x7efdac103d90 2024-05-04T03:19:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.522+0000 7efdb1d5e640 1 -- 172.21.15.39:0/2010914063 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac1a8c70 con 0x7efdac103d90 2024-05-04T03:19:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.525+0000 7efdb8c15740 1 Processor -- start 2024-05-04T03:19:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.525+0000 7efdb8c15740 1 -- start start 2024-05-04T03:19:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba576e2f0 0x55eba5b38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b3fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdb8c15740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eba5a9d5d0 con 0x55eba576e2f0 2024-05-04T03:19:54.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdb8c15740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eba5a56890 con 0x55eba57971b0 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdb8c15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eba5a99590 con 0x55eba57764d0 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdabfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba576e2f0 0x55eba5b38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdabfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b38a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:50418/0 (socket says 172.21.15.39:50418) 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba576e2f0 0x55eba5b38fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39952/0 (socket says 172.21.15.39:39952) 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdab7fe640 1 -- 172.21.15.39:0/3368675095 learned_addr learned my addr 172.21.15.39:0/3368675095 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efda97fa640 1 -- 172.21.15.39:0/2010914063 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efd94010430 con 0x7efdac103d90 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.526+0000 7efdab7fe640 1 -- 172.21.15.39:0/3368675095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 msgr2=0x55eba5b3fee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.527+0000 7efdab7fe640 1 --2- 172.21.15.39:0/3368675095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b3fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.527+0000 7efdab7fe640 1 -- 172.21.15.39:0/3368675095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 msgr2=0x55eba5b38a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.527+0000 7efdab7fe640 1 --2- 172.21.15.39:0/3368675095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b38a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.527+0000 7efdab7fe640 1 -- 172.21.15.39:0/3368675095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba57af210 con 0x55eba576e2f0 2024-05-04T03:19:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.527+0000 7efdab7fe640 1 --2- 172.21.15.39:0/3368675095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba576e2f0 0x55eba5b38fe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006010 tx=0x7efd9c031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18a00e8705e0002 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.527+0000 7efda1ffb640 1 -- 172.21.15.39:0/3368675095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c035030 con 0x55eba576e2f0 2024-05-04T03:19:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.527+0000 7efda97fa640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd980b2860 0x7efd980b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.528+0000 7efda1ffb640 1 -- 172.21.15.39:0/3368675095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c004030 con 0x55eba576e2f0 2024-05-04T03:19:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.528+0000 7efda1ffb640 1 -- 172.21.15.39:0/3368675095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c003ac0 con 0x55eba576e2f0 2024-05-04T03:19:54.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.528+0000 7efdabfff640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd980b2860 0x7efd980b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.528+0000 7efda97fa640 1 -- 172.21.15.39:0/2010914063 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efd940e2580 con 0x7efdac103d90 2024-05-04T03:19:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 -- 172.21.15.39:0/3368675095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba576e2f0 msgr2=0x55eba5b38fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 --2- 172.21.15.39:0/3368675095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba576e2f0 0x55eba5b38fe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006010 tx=0x7efd9c031580 comp rx=0 tx=0).stop 2024-05-04T03:19:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdabfff640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd980b2860 0x7efd980b4d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55eba5b39c30 tx=0x7efda4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:54.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 -- 172.21.15.39:0/3368675095 shutdown_connections 2024-05-04T03:19:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 --2- 172.21.15.39:0/3368675095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b3fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 --2- 172.21.15.39:0/3368675095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba576e2f0 0x55eba5b38fe0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 --2- 172.21.15.39:0/3368675095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 -- 172.21.15.39:0/3368675095 >> 172.21.15.39:0/3368675095 conn(0x55eba578b300 msgr2=0x55eba5b343a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 -- 172.21.15.39:0/3368675095 shutdown_connections 2024-05-04T03:19:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.530+0000 7efdb8c15740 1 -- 172.21.15.39:0/3368675095 wait complete. 2024-05-04T03:19:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.531+0000 7efdb8c15740 1 -- start start 2024-05-04T03:19:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.531+0000 7efdb8c15740 1 Processor -- start 2024-05-04T03:19:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.531+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.531+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b42e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.531+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b49d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.531+0000 7efdb8c15740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eba5b408d0 con 0x55eba5755d80 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.531+0000 7efdb8c15740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eba5a5a370 con 0x55eba57971b0 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.531+0000 7efdb8c15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eba5b40a50 con 0x55eba57764d0 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdb8c15740 10 client.0 _ll_register_callbacks cb 0x55eba57aea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:39320/0 (socket says 172.21.15.39:39320) 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 -- 172.21.15.39:0/2038788022 learned_addr learned my addr 172.21.15.39:0/2038788022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdabfff640 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b49d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efda3fff640 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b42e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 -- 172.21.15.39:0/2038788022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 msgr2=0x55eba5b42e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b42e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 -- 172.21.15.39:0/2038788022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 msgr2=0x55eba5b49d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b49d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 -- 172.21.15.39:0/2038788022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba587b620 con 0x55eba57764d0 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.532+0000 7efdab7fe640 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b428c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c304cfde55c608b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.533+0000 7efd8affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.533+0000 7efd8affd640 1 -- 172.21.15.39:0/2038788022 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c004480 con 0x55eba57764d0 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.533+0000 7efd8affd640 1 -- 172.21.15.39:0/2038788022 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efd9c031c80 con 0x55eba57764d0 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.533+0000 7efdb8c15740 1 -- 172.21.15.39:0/2038788022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c004030 con 0x55eba57764d0 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.533+0000 7efd8affd640 1 -- 172.21.15.39:0/2038788022 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efd9c003840 con 0x55eba57764d0 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.533+0000 7efdb8c15740 10 client.25957 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.533+0000 7efdb8c15740 1 -- 172.21.15.39:0/2038788022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eba5b408d0 con 0x55eba57764d0 2024-05-04T03:19:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efd8affd640 1 -- 172.21.15.39:0/2038788022 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c004030 con 0x55eba57764d0 2024-05-04T03:19:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efd8affd640 1 -- 172.21.15.39:0/2038788022 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7efd9c03a040 con 0x55eba57764d0 2024-05-04T03:19:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efdb8c15740 10 client.25957 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efdb8c15740 10 client.25957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efdb8c15740 20 client.25957 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eba572b1c0 0x55eba5b4fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efdb8c15740 4 client.25957 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efdb8c15740 1 -- 172.21.15.39:0/2038788022 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eba5aab1e0 con 0x55eba572b1c0 2024-05-04T03:19:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.534+0000 7efda3fff640 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eba572b1c0 0x55eba5b4fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.535+0000 7efda3fff640 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eba572b1c0 0x55eba5b4fe20 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cab42eb8cc08bb1 server_cookie=47298be2549864eb in_seq=0 out_seq=0 2024-05-04T03:19:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.535+0000 7efd8affd640 10 client.25957 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.536+0000 7efd8affd640 1 -- 172.21.15.39:0/2038788022 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eba5aab1e0 con 0x55eba572b1c0 2024-05-04T03:19:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.536+0000 7efd8affd640 10 client.25957 handle_command_reply: tid=0 2024-05-04T03:19:54.538 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.536+0000 7efdb8c15740 1 client.25957 shutdown 2024-05-04T03:19:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 -- 172.21.15.39:0/2038788022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 msgr2=0x55eba5b428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b428c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c004410 comp rx=0 tx=0).stop 2024-05-04T03:19:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 -- 172.21.15.39:0/2038788022 shutdown_connections 2024-05-04T03:19:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eba572b1c0 0x55eba5b4fe20 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b49d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b42e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2038788022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b428c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efd8affd640 0 client.25957 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 -- 172.21.15.39:0/2038788022 >> 172.21.15.39:0/2038788022 conn(0x55eba578b300 msgr2=0x55eba5b37300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.537+0000 7efdb8c15740 1 -- 172.21.15.39:0/2038788022 shutdown_connections 2024-05-04T03:19:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.538+0000 7efdb8c15740 1 -- 172.21.15.39:0/2038788022 wait complete. 2024-05-04T03:19:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.538+0000 7efdb8c15740 20 client.25957 trim_cache size 0 max 16384 2024-05-04T03:19:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.565+0000 7efdb8c15740 1 Processor -- start 2024-05-04T03:19:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.565+0000 7efdb8c15740 1 -- start start 2024-05-04T03:19:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.565+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b55540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.565+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b55a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.565+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.565+0000 7efdb8c15740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1b18c0 con 0x55eba5755d80 2024-05-04T03:19:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.565+0000 7efdb8c15740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1b3400 con 0x55eba57971b0 2024-05-04T03:19:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.565+0000 7efdb8c15740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1b1f90 con 0x55eba57764d0 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b55540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efda3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b55a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdabfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efda3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b55a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:39326/0 (socket says 172.21.15.39:39326) 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efda3fff640 1 -- 172.21.15.39:0/2328787995 learned_addr learned my addr 172.21.15.39:0/2328787995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdab7fe640 1 -- 172.21.15.39:0/2328787995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 msgr2=0x55eba5b55a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdab7fe640 1 --2- 172.21.15.39:0/2328787995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b55a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdab7fe640 1 -- 172.21.15.39:0/2328787995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 msgr2=0x55eba5b371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdab7fe640 1 --2- 172.21.15.39:0/2328787995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b371f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdab7fe640 1 -- 172.21.15.39:0/2328787995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba5b396f0 con 0x55eba5755d80 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdabfff640 1 --2- 172.21.15.39:0/2328787995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efdab7fe640 1 --2- 172.21.15.39:0/2328787995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b55540 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd9c03c950 tx=0x7efd9c003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6886fb7749de98e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.566+0000 7efda17fa640 1 -- 172.21.15.39:0/2328787995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c05d070 con 0x55eba5755d80 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.567+0000 7efda17fa640 1 -- 172.21.15.39:0/2328787995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c003e70 con 0x55eba5755d80 2024-05-04T03:19:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.567+0000 7efda17fa640 1 -- 172.21.15.39:0/2328787995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c035070 con 0x55eba5755d80 2024-05-04T03:19:54.570 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:54 smithi066 ceph-mon[33264]: pgmap v659: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 315 KiB/s wr, 50 op/s 2024-05-04T03:19:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.568+0000 7efdb8c15740 1 -- 172.21.15.39:0/2328787995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 msgr2=0x55eba5b55540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.568+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2328787995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b55540 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd9c03c950 tx=0x7efd9c003660 comp rx=0 tx=0).stop 2024-05-04T03:19:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.568+0000 7efdb8c15740 1 -- 172.21.15.39:0/2328787995 shutdown_connections 2024-05-04T03:19:54.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.568+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2328787995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57971b0 0x55eba5b371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.568+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2328787995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57764d0 0x55eba5b55a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.568+0000 7efdb8c15740 1 --2- 172.21.15.39:0/2328787995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b55540 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.568+0000 7efdb8c15740 1 -- 172.21.15.39:0/2328787995 >> 172.21.15.39:0/2328787995 conn(0x55eba578b300 msgr2=0x55eba5b40ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.568+0000 7efdb8c15740 1 -- 172.21.15.39:0/2328787995 shutdown_connections 2024-05-04T03:19:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.569+0000 7efdb8c15740 1 -- 172.21.15.39:0/2328787995 wait complete. 2024-05-04T03:19:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.569+0000 7efdb8c15740 1 -- start start 2024-05-04T03:19:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.569+0000 7efdb8c15740 1 Processor -- start 2024-05-04T03:19:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b48530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57764d0 0x55eba5b48a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdb8c15740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57971b0 0x55eba5bb60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b48530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdab7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b48530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39988/0 (socket says 172.21.15.39:39988) 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdab7fe640 1 -- 172.21.15.39:0/4191502825 learned_addr learned my addr 172.21.15.39:0/4191502825 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1b2d30 con 0x55eba5755d80 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1b2660 con 0x55eba57764d0 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdac1af8a0 con 0x55eba57971b0 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efda3fff640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57764d0 0x55eba5b48a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdabfff640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57971b0 0x55eba5bb60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.570+0000 7efdb8c15740 10 client.0 _ll_register_callbacks cb 0x7efda40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efdabfff640 1 -- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57764d0 msgr2=0x55eba5b48a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efdabfff640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57764d0 0x55eba5b48a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efdabfff640 1 -- 172.21.15.39:0/4191502825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 msgr2=0x55eba5b48530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efdabfff640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b48530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efdabfff640 1 -- 172.21.15.39:0/4191502825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba5bb6650 con 0x55eba57971b0 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efda3fff640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57764d0 0x55eba5b48a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efdab7fe640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b48530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efdabfff640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57971b0 0x55eba5bb60e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efda400fa00 tx=0x7efda4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b308de22ca3ee3f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.571+0000 7efd8affd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.572+0000 7efd8affd640 1 -- 172.21.15.39:0/4191502825 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda40043d0 con 0x55eba57971b0 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.572+0000 7efd8affd640 1 -- 172.21.15.39:0/4191502825 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efda4004570 con 0x55eba57971b0 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.572+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c003e70 con 0x55eba57971b0 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.572+0000 7efd8affd640 1 -- 172.21.15.39:0/4191502825 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efda4004770 con 0x55eba57971b0 2024-05-04T03:19:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.572+0000 7efdb8c15740 10 client.25963 fetch_fsmap learned FSMap version 32 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.572+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eba5bb79a0 con 0x55eba57971b0 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.572+0000 7efd8affd640 1 -- 172.21.15.39:0/4191502825 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd9c003e70 con 0x55eba57971b0 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.573+0000 7efd8affd640 1 -- 172.21.15.39:0/4191502825 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7efda401d410 con 0x55eba57971b0 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.573+0000 7efdb8c15740 10 client.25963 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.573+0000 7efdb8c15740 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.573+0000 7efdb8c15740 20 client.25963 populate_metadata read hostname 'smithi039' 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.573+0000 7efdb8c15740 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eba5bb8400 0x55eba5bba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.573+0000 7efdb8c15740 4 client.25963 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.573+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eba5aa5050 con 0x55eba5bb8400 2024-05-04T03:19:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.573+0000 7efdab7fe640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eba5bb8400 0x55eba5bba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.574+0000 7efd8affd640 10 client.25963 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.574+0000 7efdab7fe640 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eba5bb8400 0x55eba5bba840 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e98b8f25ecbeb69 server_cookie=b1f60aa8589611f1 in_seq=0 out_seq=0 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.577+0000 7efd8affd640 1 -- 172.21.15.39:0/4191502825 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eba5aa5050 con 0x55eba5bb8400 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.577+0000 7efd8affd640 10 client.25963 handle_command_reply: tid=0 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.577+0000 7efdb8c15740 1 client.25963 shutdown 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57971b0 msgr2=0x55eba5bb60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57971b0 0x55eba5bb60e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efda400fa00 tx=0x7efda4004ab0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 shutdown_connections 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eba5bb8400 0x55eba5bba840 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eba57971b0 0x55eba5bb60e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eba57764d0 0x55eba5b48a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efd8affd640 0 client.25963 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 --2- 172.21.15.39:0/4191502825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eba5755d80 0x55eba5b48530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 >> 172.21.15.39:0/4191502825 conn(0x55eba578b300 msgr2=0x55eba5b371f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 shutdown_connections 2024-05-04T03:19:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 1 -- 172.21.15.39:0/4191502825 wait complete. 2024-05-04T03:19:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.578+0000 7efdb8c15740 20 client.25963 trim_cache size 0 max 16384 2024-05-04T03:19:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:19:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:19:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:19:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.579+0000 7efdb1d5e640 1 -- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd980b2860 msgr2=0x7efd980b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd980b2860 0x7efd980b4d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55eba5b39c30 tx=0x7efda4009500 comp rx=0 tx=0).stop 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 -- 172.21.15.39:0/2010914063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdac103d90 msgr2=0x7efdac1a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdac103d90 0x7efdac1a0de0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e9f0 tx=0x7efd9400eeb0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 -- 172.21.15.39:0/2010914063 shutdown_connections 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efd980b2860 0x7efd980b4d70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdac108a90 0x7efdac1a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdac103d90 0x7efdac1a0de0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 --2- 172.21.15.39:0/2010914063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac102b50 0x7efdac1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 -- 172.21.15.39:0/2010914063 >> 172.21.15.39:0/2010914063 conn(0x7efdac0fdd20 msgr2=0x7efdac104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.580+0000 7efdb1d5e640 1 -- 172.21.15.39:0/2010914063 shutdown_connections 2024-05-04T03:19:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:19:54.581+0000 7efdb1d5e640 1 -- 172.21.15.39:0/2010914063 wait complete. 2024-05-04T03:19:54.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:54 smithi167 ceph-mon[33614]: pgmap v659: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 315 KiB/s wr, 50 op/s 2024-05-04T03:19:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:54 smithi039 ceph-mon[29715]: pgmap v659: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 315 KiB/s wr, 50 op/s 2024-05-04T03:19:56.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:56 smithi066 ceph-mon[33264]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-05-04T03:19:56.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:56 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:19:56.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:56 smithi167 ceph-mon[33614]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-05-04T03:19:56.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:56 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:19:56.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:56 smithi039 ceph-mon[29715]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-05-04T03:19:56.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:56 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:19:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 365.0 s, 1350.3 tps, lat 73.896 ms stddev 20.545 2024-05-04T03:19:58.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:58 smithi167 ceph-mon[33614]: pgmap v661: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.9 MiB/s wr, 340 op/s 2024-05-04T03:19:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:58 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:58 smithi039 ceph-mon[29715]: pgmap v661: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.9 MiB/s wr, 340 op/s 2024-05-04T03:19:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:58 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:58 smithi066 ceph-mon[33264]: pgmap v661: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.9 MiB/s wr, 340 op/s 2024-05-04T03:19:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:58 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:59.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:19:59.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:19:59.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:19:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:59.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:19:59.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:19:59.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:19:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:19:59.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:19:59.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:19:59.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:19:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:20:00.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:00 smithi167 ceph-mon[33614]: pgmap v662: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 542 op/s 2024-05-04T03:20:00.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:00 smithi167 ceph-mon[33614]: overall HEALTH_OK 2024-05-04T03:20:00.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:00 smithi039 ceph-mon[29715]: pgmap v662: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 542 op/s 2024-05-04T03:20:00.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:00 smithi039 ceph-mon[29715]: overall HEALTH_OK 2024-05-04T03:20:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:00 smithi066 ceph-mon[33264]: pgmap v662: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.6 MiB/s wr, 542 op/s 2024-05-04T03:20:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:00 smithi066 ceph-mon[33264]: overall HEALTH_OK 2024-05-04T03:20:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 370.0 s, 1359.7 tps, lat 73.689 ms stddev 42.753 2024-05-04T03:20:02.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:02 smithi167 ceph-mon[33614]: pgmap v663: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 757 op/s 2024-05-04T03:20:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:02 smithi039 ceph-mon[29715]: pgmap v663: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 757 op/s 2024-05-04T03:20:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:02 smithi066 ceph-mon[33264]: pgmap v663: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 757 op/s 2024-05-04T03:20:04.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:04 smithi167 ceph-mon[33614]: pgmap v664: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.5 MiB/s wr, 1.08k op/s 2024-05-04T03:20:04.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:04 smithi039 ceph-mon[29715]: pgmap v664: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.5 MiB/s wr, 1.08k op/s 2024-05-04T03:20:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:04 smithi066 ceph-mon[33264]: pgmap v664: 97 pgs: 97 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.5 MiB/s wr, 1.08k op/s 2024-05-04T03:20:06.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:06 smithi167 ceph-mon[33614]: pgmap v665: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-04T03:20:06.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:06 smithi039 ceph-mon[29715]: pgmap v665: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-04T03:20:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 375.0 s, 1524.8 tps, lat 65.671 ms stddev 19.046 2024-05-04T03:20:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:06 smithi066 ceph-mon[33264]: pgmap v665: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.21k op/s 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.736+0000 7f64fbaa7640 1 -- 172.21.15.39:0/615894366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f4103d90 msgr2=0x7f64f41041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.736+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/615894366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f4103d90 0x7f64f41041a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f64e40098c0 tx=0x7f64e402c580 comp rx=0 tx=0).stop 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.736+0000 7f64fbaa7640 1 -- 172.21.15.39:0/615894366 shutdown_connections 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.736+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/615894366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f4108a90 0x7f64f410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.736+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/615894366 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f4103d90 0x7f64f41041a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.736+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/615894366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f4102b50 0x7f64f4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.737+0000 7f64fbaa7640 1 -- 172.21.15.39:0/615894366 >> 172.21.15.39:0/615894366 conn(0x7f64f40fdd40 msgr2=0x7f64f41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.737+0000 7f64fbaa7640 1 -- 172.21.15.39:0/615894366 shutdown_connections 2024-05-04T03:20:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.737+0000 7f64fbaa7640 1 -- 172.21.15.39:0/615894366 wait complete. 2024-05-04T03:20:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.739+0000 7f64fbaa7640 1 Processor -- start 2024-05-04T03:20:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.739+0000 7f64fbaa7640 1 -- start start 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64fbaa7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f4102b50 0x7f64f41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64fbaa7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f4103d90 0x7f64f41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64f9843640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f4102b50 0x7f64f41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64f9843640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f4102b50 0x7f64f41a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:40520/0 (socket says 172.21.15.39:40520) 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64f9042640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f4103d90 0x7f64f41a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64f9042640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f4103d90 0x7f64f41a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44728/0 (socket says 172.21.15.39:44728) 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64fbaa7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f4108a90 0x7f64f41a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64f9042640 1 -- 172.21.15.39:0/1128407933 learned_addr learned my addr 172.21.15.39:0/1128407933 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410d460 con 0x7f64f4103d90 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410d2e0 con 0x7f64f4108a90 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.740+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410d5e0 con 0x7f64f4102b50 2024-05-04T03:20:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.741+0000 7f64fa044640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f4108a90 0x7f64f41a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.741+0000 7f64f9042640 1 -- 172.21.15.39:0/1128407933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f4102b50 msgr2=0x7f64f41a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.741+0000 7f64f9042640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f4102b50 0x7f64f41a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.741+0000 7f64f9042640 1 -- 172.21.15.39:0/1128407933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f4108a90 msgr2=0x7f64f41a7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.741+0000 7f64f9042640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f4108a90 0x7f64f41a7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.741+0000 7f64f9042640 1 -- 172.21.15.39:0/1128407933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f41a81a0 con 0x7f64f4103d90 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.741+0000 7f64f9843640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f4102b50 0x7f64f41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.742+0000 7f64f9042640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f4103d90 0x7f64f41a0ba0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f64e402caa0 tx=0x7f64e4032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.742+0000 7f64eaffd640 1 -- 172.21.15.39:0/1128407933 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4041030 con 0x7f64f4103d90 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.742+0000 7f64eaffd640 1 -- 172.21.15.39:0/1128407933 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64e4005390 con 0x7f64f4103d90 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.742+0000 7f64eaffd640 1 -- 172.21.15.39:0/1128407933 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4033a40 con 0x7f64f4103d90 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.742+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f41a8430 con 0x7f64f4103d90 2024-05-04T03:20:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.742+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f41a8810 con 0x7f64f4103d90 2024-05-04T03:20:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.746+0000 7f6502a31740 1 Processor -- start 2024-05-04T03:20:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.747+0000 7f6502a31740 1 -- start start 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.747+0000 7f6502a31740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d701b0 0x556af2111a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.747+0000 7f6502a31740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d472f0 0x556af2111fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.747+0000 7f64eaffd640 1 -- 172.21.15.39:0/1128407933 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f64e403b040 con 0x7f64f4103d90 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.747+0000 7f6502a31740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d4f4d0 0x556af2118ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.747+0000 7f6502a31740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556af20765d0 con 0x556af1d4f4d0 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.747+0000 7f6502a31740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556af202f890 con 0x556af1d472f0 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.747+0000 7f6502a31740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556af2072590 con 0x556af1d701b0 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9843640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d472f0 0x556af2111fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9843640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d472f0 0x556af2111fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:58820/0 (socket says 172.21.15.39:58820) 2024-05-04T03:20:07.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9843640 1 -- 172.21.15.39:0/2872168565 learned_addr learned my addr 172.21.15.39:0/2872168565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64fa044640 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d701b0 0x556af2111a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9042640 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d4f4d0 0x556af2118ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9042640 1 -- 172.21.15.39:0/2872168565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d701b0 msgr2=0x556af2111a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9042640 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d701b0 0x556af2111a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9042640 1 -- 172.21.15.39:0/2872168565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d472f0 msgr2=0x556af2111fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9042640 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d472f0 0x556af2111fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.748+0000 7f64f9042640 1 -- 172.21.15.39:0/2872168565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af1d88210 con 0x556af1d4f4d0 2024-05-04T03:20:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.749+0000 7f64f9042640 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d4f4d0 0x556af2118ee0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f64e4100bd0 tx=0x7f64e410b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c7df368102bf7d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.749+0000 7f64f9843640 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d472f0 0x556af2111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.749+0000 7f64dffff640 1 -- 172.21.15.39:0/2872168565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e410b7e0 con 0x556af1d4f4d0 2024-05-04T03:20:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.749+0000 7f64eaffd640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f64e00b2860 0x7f64e00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.749+0000 7f64dffff640 1 -- 172.21.15.39:0/2872168565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64e4110030 con 0x556af1d4f4d0 2024-05-04T03:20:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.749+0000 7f64dffff640 1 -- 172.21.15.39:0/2872168565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e40ee710 con 0x556af1d4f4d0 2024-05-04T03:20:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.750+0000 7f64eaffd640 1 -- 172.21.15.39:0/1128407933 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f64e4103180 con 0x7f64f4103d90 2024-05-04T03:20:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.750+0000 7f64fa044640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f64e00b2860 0x7f64e00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 -- 172.21.15.39:0/2872168565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d4f4d0 msgr2=0x556af2118ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f64fa044640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f64e00b2860 0x7f64e00b4d70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x556af2112c30 tx=0x7f64ec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d4f4d0 0x556af2118ee0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f64e4100bd0 tx=0x7f64e410b3b0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 -- 172.21.15.39:0/2872168565 shutdown_connections 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d4f4d0 0x556af2118ee0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d472f0 0x556af2111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 --2- 172.21.15.39:0/2872168565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d701b0 0x556af2111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 -- 172.21.15.39:0/2872168565 >> 172.21.15.39:0/2872168565 conn(0x556af1d64300 msgr2=0x556af210d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 -- 172.21.15.39:0/2872168565 shutdown_connections 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.751+0000 7f6502a31740 1 -- 172.21.15.39:0/2872168565 wait complete. 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.752+0000 7f6502a31740 1 -- start start 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.752+0000 7f6502a31740 1 Processor -- start 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f6502a31740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 0x556af211b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f6502a31740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af211b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f6502a31740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af2122950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f6502a31740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556af21196f0 con 0x556af1d701b0 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f6502a31740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556af2033370 con 0x556af1d2ed80 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f6502a31740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556af2119870 con 0x556af1d472f0 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f6502a31740 10 client.0 _ll_register_callbacks cb 0x556af1d87a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64fa044640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af2122950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64f9843640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 0x556af211b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64fa044640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af2122950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:58826/0 (socket says 172.21.15.39:58826) 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64fa044640 1 -- 172.21.15.39:0/1430297088 learned_addr learned my addr 172.21.15.39:0/1430297088 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64f9042640 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af211b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64fa044640 1 -- 172.21.15.39:0/1430297088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 msgr2=0x556af211b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64fa044640 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 0x556af211b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64fa044640 1 -- 172.21.15.39:0/1430297088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 msgr2=0x556af211b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64fa044640 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af211b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64fa044640 1 -- 172.21.15.39:0/1430297088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af1e54620 con 0x556af1d2ed80 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.753+0000 7f64f9042640 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af211b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.754+0000 7f64fa044640 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af2122950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f64ec01f6a0 tx=0x7f64ec004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6af23fa4e0c0c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.754+0000 7f64dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.754+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1430297088 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec02c060 con 0x556af1d2ed80 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.754+0000 7f6502a31740 1 -- 172.21.15.39:0/1430297088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e4110030 con 0x556af1d2ed80 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.754+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1430297088 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f64ec017030 con 0x556af1d2ed80 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.754+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1430297088 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f64ec011810 con 0x556af1d2ed80 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.754+0000 7f6502a31740 10 client.25955 fetch_fsmap learned FSMap version 32 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.754+0000 7f6502a31740 1 -- 172.21.15.39:0/1430297088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556af2125950 con 0x556af1d2ed80 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1430297088 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64e4110030 con 0x556af1d2ed80 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1430297088 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f64ec027a60 con 0x556af1d2ed80 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f6502a31740 10 client.25955 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f6502a31740 10 client.25955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f6502a31740 20 client.25955 populate_metadata read hostname 'smithi039' 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f6502a31740 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556af1d041c0 0x556af2129070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f6502a31740 4 client.25955 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f6502a31740 1 -- 172.21.15.39:0/1430297088 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556af20841e0 con 0x556af1d041c0 2024-05-04T03:20:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.755+0000 7f64f9843640 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556af1d041c0 0x556af2129070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.757+0000 7f64dd7fa640 10 client.25955 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.757+0000 7f64f9843640 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556af1d041c0 0x556af2129070 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a7a48d61ac6dd7 server_cookie=4082584c58ddb2bc in_seq=0 out_seq=0 2024-05-04T03:20:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.757+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1430297088 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556af20841e0 con 0x556af1d041c0 2024-05-04T03:20:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.757+0000 7f64dd7fa640 10 client.25955 handle_command_reply: tid=0 2024-05-04T03:20:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 client.25955 shutdown 2024-05-04T03:20:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 -- 172.21.15.39:0/1430297088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 msgr2=0x556af2122950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af2122950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f64ec01f6a0 tx=0x7f64ec004ab0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 -- 172.21.15.39:0/1430297088 shutdown_connections 2024-05-04T03:20:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556af1d041c0 0x556af2129070 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f64dd7fa640 0 client.25955 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af2122950 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af211b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 --2- 172.21.15.39:0/1430297088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 0x556af211b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 -- 172.21.15.39:0/1430297088 >> 172.21.15.39:0/1430297088 conn(0x556af1d64300 msgr2=0x556af210c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 -- 172.21.15.39:0/1430297088 shutdown_connections 2024-05-04T03:20:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.758+0000 7f6502a31740 1 -- 172.21.15.39:0/1430297088 wait complete. 2024-05-04T03:20:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.759+0000 7f6502a31740 20 client.25955 trim_cache size 0 max 16384 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.788+0000 7f6502a31740 1 Processor -- start 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.788+0000 7f6502a31740 1 -- start start 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.788+0000 7f6502a31740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d2ed80 0x556af210d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.788+0000 7f6502a31740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d472f0 0x556af2122910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.788+0000 7f6502a31740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d701b0 0x556af2122e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64f9843640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d2ed80 0x556af210d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64f9843640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d2ed80 0x556af210d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:40540/0 (socket says 172.21.15.39:40540) 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64f9843640 1 -- 172.21.15.39:0/1985902519 learned_addr learned my addr 172.21.15.39:0/1985902519 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64f9042640 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d472f0 0x556af2122910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f6502a31740 1 -- 172.21.15.39:0/1985902519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41b1610 con 0x556af1d472f0 2024-05-04T03:20:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f6502a31740 1 -- 172.21.15.39:0/1985902519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41b3150 con 0x556af1d701b0 2024-05-04T03:20:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f6502a31740 1 -- 172.21.15.39:0/1985902519 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41b1ce0 con 0x556af1d2ed80 2024-05-04T03:20:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64fa044640 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d701b0 0x556af2122e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64fa044640 1 -- 172.21.15.39:0/1985902519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d2ed80 msgr2=0x556af210d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64fa044640 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d2ed80 0x556af210d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64fa044640 1 -- 172.21.15.39:0/1985902519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d472f0 msgr2=0x556af2122910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64fa044640 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d472f0 0x556af2122910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64fa044640 1 -- 172.21.15.39:0/1985902519 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556af210d940 con 0x556af1d701b0 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64f9843640 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d2ed80 0x556af210d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.789+0000 7f64f9042640 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d472f0 0x556af2122910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.790+0000 7f64fa044640 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d701b0 0x556af2122e50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009600 tx=0x7f64ec011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21eda76b6b5ecc07 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.790+0000 7f64df7fe640 1 -- 172.21.15.39:0/1985902519 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec037030 con 0x556af1d701b0 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.790+0000 7f64df7fe640 1 -- 172.21.15.39:0/1985902519 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64ec01c6a0 con 0x556af1d701b0 2024-05-04T03:20:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.790+0000 7f64df7fe640 1 -- 172.21.15.39:0/1985902519 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0273e0 con 0x556af1d701b0 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.791+0000 7f6502a31740 1 -- 172.21.15.39:0/1985902519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d701b0 msgr2=0x556af2122e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.791+0000 7f6502a31740 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d701b0 0x556af2122e50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009600 tx=0x7f64ec011440 comp rx=0 tx=0).stop 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.792+0000 7f6502a31740 1 -- 172.21.15.39:0/1985902519 shutdown_connections 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.792+0000 7f6502a31740 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d701b0 0x556af2122e50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.792+0000 7f6502a31740 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d472f0 0x556af2122910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.792+0000 7f6502a31740 1 --2- 172.21.15.39:0/1985902519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d2ed80 0x556af210d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.792+0000 7f6502a31740 1 -- 172.21.15.39:0/1985902519 >> 172.21.15.39:0/1985902519 conn(0x556af1d64300 msgr2=0x556af210ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.792+0000 7f6502a31740 1 -- 172.21.15.39:0/1985902519 shutdown_connections 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.792+0000 7f6502a31740 1 -- 172.21.15.39:0/1985902519 wait complete. 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.792+0000 7f6502a31740 1 -- start start 2024-05-04T03:20:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.793+0000 7f6502a31740 1 Processor -- start 2024-05-04T03:20:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.793+0000 7f6502a31740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af211e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.793+0000 7f6502a31740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 0x556af211eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.793+0000 7f6502a31740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af2121d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.793+0000 7f6502a31740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41b2a80 con 0x556af1d701b0 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.793+0000 7f6502a31740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41b23b0 con 0x556af1d2ed80 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.793+0000 7f6502a31740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41af5f0 con 0x556af1d472f0 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f6502a31740 10 client.0 _ll_register_callbacks cb 0x7f64ec009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64f9042640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 0x556af211eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64fa044640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af2121d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64fa044640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af2121d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44758/0 (socket says 172.21.15.39:44758) 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64fa044640 1 -- 172.21.15.39:0/1919966151 learned_addr learned my addr 172.21.15.39:0/1919966151 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64fa044640 1 -- 172.21.15.39:0/1919966151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 msgr2=0x556af211eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64fa044640 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 0x556af211eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64fa044640 1 -- 172.21.15.39:0/1919966151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 msgr2=0x556af211e960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64fa044640 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af211e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.794+0000 7f64fa044640 1 -- 172.21.15.39:0/1919966151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556af2122350 con 0x556af1d701b0 2024-05-04T03:20:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.795+0000 7f64fa044640 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af2121d90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f64ec01c4b0 tx=0x7f64ec01f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6329219eb6547c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.795+0000 7f64dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:20:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.795+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1919966151 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec02c070 con 0x556af1d701b0 2024-05-04T03:20:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.795+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1919966151 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f64ec004db0 con 0x556af1d701b0 2024-05-04T03:20:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.796+0000 7f6502a31740 1 -- 172.21.15.39:0/1919966151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ec01c6a0 con 0x556af1d701b0 2024-05-04T03:20:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.796+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1919966151 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f64ec037070 con 0x556af1d701b0 2024-05-04T03:20:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.796+0000 7f6502a31740 10 client.16371 fetch_fsmap learned FSMap version 32 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.796+0000 7f6502a31740 1 -- 172.21.15.39:0/1919966151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556af2191470 con 0x556af1d701b0 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.796+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1919966151 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64ec01c6a0 con 0x556af1d701b0 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.796+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1919966151 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f64ec005410 con 0x556af1d701b0 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.797+0000 7f6502a31740 10 client.16371 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.797+0000 7f6502a31740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.797+0000 7f6502a31740 20 client.16371 populate_metadata read hostname 'smithi039' 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.797+0000 7f6502a31740 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556af2191cd0 0x556af21940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.797+0000 7f6502a31740 4 client.16371 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.797+0000 7f6502a31740 1 -- 172.21.15.39:0/1919966151 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556af207e050 con 0x556af2191cd0 2024-05-04T03:20:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.797+0000 7f64f9843640 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556af2191cd0 0x556af21940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.798+0000 7f64dd7fa640 10 client.16371 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.798+0000 7f64f9843640 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556af2191cd0 0x556af21940f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e9aa4347abbbc50 server_cookie=f09ce6dfd78325b2 in_seq=0 out_seq=0 2024-05-04T03:20:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.807+0000 7f64dd7fa640 1 -- 172.21.15.39:0/1919966151 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556af207e050 con 0x556af2191cd0 2024-05-04T03:20:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.807+0000 7f64dd7fa640 10 client.16371 handle_command_reply: tid=0 2024-05-04T03:20:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.807+0000 7f6502a31740 1 client.16371 shutdown 2024-05-04T03:20:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 -- 172.21.15.39:0/1919966151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 msgr2=0x556af2121d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af2121d90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f64ec01c4b0 tx=0x7f64ec01f5b0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 -- 172.21.15.39:0/1919966151 shutdown_connections 2024-05-04T03:20:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556af2191cd0 0x556af21940f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556af1d701b0 0x556af2121d90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556af1d472f0 0x556af211eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 --2- 172.21.15.39:0/1919966151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556af1d2ed80 0x556af211e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 -- 172.21.15.39:0/1919966151 >> 172.21.15.39:0/1919966151 conn(0x556af1d64300 msgr2=0x556af210b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f64dd7fa640 0 client.16371 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 -- 172.21.15.39:0/1919966151 shutdown_connections 2024-05-04T03:20:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 1 -- 172.21.15.39:0/1919966151 wait complete. 2024-05-04T03:20:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.808+0000 7f6502a31740 20 client.16371 trim_cache size 0 max 16384 2024-05-04T03:20:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:20:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:20:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:20:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:20:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.809+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f64e00b2860 msgr2=0x7f64e00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.809+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f64e00b2860 0x7f64e00b4d70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x556af2112c30 tx=0x7f64ec009500 comp rx=0 tx=0).stop 2024-05-04T03:20:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.809+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f4103d90 msgr2=0x7f64f41a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.809+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f4103d90 0x7f64f41a0ba0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f64e402caa0 tx=0x7f64e4032ec0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.810+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 shutdown_connections 2024-05-04T03:20:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.810+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f64e00b2860 0x7f64e00b4d70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.810+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64f4108a90 0x7f64f41a7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.810+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f4103d90 0x7f64f41a0ba0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.810+0000 7f64fbaa7640 1 --2- 172.21.15.39:0/1128407933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64f4102b50 0x7f64f41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.810+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 >> 172.21.15.39:0/1128407933 conn(0x7f64f40fdd40 msgr2=0x7f64f41001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.810+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 shutdown_connections 2024-05-04T03:20:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:07.810+0000 7f64fbaa7640 1 -- 172.21.15.39:0/1128407933 wait complete. 2024-05-04T03:20:08.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:08 smithi167 ceph-mon[33614]: pgmap v666: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:20:08.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:08 smithi039 ceph-mon[29715]: pgmap v666: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:20:08.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:08 smithi066 ceph-mon[33264]: pgmap v666: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:20:10.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:10 smithi167 ceph-mon[33614]: pgmap v667: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 1.48k op/s 2024-05-04T03:20:10.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:10 smithi039 ceph-mon[29715]: pgmap v667: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 1.48k op/s 2024-05-04T03:20:10.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:10 smithi066 ceph-mon[33264]: pgmap v667: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 1.48k op/s 2024-05-04T03:20:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 380.0 s, 1418.2 tps, lat 70.314 ms stddev 19.779 2024-05-04T03:20:12.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:12 smithi167 ceph-mon[33614]: pgmap v668: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:20:12.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:12 smithi039 ceph-mon[29715]: pgmap v668: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:20:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:12 smithi066 ceph-mon[33264]: pgmap v668: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:20:14.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:14 smithi167 ceph-mon[33614]: pgmap v669: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 1.59k op/s 2024-05-04T03:20:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:14 smithi039 ceph-mon[29715]: pgmap v669: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 1.59k op/s 2024-05-04T03:20:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:14 smithi066 ceph-mon[33264]: pgmap v669: 97 pgs: 97 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 1.59k op/s 2024-05-04T03:20:16.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:16 smithi167 ceph-mon[33614]: pgmap v670: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:20:16.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:16 smithi039 ceph-mon[29715]: pgmap v670: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:20:16.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 385.0 s, 1453.6 tps, lat 68.894 ms stddev 18.095 2024-05-04T03:20:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:16 smithi066 ceph-mon[33264]: pgmap v670: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:20:18.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:18 smithi167 ceph-mon[33614]: pgmap v671: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:18.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:18 smithi039 ceph-mon[29715]: pgmap v671: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:18 smithi066 ceph-mon[33264]: pgmap v671: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:20.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:20 smithi167 ceph-mon[33614]: pgmap v672: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:20.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:20 smithi039 ceph-mon[29715]: pgmap v672: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:20.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:20 smithi066 ceph-mon[33264]: pgmap v672: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.967+0000 7f1fa3184640 1 -- 172.21.15.39:0/3437459633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c105280 msgr2=0x7f1f9c105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.967+0000 7f1fa3184640 1 --2- 172.21.15.39:0/3437459633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c105280 0x7f1f9c105690 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009ca0 tx=0x7f1f9402a8c0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.968+0000 7f1fa3184640 1 -- 172.21.15.39:0/3437459633 shutdown_connections 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.968+0000 7f1fa3184640 1 --2- 172.21.15.39:0/3437459633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c102430 0x7f1f9c10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.968+0000 7f1fa3184640 1 --2- 172.21.15.39:0/3437459633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c101930 0x7f1f9c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.968+0000 7f1fa3184640 1 --2- 172.21.15.39:0/3437459633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c105280 0x7f1f9c105690 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.968+0000 7f1fa3184640 1 -- 172.21.15.39:0/3437459633 >> 172.21.15.39:0/3437459633 conn(0x7f1f9c0fd530 msgr2=0x7f1f9c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.968+0000 7f1fa3184640 1 -- 172.21.15.39:0/3437459633 shutdown_connections 2024-05-04T03:20:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.968+0000 7f1fa3184640 1 -- 172.21.15.39:0/3437459633 wait complete. 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.969+0000 7f1fa3184640 1 Processor -- start 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.969+0000 7f1fa3184640 1 -- start start 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.969+0000 7f1fa3184640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c101930 0x7f1f9c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.969+0000 7f1fa3184640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c102430 0x7f1f9c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.969+0000 7f1fa3184640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c105280 0x7f1f9c1950c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.969+0000 7f1fa3184640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10f660 con 0x7f1f9c105280 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.969+0000 7f1fa3184640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10f4e0 con 0x7f1f9c101930 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.969+0000 7f1fa3184640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10f7e0 con 0x7f1f9c102430 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c102430 0x7f1f9c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c102430 0x7f1f9c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:60244/0 (socket says 172.21.15.39:60244) 2024-05-04T03:20:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1f9bfff640 1 -- 172.21.15.39:0/2606459164 learned_addr learned my addr 172.21.15.39:0/2606459164 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c101930 0x7f1f9c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1fa1721640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c105280 0x7f1f9c1950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1fa0f20640 1 -- 172.21.15.39:0/2606459164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c102430 msgr2=0x7f1f9c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c102430 0x7f1f9c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1fa0f20640 1 -- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c105280 msgr2=0x7f1f9c1950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c105280 0x7f1f9c1950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1fa0f20640 1 -- 172.21.15.39:0/2606459164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c1958e0 con 0x7f1f9c101930 2024-05-04T03:20:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.970+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c101930 0x7f1f9c198570 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402ade0 tx=0x7f1f940093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.971+0000 7f1f99ffb640 1 -- 172.21.15.39:0/2606459164 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94003cf0 con 0x7f1f9c101930 2024-05-04T03:20:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.971+0000 7f1f99ffb640 1 -- 172.21.15.39:0/2606459164 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f940044d0 con 0x7f1f9c101930 2024-05-04T03:20:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.971+0000 7f1f99ffb640 1 -- 172.21.15.39:0/2606459164 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94030be0 con 0x7f1f9c101930 2024-05-04T03:20:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.971+0000 7f1fa3184640 1 -- 172.21.15.39:0/2606459164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c195b70 con 0x7f1f9c101930 2024-05-04T03:20:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.971+0000 7f1fa3184640 1 -- 172.21.15.39:0/2606459164 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1a8ba0 con 0x7f1f9c101930 2024-05-04T03:20:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1faa036740 1 Processor -- start 2024-05-04T03:20:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1faa036740 1 -- start start 2024-05-04T03:20:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1f99ffb640 1 -- 172.21.15.39:0/2606459164 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1f94004040 con 0x7f1f9c101930 2024-05-04T03:20:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1faa036740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c77a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1faa036740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448ad2f0 0x55e544c77fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1faa036740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c7eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1faa036740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e544bdc5d0 con 0x55e5448d61b0 2024-05-04T03:20:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1faa036740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e544b95890 con 0x55e5448b54d0 2024-05-04T03:20:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.975+0000 7f1faa036740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e544bd8590 con 0x55e5448ad2f0 2024-05-04T03:20:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1fa0f20640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448ad2f0 0x55e544c77fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1fa1721640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c77a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c7eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1fa1721640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c77a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53242/0 (socket says 172.21.15.39:53242) 2024-05-04T03:20:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c7eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:49552/0 (socket says 172.21.15.39:49552) 2024-05-04T03:20:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1fa1721640 1 -- 172.21.15.39:0/2806018220 learned_addr learned my addr 172.21.15.39:0/2806018220 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1f99ffb640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b2860 0x7f1f880b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1f9bfff640 1 -- 172.21.15.39:0/2806018220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448ad2f0 msgr2=0x55e544c77fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1f9bfff640 1 --2- 172.21.15.39:0/2806018220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448ad2f0 0x55e544c77fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1f9bfff640 1 -- 172.21.15.39:0/2806018220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 msgr2=0x55e544c77a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1f9bfff640 1 --2- 172.21.15.39:0/2806018220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c77a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1f9bfff640 1 -- 172.21.15.39:0/2806018220 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5448ee210 con 0x55e5448b54d0 2024-05-04T03:20:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1fa1721640 1 --2- 172.21.15.39:0/2806018220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c77a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.976+0000 7f1fa1721640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b2860 0x7f1f880b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.977+0000 7f1f99ffb640 1 -- 172.21.15.39:0/2606459164 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1f94103750 con 0x7f1f9c101930 2024-05-04T03:20:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.977+0000 7f1f9bfff640 1 --2- 172.21.15.39:0/2806018220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c7eee0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009c80 tx=0x7f1f8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885f90dcd2c30caa server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.977+0000 7f1f92ffd640 1 -- 172.21.15.39:0/2806018220 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c004230 con 0x55e5448b54d0 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.977+0000 7f1f92ffd640 1 -- 172.21.15.39:0/2806018220 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f8c0043d0 con 0x55e5448b54d0 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.977+0000 7f1f92ffd640 1 -- 172.21.15.39:0/2806018220 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c004ca0 con 0x55e5448b54d0 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.977+0000 7f1fa1721640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b2860 0x7f1f880b4d70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1f84004650 tx=0x7f1f840046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 -- 172.21.15.39:0/2806018220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 msgr2=0x55e544c7eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 --2- 172.21.15.39:0/2806018220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c7eee0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009c80 tx=0x7f1f8c011810 comp rx=0 tx=0).stop 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 -- 172.21.15.39:0/2806018220 shutdown_connections 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 --2- 172.21.15.39:0/2806018220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c7eee0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 --2- 172.21.15.39:0/2806018220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448ad2f0 0x55e544c77fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 --2- 172.21.15.39:0/2806018220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c77a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 -- 172.21.15.39:0/2806018220 >> 172.21.15.39:0/2806018220 conn(0x55e5448ca300 msgr2=0x55e544c733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 -- 172.21.15.39:0/2806018220 shutdown_connections 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 -- 172.21.15.39:0/2806018220 wait complete. 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 -- start start 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.979+0000 7f1faa036740 1 Processor -- start 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.980+0000 7f1faa036740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.980+0000 7f1faa036740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448b54d0 0x55e544c81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.980+0000 7f1faa036740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c88b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa0f20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa0f20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c816b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:49564/0 (socket says 172.21.15.39:49564) 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448b54d0 0x55e544c81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa1721640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c88b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa0f20640 1 -- 172.21.15.39:0/3596179773 learned_addr learned my addr 172.21.15.39:0/3596179773 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e544c7f930 con 0x55e5448b54d0 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55e544b99370 con 0x55e5448ad2f0 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e544c7fab0 con 0x55e5448d61b0 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa1721640 1 -- 172.21.15.39:0/3596179773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 msgr2=0x55e544c816b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa1721640 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c816b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa1721640 1 -- 172.21.15.39:0/3596179773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448b54d0 msgr2=0x55e544c81bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa1721640 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448b54d0 0x55e544c81bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1fa1721640 1 -- 172.21.15.39:0/3596179773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5449ba620 con 0x55e5448d61b0 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1f9bfff640 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448b54d0 0x55e544c81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1faa036740 10 client.0 _ll_register_callbacks cb 0x55e5448eda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.981+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8c0043d0 con 0x55e5448d61b0 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.982+0000 7f1fa1721640 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c88b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55e544c78c30 tx=0x7f1f84004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5030ea603614389a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.982+0000 7f1f7bfff640 10 client.25993 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:20:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.982+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3596179773 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8401e830 con 0x55e5448d61b0 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.982+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3596179773 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1f8401e9d0 con 0x55e5448d61b0 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.982+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3596179773 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f8401ebd0 con 0x55e5448d61b0 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.982+0000 7f1faa036740 10 client.25993 fetch_fsmap learned FSMap version 32 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.982+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55e544c8ba60 con 0x55e5448d61b0 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.983+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3596179773 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1f84028420 con 0x55e5448d61b0 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.983+0000 7f1faa036740 10 client.25993 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.983+0000 7f1faa036740 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.983+0000 7f1faa036740 20 client.25993 populate_metadata read hostname 'smithi039' 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.983+0000 7f1faa036740 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e544894d80 0x55e544c8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.983+0000 7f1faa036740 4 client.25993 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.983+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e544bea1e0 con 0x55e544894d80 2024-05-04T03:20:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.983+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e544894d80 0x55e544c8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.984+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e544894d80 0x55e544c8f5f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0bf8ea3fd5fd97c server_cookie=33ff5cf23461ae3c in_seq=0 out_seq=0 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.984+0000 7f1f7bfff640 10 client.25993 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3596179773 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e544bea1e0 con 0x55e544894d80 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1f7bfff640 10 client.25993 handle_command_reply: tid=0 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 client.25993 shutdown 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 msgr2=0x55e544c88b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c88b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55e544c78c30 tx=0x7f1f84004730 comp rx=0 tx=0).stop 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 shutdown_connections 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e544894d80 0x55e544c8f5f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c88b50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448b54d0 0x55e544c81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 --2- 172.21.15.39:0/3596179773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c816b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1f7bfff640 0 client.25993 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 >> 172.21.15.39:0/3596179773 conn(0x55e5448ca300 msgr2=0x55e544c723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 shutdown_connections 2024-05-04T03:20:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.985+0000 7f1faa036740 1 -- 172.21.15.39:0/3596179773 wait complete. 2024-05-04T03:20:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:20.986+0000 7f1faa036740 20 client.25993 trim_cache size 0 max 16384 2024-05-04T03:20:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.012+0000 7f1faa036740 1 Processor -- start 2024-05-04T03:20:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.012+0000 7f1faa036740 1 -- start start 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.012+0000 7f1faa036740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c72cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.012+0000 7f1faa036740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448b54d0 0x55e544c73230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1faa036740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c88a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1faa036740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b18e0 con 0x55e5448d61b0 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1faa036740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b3420 con 0x55e5448ad2f0 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1faa036740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b1fb0 con 0x55e5448b54d0 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa0f20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c72cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa0f20640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c72cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:49570/0 (socket says 172.21.15.39:49570) 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa0f20640 1 -- 172.21.15.39:0/2059955002 learned_addr learned my addr 172.21.15.39:0/2059955002 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448b54d0 0x55e544c73230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa1721640 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c88a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa1721640 1 -- 172.21.15.39:0/2059955002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448b54d0 msgr2=0x55e544c73230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa1721640 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448b54d0 0x55e544c73230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa1721640 1 -- 172.21.15.39:0/2059955002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 msgr2=0x55e544c72cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa1721640 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c72cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa1721640 1 -- 172.21.15.39:0/2059955002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e544c73d00 con 0x55e5448d61b0 2024-05-04T03:20:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.013+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c72cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.014+0000 7f1fa1721640 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c88a70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1f84000c00 tx=0x7f1f84021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e05a755727b150 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.014+0000 7f1f927fc640 1 -- 172.21.15.39:0/2059955002 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8401ed70 con 0x55e5448d61b0 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.014+0000 7f1f927fc640 1 -- 172.21.15.39:0/2059955002 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f8c0043d0 con 0x55e5448d61b0 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.014+0000 7f1f927fc640 1 -- 172.21.15.39:0/2059955002 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8401e630 con 0x55e5448d61b0 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 -- 172.21.15.39:0/2059955002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 msgr2=0x55e544c88a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c88a70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1f84000c00 tx=0x7f1f84021070 comp rx=0 tx=0).stop 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 -- 172.21.15.39:0/2059955002 shutdown_connections 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448d61b0 0x55e544c88a70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448b54d0 0x55e544c73230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 --2- 172.21.15.39:0/2059955002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448ad2f0 0x55e544c72cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 -- 172.21.15.39:0/2059955002 >> 172.21.15.39:0/2059955002 conn(0x55e5448ca300 msgr2=0x55e544c7bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 -- 172.21.15.39:0/2059955002 shutdown_connections 2024-05-04T03:20:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 -- 172.21.15.39:0/2059955002 wait complete. 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 -- start start 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.016+0000 7f1faa036740 1 Processor -- start 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1faa036740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448ad2f0 0x55e544c89070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1faa036740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c85bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1faa036740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c86130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1faa036740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b2d50 con 0x55e5448ad2f0 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1faa036740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b2680 con 0x55e5448b54d0 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1faa036740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1af8c0 con 0x55e5448d61b0 2024-05-04T03:20:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1faa036740 10 client.0 _ll_register_callbacks cb 0x7f1f840043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1fa0f20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448ad2f0 0x55e544c89070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1fa1721640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c86130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c85bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1fa0f20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448ad2f0 0x55e544c89070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53264/0 (socket says 172.21.15.39:53264) 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1fa1721640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c86130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:60272/0 (socket says 172.21.15.39:60272) 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.017+0000 7f1fa0f20640 1 -- 172.21.15.39:0/3183149891 learned_addr learned my addr 172.21.15.39:0/3183149891 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f9bfff640 1 -- 172.21.15.39:0/3183149891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 msgr2=0x55e544c86130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f9bfff640 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c86130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f9bfff640 1 -- 172.21.15.39:0/3183149891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448ad2f0 msgr2=0x55e544c89070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f9bfff640 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448ad2f0 0x55e544c89070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f9bfff640 1 -- 172.21.15.39:0/3183149891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e544c86770 con 0x55e5448b54d0 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1fa1721640 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c86130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448ad2f0 0x55e544c89070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f9bfff640 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c85bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c000c00 tx=0x7f1f8c005630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88847d29425a806d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:20:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3183149891 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c01e040 con 0x55e5448b54d0 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3183149891 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1f8c019b10 con 0x55e5448b54d0 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.018+0000 7f1faa036740 1 -- 172.21.15.39:0/3183149891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8c0043d0 con 0x55e5448b54d0 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.019+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3183149891 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f8c019d30 con 0x55e5448b54d0 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.019+0000 7f1faa036740 10 client.25976 fetch_fsmap learned FSMap version 32 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.019+0000 7f1faa036740 1 -- 172.21.15.39:0/3183149891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e544cf6a20 con 0x55e5448b54d0 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.019+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3183149891 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f8c01f040 con 0x55e5448b54d0 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.020+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3183149891 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1f8c024050 con 0x55e5448b54d0 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.020+0000 7f1faa036740 10 client.25976 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.020+0000 7f1faa036740 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.020+0000 7f1faa036740 20 client.25976 populate_metadata read hostname 'smithi039' 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.020+0000 7f1faa036740 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e54486a1c0 0x55e544cf91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.020+0000 7f1faa036740 4 client.25976 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.020+0000 7f1faa036740 1 -- 172.21.15.39:0/3183149891 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1f8c0043d0 con 0x55e54486a1c0 2024-05-04T03:20:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.020+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e54486a1c0 0x55e544cf91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.021+0000 7f1f7bfff640 10 client.25976 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.021+0000 7f1fa0f20640 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e54486a1c0 0x55e544cf91e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68628e4a9b86dbc server_cookie=f44076c3976ba87 in_seq=0 out_seq=0 2024-05-04T03:20:21.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.030+0000 7f1f7bfff640 1 -- 172.21.15.39:0/3183149891 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1f8c0043d0 con 0x55e54486a1c0 2024-05-04T03:20:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.030+0000 7f1f7bfff640 10 client.25976 handle_command_reply: tid=0 2024-05-04T03:20:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.030+0000 7f1faa036740 1 client.25976 shutdown 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.030+0000 7f1faa036740 1 -- 172.21.15.39:0/3183149891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 msgr2=0x55e544c85bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c85bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c000c00 tx=0x7f1f8c005630 comp rx=0 tx=0).stop 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 -- 172.21.15.39:0/3183149891 shutdown_connections 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55e54486a1c0 0x55e544cf91e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e5448d61b0 0x55e544c86130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e5448b54d0 0x55e544c85bf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1f7bfff640 0 client.25976 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 --2- 172.21.15.39:0/3183149891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e5448ad2f0 0x55e544c89070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 -- 172.21.15.39:0/3183149891 >> 172.21.15.39:0/3183149891 conn(0x55e5448ca300 msgr2=0x55e544c72cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 -- 172.21.15.39:0/3183149891 shutdown_connections 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 1 -- 172.21.15.39:0/3183149891 wait complete. 2024-05-04T03:20:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.031+0000 7f1faa036740 20 client.25976 trim_cache size 0 max 16384 2024-05-04T03:20:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:20:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:20:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:20:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 -- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b2860 msgr2=0x7f1f880b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b2860 0x7f1f880b4d70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1f84004650 tx=0x7f1f840046c0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 -- 172.21.15.39:0/2606459164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c101930 msgr2=0x7f1f9c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c101930 0x7f1f9c198570 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402ade0 tx=0x7f1f940093e0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 -- 172.21.15.39:0/2606459164 shutdown_connections 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b2860 0x7f1f880b4d70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c105280 0x7f1f9c1950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c102430 0x7f1f9c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 --2- 172.21.15.39:0/2606459164 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c101930 0x7f1f9c198570 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.032+0000 7f1fa3184640 1 -- 172.21.15.39:0/2606459164 >> 172.21.15.39:0/2606459164 conn(0x7f1f9c0fd530 msgr2=0x7f1f9c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.033+0000 7f1fa3184640 1 -- 172.21.15.39:0/2606459164 shutdown_connections 2024-05-04T03:20:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:21.033+0000 7f1fa3184640 1 -- 172.21.15.39:0/2606459164 wait complete. 2024-05-04T03:20:21.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 390.0 s, 1463.6 tps, lat 68.241 ms stddev 17.739 2024-05-04T03:20:22.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:22 smithi167 ceph-mon[33614]: pgmap v673: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:22.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:22 smithi039 ceph-mon[29715]: pgmap v673: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:22.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:22 smithi066 ceph-mon[33264]: pgmap v673: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:24.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:24 smithi167 ceph-mon[33614]: pgmap v674: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:20:24.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:24 smithi039 ceph-mon[29715]: pgmap v674: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:20:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:24 smithi066 ceph-mon[33264]: pgmap v674: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:20:26.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:26 smithi167 ceph-mon[33614]: pgmap v675: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-05-04T03:20:26.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:26 smithi039 ceph-mon[29715]: pgmap v675: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-05-04T03:20:26.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 395.0 s, 1505.4 tps, lat 66.428 ms stddev 16.738 2024-05-04T03:20:26.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:26 smithi066 ceph-mon[33264]: pgmap v675: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 MiB/s wr, 1.35k op/s 2024-05-04T03:20:28.653 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:28 smithi039 ceph-mon[29715]: pgmap v676: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:28 smithi167 ceph-mon[33614]: pgmap v676: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:28.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:28 smithi066 ceph-mon[33264]: pgmap v676: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-05-04T03:20:30.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:30 smithi167 ceph-mon[33614]: pgmap v677: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-05-04T03:20:30.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:30 smithi039 ceph-mon[29715]: pgmap v677: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-05-04T03:20:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:30 smithi066 ceph-mon[33264]: pgmap v677: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-05-04T03:20:31.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 400.0 s, 1509.6 tps, lat 66.239 ms stddev 17.583 2024-05-04T03:20:32.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:32 smithi167 ceph-mon[33614]: pgmap v678: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:20:32.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:32 smithi039 ceph-mon[29715]: pgmap v678: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:20:32.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:32 smithi066 ceph-mon[33264]: pgmap v678: 97 pgs: 97 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/2714722386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc4104cc0 msgr2=0x7f0cc41050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/2714722386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc4104cc0 0x7f0cc41050d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009900 tx=0x7f0cb002a410 comp rx=0 tx=0).stop 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/2714722386 shutdown_connections 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/2714722386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41099c0 0x7f0cc410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/2714722386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc4104cc0 0x7f0cc41050d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/2714722386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cc4103a80 0x7f0cc4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/2714722386 >> 172.21.15.39:0/2714722386 conn(0x7f0cc40fec70 msgr2=0x7f0cc4101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/2714722386 shutdown_connections 2024-05-04T03:20:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.188+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/2714722386 wait complete. 2024-05-04T03:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.189+0000 7f0cc9e9f640 1 Processor -- start 2024-05-04T03:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.189+0000 7f0cc9e9f640 1 -- start start 2024-05-04T03:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc9e9f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cc4103a80 0x7f0cc4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc9e9f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41099c0 0x7f0cc41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc9e9f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc4198f20 0x7f0cc419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc9e9f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410f430 con 0x7f0cc4198f20 2024-05-04T03:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc9e9f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410f2b0 con 0x7f0cc4103a80 2024-05-04T03:20:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc9e9f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410f5b0 con 0x7f0cc41099c0 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc8e9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cc4103a80 0x7f0cc4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc8e9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cc4103a80 0x7f0cc4197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:34772/0 (socket says 172.21.15.39:34772) 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc8e9d640 1 -- 172.21.15.39:0/3982212306 learned_addr learned my addr 172.21.15.39:0/3982212306 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.190+0000 7f0cc969e640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc4198f20 0x7f0cc419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc8e9d640 1 -- 172.21.15.39:0/3982212306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41099c0 msgr2=0x7f0cc41984f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41099c0 0x7f0cc41984f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc8e9d640 1 -- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc4198f20 msgr2=0x7f0cc419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc4198f20 0x7f0cc419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc8e9d640 1 -- 172.21.15.39:0/3982212306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc419ba70 con 0x7f0cc4103a80 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cc4103a80 0x7f0cc4197fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800e980 tx=0x7f0cb800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc1ffb640 1 -- 172.21.15.39:0/3982212306 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb800cd80 con 0x7f0cc4103a80 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc1ffb640 1 -- 172.21.15.39:0/3982212306 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cb80108c0 con 0x7f0cc4103a80 2024-05-04T03:20:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/3982212306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc419bd60 con 0x7f0cc4103a80 2024-05-04T03:20:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/3982212306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc419c2b0 con 0x7f0cc4103a80 2024-05-04T03:20:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.191+0000 7f0cc1ffb640 1 -- 172.21.15.39:0/3982212306 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb80188b0 con 0x7f0cc4103a80 2024-05-04T03:20:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.195+0000 7f0cd1fb0740 1 Processor -- start 2024-05-04T03:20:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.195+0000 7f0cc1ffb640 1 -- 172.21.15.39:0/3982212306 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0cb801f020 con 0x7f0cc4103a80 2024-05-04T03:20:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.195+0000 7f0cd1fb0740 1 -- start start 2024-05-04T03:20:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.195+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4331b0 0x564bef7d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4124d0 0x564bef7dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564bef7395d0 con 0x564bef4124d0 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564bef6f2890 con 0x564bef40a2f0 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564bef735590 con 0x564bef4331b0 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cc1ffb640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0cb40b27e0 0x7f0cb40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cc8e9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cc969e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4331b0 0x564bef7d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cc8e9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7d51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:34782/0 (socket says 172.21.15.39:34782) 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cc8e9d640 1 -- 172.21.15.39:0/1986142937 learned_addr learned my addr 172.21.15.39:0/1986142937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cc969e640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0cb40b27e0 0x7f0cb40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cc1ffb640 1 -- 172.21.15.39:0/3982212306 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0cb80e2430 con 0x7f0cc4103a80 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.196+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/1986142937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4124d0 0x564bef7dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.197+0000 7f0cc3fff640 1 -- 172.21.15.39:0/1986142937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4331b0 msgr2=0x564bef7d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.197+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/1986142937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4331b0 0x564bef7d4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.197+0000 7f0cc3fff640 1 -- 172.21.15.39:0/1986142937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 msgr2=0x564bef7d51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.197+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/1986142937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7d51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.197+0000 7f0cc969e640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0cb40b27e0 0x7f0cb40b4cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005fd0 tx=0x7f0cbc009d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.197+0000 7f0cc3fff640 1 -- 172.21.15.39:0/1986142937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bef44b210 con 0x564bef4124d0 2024-05-04T03:20:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.198+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/1986142937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4124d0 0x564bef7dc0b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0008000 tx=0x7f0cb0031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddae9f867049b44 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.198+0000 7f0caeffd640 1 -- 172.21.15.39:0/1986142937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb003f070 con 0x564bef4124d0 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.198+0000 7f0caeffd640 1 -- 172.21.15.39:0/1986142937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cb0031870 con 0x564bef4124d0 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.198+0000 7f0caeffd640 1 -- 172.21.15.39:0/1986142937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0035070 con 0x564bef4124d0 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.199+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/1986142937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4124d0 msgr2=0x564bef7dc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.199+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/1986142937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4124d0 0x564bef7dc0b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0008000 tx=0x7f0cb0031500 comp rx=0 tx=0).stop 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.199+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/1986142937 shutdown_connections 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.200+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/1986142937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4124d0 0x564bef7dc0b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.200+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/1986142937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7d51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.200+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/1986142937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4331b0 0x564bef7d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.200+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/1986142937 >> 172.21.15.39:0/1986142937 conn(0x564bef427300 msgr2=0x564bef7d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.200+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/1986142937 shutdown_connections 2024-05-04T03:20:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.200+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/1986142937 wait complete. 2024-05-04T03:20:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.201+0000 7f0cd1fb0740 1 -- start start 2024-05-04T03:20:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.201+0000 7f0cd1fb0740 1 Processor -- start 2024-05-04T03:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4124d0 0x564bef7ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4331b0 0x564bef7e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564bef7dcaa0 con 0x564bef4331b0 2024-05-04T03:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564bef6f6370 con 0x564bef40a2f0 2024-05-04T03:20:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564bef7dcc20 con 0x564bef4124d0 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cc969e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4331b0 0x564bef7e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cc969e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4331b0 0x564bef7e5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51636/0 (socket says 172.21.15.39:51636) 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cc969e640 1 -- 172.21.15.39:0/3276235493 learned_addr learned my addr 172.21.15.39:0/3276235493 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cd1fb0740 10 client.0 _ll_register_callbacks cb 0x564bef44aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.202+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4124d0 0x564bef7ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.203+0000 7f0cc969e640 1 -- 172.21.15.39:0/3276235493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4124d0 msgr2=0x564bef7ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.203+0000 7f0cc969e640 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4124d0 0x564bef7ded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.203+0000 7f0cc969e640 1 -- 172.21.15.39:0/3276235493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 msgr2=0x564bef7de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.203+0000 7f0cc969e640 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.203+0000 7f0cc969e640 1 -- 172.21.15.39:0/3276235493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bef517620 con 0x564bef4331b0 2024-05-04T03:20:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.203+0000 7f0cc969e640 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4331b0 0x564bef7e5cf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc010ad0 tx=0x7f0cbc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f8892bb0098703 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.203+0000 7f0ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.204+0000 7f0ca3fff640 1 -- 172.21.15.39:0/3276235493 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc02a060 con 0x564bef4331b0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.204+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/3276235493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb0031870 con 0x564bef4331b0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.204+0000 7f0ca3fff640 1 -- 172.21.15.39:0/3276235493 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0cbc019070 con 0x564bef4331b0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.204+0000 7f0ca3fff640 1 -- 172.21.15.39:0/3276235493 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cbc012ae0 con 0x564bef4331b0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.204+0000 7f0cd1fb0740 10 client.16401 fetch_fsmap learned FSMap version 32 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.204+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/3276235493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564bef7e8ce0 con 0x564bef4331b0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0ca3fff640 1 -- 172.21.15.39:0/3276235493 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cbc012c80 con 0x564bef4331b0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0ca3fff640 1 -- 172.21.15.39:0/3276235493 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0cbc024070 con 0x564bef4331b0 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0cd1fb0740 10 client.16401 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0cd1fb0740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0cd1fb0740 20 client.16401 populate_metadata read hostname 'smithi039' 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564bef3f1d80 0x564bef7ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0cd1fb0740 4 client.16401 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/3276235493 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0cb0031870 con 0x564bef3f1d80 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.205+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564bef3f1d80 0x564bef7ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.206+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564bef3f1d80 0x564bef7ec3c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba262662d4ae8dc0 server_cookie=2fbf9e685b73f7fc in_seq=0 out_seq=0 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.206+0000 7f0ca3fff640 10 client.16401 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.207+0000 7f0ca3fff640 1 -- 172.21.15.39:0/3276235493 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f0cb0031870 con 0x564bef3f1d80 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.207+0000 7f0ca3fff640 10 client.16401 handle_command_reply: tid=0 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.207+0000 7f0cd1fb0740 1 client.16401 shutdown 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.207+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/3276235493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4331b0 msgr2=0x564bef7e5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.207+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4331b0 0x564bef7e5cf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc010ad0 tx=0x7f0cbc018000 comp rx=0 tx=0).stop 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.207+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/3276235493 shutdown_connections 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.207+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564bef3f1d80 0x564bef7ec3c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.208+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef4331b0 0x564bef7e5cf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.208+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef4124d0 0x564bef7ded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.208+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/3276235493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef40a2f0 0x564bef7de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.208+0000 7f0ca3fff640 0 client.16401 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.208+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/3276235493 >> 172.21.15.39:0/3276235493 conn(0x564bef427300 msgr2=0x564bef7cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.208+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/3276235493 shutdown_connections 2024-05-04T03:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.208+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/3276235493 wait complete. 2024-05-04T03:20:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.208+0000 7f0cd1fb0740 20 client.16401 trim_cache size 0 max 16384 2024-05-04T03:20:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.242+0000 7f0cd1fb0740 1 Processor -- start 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cd1fb0740 1 -- start start 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef427300 0x564bef7d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef3f1d80 0x564bef7e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef40a2f0 0x564bef7e6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc41ba520 con 0x564bef40a2f0 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc41bc060 con 0x564bef3f1d80 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cd1fb0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc41babf0 con 0x564bef427300 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef3f1d80 0x564bef7e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef3f1d80 0x564bef7e5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:34792/0 (socket says 172.21.15.39:34792) 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cc3fff640 1 -- 172.21.15.39:0/2165205888 learned_addr learned my addr 172.21.15.39:0/2165205888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cc969e640 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef40a2f0 0x564bef7e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.243+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef427300 0x564bef7d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cc3fff640 1 -- 172.21.15.39:0/2165205888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef427300 msgr2=0x564bef7d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef427300 0x564bef7d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cc3fff640 1 -- 172.21.15.39:0/2165205888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef40a2f0 msgr2=0x564bef7e6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef40a2f0 0x564bef7e6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cc3fff640 1 -- 172.21.15.39:0/2165205888 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bef7dc470 con 0x564bef3f1d80 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef3f1d80 0x564bef7e5c10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009850 tx=0x7f0cb0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c4e031c1a57a0e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cae7fc640 1 -- 172.21.15.39:0/2165205888 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb003f070 con 0x564bef3f1d80 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cae7fc640 1 -- 172.21.15.39:0/2165205888 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cb0048030 con 0x564bef3f1d80 2024-05-04T03:20:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.244+0000 7f0cae7fc640 1 -- 172.21.15.39:0/2165205888 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0035040 con 0x564bef3f1d80 2024-05-04T03:20:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.246+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2165205888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef3f1d80 msgr2=0x564bef7e5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.246+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef3f1d80 0x564bef7e5c10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009850 tx=0x7f0cb0000bc0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2165205888 shutdown_connections 2024-05-04T03:20:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef40a2f0 0x564bef7e6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef3f1d80 0x564bef7e5c10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2165205888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef427300 0x564bef7d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2165205888 >> 172.21.15.39:0/2165205888 conn(0x564bef7ec730 msgr2=0x564bef7cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2165205888 shutdown_connections 2024-05-04T03:20:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2165205888 wait complete. 2024-05-04T03:20:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 -- start start 2024-05-04T03:20:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.247+0000 7f0cd1fb0740 1 Processor -- start 2024-05-04T03:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef3f1d80 0x564bef7e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef40a2f0 0x564bef7e3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cd1fb0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef427300 0x564bef7e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef40a2f0 0x564bef7e3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef40a2f0 0x564bef7e3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:59952/0 (socket says 172.21.15.39:59952) 2024-05-04T03:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cc3fff640 1 -- 172.21.15.39:0/2819191204 learned_addr learned my addr 172.21.15.39:0/2819191204 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc41bb990 con 0x564bef3f1d80 2024-05-04T03:20:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc41bb2c0 con 0x564bef427300 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc41a15f0 con 0x564bef40a2f0 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef3f1d80 0x564bef7e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.248+0000 7f0cc969e640 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef427300 0x564bef7e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0cd1fb0740 10 client.0 _ll_register_callbacks cb 0x7f0cbc0107d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0cc3fff640 1 -- 172.21.15.39:0/2819191204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef427300 msgr2=0x564bef7e4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef427300 0x564bef7e4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0cc3fff640 1 -- 172.21.15.39:0/2819191204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef3f1d80 msgr2=0x564bef7e3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef3f1d80 0x564bef7e3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0cc3fff640 1 -- 172.21.15.39:0/2819191204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bef7e52c0 con 0x564bef40a2f0 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb0048030 con 0x564bef40a2f0 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0cc3fff640 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef40a2f0 0x564bef7e3dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00057c0 tx=0x7f0cb0005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f9d9a3a09a7a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0ca3fff640 10 client.26011 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:20:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0ca3fff640 1 -- 172.21.15.39:0/2819191204 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb005b050 con 0x564bef40a2f0 2024-05-04T03:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.249+0000 7f0ca3fff640 1 -- 172.21.15.39:0/2819191204 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0cb0031c70 con 0x564bef40a2f0 2024-05-04T03:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0ca3fff640 1 -- 172.21.15.39:0/2819191204 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cb003f070 con 0x564bef40a2f0 2024-05-04T03:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cd1fb0740 10 client.26011 fetch_fsmap learned FSMap version 32 2024-05-04T03:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=32}) -- 0x564bef7e63d0 con 0x564bef40a2f0 2024-05-04T03:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0ca3fff640 1 -- 172.21.15.39:0/2819191204 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0cb003b820 con 0x564bef40a2f0 2024-05-04T03:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cd1fb0740 10 client.26011 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cd1fb0740 10 client.26011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cd1fb0740 20 client.26011 populate_metadata read hostname 'smithi039' 2024-05-04T03:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564bef4124d0 0x564bef856760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cd1fb0740 4 client.26011 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bef7471e0 con 0x564bef4124d0 2024-05-04T03:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.250+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564bef4124d0 0x564bef856760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.251+0000 7f0cc8e9d640 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564bef4124d0 0x564bef856760 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2faa30d1de2e46e5 server_cookie=e95ae4a4774a068d in_seq=0 out_seq=0 2024-05-04T03:20:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.251+0000 7f0ca3fff640 10 client.26011 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.260+0000 7f0ca3fff640 1 -- 172.21.15.39:0/2819191204 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bef7471e0 con 0x564bef4124d0 2024-05-04T03:20:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.260+0000 7f0ca3fff640 10 client.26011 handle_command_reply: tid=0 2024-05-04T03:20:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.260+0000 7f0cd1fb0740 1 client.26011 shutdown 2024-05-04T03:20:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.260+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef40a2f0 msgr2=0x564bef7e3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.260+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef40a2f0 0x564bef7e3dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00057c0 tx=0x7f0cb0005e40 comp rx=0 tx=0).stop 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 shutdown_connections 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564bef4124d0 0x564bef856760 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564bef427300 0x564bef7e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564bef40a2f0 0x564bef7e3dd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0cd1fb0740 1 --2- 172.21.15.39:0/2819191204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bef3f1d80 0x564bef7e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 >> 172.21.15.39:0/2819191204 conn(0x564bef7ec730 msgr2=0x564bef7d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0ca3fff640 0 client.26011 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 shutdown_connections 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.261+0000 7f0cd1fb0740 1 -- 172.21.15.39:0/2819191204 wait complete. 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.262+0000 7f0cd1fb0740 20 client.26011 trim_cache size 0 max 16384 2024-05-04T03:20:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:20:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:20:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:20:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:20:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0cb40b27e0 msgr2=0x7f0cb40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0cb40b27e0 0x7f0cb40b4cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005fd0 tx=0x7f0cbc009d10 comp rx=0 tx=0).stop 2024-05-04T03:20:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/3982212306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cc4103a80 msgr2=0x7f0cc4197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cc4103a80 0x7f0cc4197fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800e980 tx=0x7f0cb800ee40 comp rx=0 tx=0).stop 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/3982212306 shutdown_connections 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0cb40b27e0 0x7f0cb40b4cf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc4198f20 0x7f0cc419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc41099c0 0x7f0cc41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 --2- 172.21.15.39:0/3982212306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0cc4103a80 0x7f0cc4197fb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/3982212306 >> 172.21.15.39:0/3982212306 conn(0x7f0cc40fec70 msgr2=0x7f0cc4101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.263+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/3982212306 shutdown_connections 2024-05-04T03:20:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:34.264+0000 7f0cc9e9f640 1 -- 172.21.15.39:0/3982212306 wait complete. 2024-05-04T03:20:34.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:34 smithi167 ceph-mon[33614]: pgmap v679: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-05-04T03:20:34.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:34 smithi039 ceph-mon[29715]: pgmap v679: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-05-04T03:20:34.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:34 smithi066 ceph-mon[33264]: pgmap v679: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.54k op/s 2024-05-04T03:20:36.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:36 smithi167 ceph-mon[33614]: pgmap v680: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:36.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:36 smithi039 ceph-mon[29715]: pgmap v680: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:36.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 405.0 s, 1452.2 tps, lat 68.846 ms stddev 18.259 2024-05-04T03:20:36.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:36 smithi066 ceph-mon[33264]: pgmap v680: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:38.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:38 smithi167 ceph-mon[33614]: pgmap v681: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:20:38.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:38 smithi039 ceph-mon[29715]: pgmap v681: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:20:38.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:38 smithi066 ceph-mon[33264]: pgmap v681: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.57k op/s 2024-05-04T03:20:40.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:40 smithi167 ceph-mon[33614]: pgmap v682: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-05-04T03:20:40.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:40 smithi039 ceph-mon[29715]: pgmap v682: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-05-04T03:20:40.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:40 smithi066 ceph-mon[33264]: pgmap v682: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 1.58k op/s 2024-05-04T03:20:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 410.0 s, 1367.0 tps, lat 73.034 ms stddev 17.683 2024-05-04T03:20:42.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:42 smithi167 ceph-mon[33614]: pgmap v683: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 1.45k op/s 2024-05-04T03:20:42.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:42 smithi039 ceph-mon[29715]: pgmap v683: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 1.45k op/s 2024-05-04T03:20:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:42 smithi066 ceph-mon[33264]: pgmap v683: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 1.45k op/s 2024-05-04T03:20:44.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:44 smithi167 ceph-mon[33614]: pgmap v684: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 12 MiB/s wr, 1.42k op/s 2024-05-04T03:20:44.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:44 smithi039 ceph-mon[29715]: pgmap v684: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 12 MiB/s wr, 1.42k op/s 2024-05-04T03:20:44.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:44 smithi066 ceph-mon[33264]: pgmap v684: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 12 MiB/s wr, 1.42k op/s 2024-05-04T03:20:46.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:46 smithi167 ceph-mon[33614]: pgmap v685: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:20:46.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:46 smithi039 ceph-mon[29715]: pgmap v685: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:20:46.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 415.0 s, 1383.3 tps, lat 72.308 ms stddev 18.003 2024-05-04T03:20:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:46 smithi066 ceph-mon[33264]: pgmap v685: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:20:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.433+0000 7f89ab1c7640 1 -- 172.21.15.39:0/3141473691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a4109bc0 msgr2=0x7f89a410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.433+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/3141473691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a4109bc0 0x7f89a410bfe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f899c009be0 tx=0x7f899c02a6c0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.433+0000 7f89ab1c7640 1 -- 172.21.15.39:0/3141473691 shutdown_connections 2024-05-04T03:20:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.434+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/3141473691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a4109bc0 0x7f89a410bfe0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.434+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/3141473691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89a4104ec0 0x7f89a41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.434+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/3141473691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89a4103c80 0x7f89a4104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.434+0000 7f89ab1c7640 1 -- 172.21.15.39:0/3141473691 >> 172.21.15.39:0/3141473691 conn(0x7f89a40fee70 msgr2=0x7f89a4101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.434+0000 7f89ab1c7640 1 -- 172.21.15.39:0/3141473691 shutdown_connections 2024-05-04T03:20:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.434+0000 7f89ab1c7640 1 -- 172.21.15.39:0/3141473691 wait complete. 2024-05-04T03:20:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.436+0000 7f89ab1c7640 1 Processor -- start 2024-05-04T03:20:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.436+0000 7f89ab1c7640 1 -- start start 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.436+0000 7f89ab1c7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89a4103c80 0x7f89a419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89ab1c7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89a4104ec0 0x7f89a419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89ab1c7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a4109bc0 0x7f89a41a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89ab1c7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89a410f640 con 0x7f89a4104ec0 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89ab1c7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89a410f4c0 con 0x7f89a4103c80 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89ab1c7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89a410f7c0 con 0x7f89a4109bc0 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a8f63640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89a4103c80 0x7f89a419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a9764640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a4109bc0 0x7f89a41a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a8f63640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89a4103c80 0x7f89a419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:55588/0 (socket says 172.21.15.39:55588) 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a8f63640 1 -- 172.21.15.39:0/1882915264 learned_addr learned my addr 172.21.15.39:0/1882915264 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a3fff640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89a4104ec0 0x7f89a419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a8f63640 1 -- 172.21.15.39:0/1882915264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a4109bc0 msgr2=0x7f89a41a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a8f63640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a4109bc0 0x7f89a41a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a8f63640 1 -- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89a4104ec0 msgr2=0x7f89a419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a8f63640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89a4104ec0 0x7f89a419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.437+0000 7f89a8f63640 1 -- 172.21.15.39:0/1882915264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a4076010 con 0x7f89a4103c80 2024-05-04T03:20:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.438+0000 7f89a8f63640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89a4103c80 0x7f89a419a300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f899800e980 tx=0x7f899800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.438+0000 7f89a1ffb640 1 -- 172.21.15.39:0/1882915264 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899800cd00 con 0x7f89a4103c80 2024-05-04T03:20:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.438+0000 7f89ab1c7640 1 -- 172.21.15.39:0/1882915264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a4076300 con 0x7f89a4103c80 2024-05-04T03:20:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.438+0000 7f89a1ffb640 1 -- 172.21.15.39:0/1882915264 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89980108c0 con 0x7f89a4103c80 2024-05-04T03:20:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.438+0000 7f89a1ffb640 1 -- 172.21.15.39:0/1882915264 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899800cd00 con 0x7f89a4103c80 2024-05-04T03:20:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.438+0000 7f89ab1c7640 1 -- 172.21.15.39:0/1882915264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a4076850 con 0x7f89a4103c80 2024-05-04T03:20:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.442+0000 7f89b207c740 1 Processor -- start 2024-05-04T03:20:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.442+0000 7f89b207c740 1 -- start start 2024-05-04T03:20:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.442+0000 7f89b207c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b3c1b0 0x5652d7edda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.442+0000 7f89b207c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 0x5652d7eddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89b207c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89b207c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5652d7e425d0 con 0x5652d7b132f0 2024-05-04T03:20:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89b207c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5652d7dfb890 con 0x5652d7b1b4d0 2024-05-04T03:20:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89b207c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5652d7e3e590 con 0x5652d7b3c1b0 2024-05-04T03:20:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b3c1b0 0x5652d7edda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b3c1b0 0x5652d7edda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:41780/0 (socket says 172.21.15.39:41780) 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 -- 172.21.15.39:0/701601431 learned_addr learned my addr 172.21.15.39:0/701601431 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a1ffb640 1 -- 172.21.15.39:0/1882915264 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f899801f020 con 0x7f89a4103c80 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 -- 172.21.15.39:0/701601431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 msgr2=0x5652d7ee4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a3fff640 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 -- 172.21.15.39:0/701601431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 msgr2=0x5652d7eddfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 0x5652d7eddfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 -- 172.21.15.39:0/701601431 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d7b54210 con 0x5652d7b3c1b0 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a3fff640 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:20:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.443+0000 7f89a9764640 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b3c1b0 0x5652d7edda60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f899c033000 tx=0x7f899c030fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ea983f0fb9eade server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.444+0000 7f898affd640 1 -- 172.21.15.39:0/701601431 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c041030 con 0x5652d7b3c1b0 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.444+0000 7f898affd640 1 -- 172.21.15.39:0/701601431 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f899c037040 con 0x5652d7b3c1b0 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.444+0000 7f898affd640 1 -- 172.21.15.39:0/701601431 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c036040 con 0x5652d7b3c1b0 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.444+0000 7f89a1ffb640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f898c0b2c30 0x7f898c0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.444+0000 7f89a1ffb640 1 -- 172.21.15.39:0/1882915264 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f899800cb00 con 0x7f89a4103c80 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.445+0000 7f89a8f63640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f898c0b2c30 0x7f898c0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.445+0000 7f89b207c740 1 -- 172.21.15.39:0/701601431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b3c1b0 msgr2=0x5652d7edda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.445+0000 7f89b207c740 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b3c1b0 0x5652d7edda60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f899c033000 tx=0x7f899c030fd0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.445+0000 7f89b207c740 1 -- 172.21.15.39:0/701601431 shutdown_connections 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.445+0000 7f89b207c740 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.445+0000 7f89b207c740 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 0x5652d7eddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.445+0000 7f89b207c740 1 --2- 172.21.15.39:0/701601431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b3c1b0 0x5652d7edda60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.445+0000 7f89b207c740 1 -- 172.21.15.39:0/701601431 >> 172.21.15.39:0/701601431 conn(0x5652d7b30300 msgr2=0x5652d7ed93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.446+0000 7f89b207c740 1 -- 172.21.15.39:0/701601431 shutdown_connections 2024-05-04T03:20:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.446+0000 7f89a8f63640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f898c0b2c30 0x7f898c0b5140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f89980a8fe0 tx=0x7f8998010c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.446+0000 7f89b207c740 1 -- 172.21.15.39:0/701601431 wait complete. 2024-05-04T03:20:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.446+0000 7f89b207c740 1 -- start start 2024-05-04T03:20:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.446+0000 7f89b207c740 1 Processor -- start 2024-05-04T03:20:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89b207c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b132f0 0x5652d7ee7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89b207c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89b207c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 0x5652d7eeeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89b207c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5652d7ee58d0 con 0x5652d7b3c1b0 2024-05-04T03:20:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89b207c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5652d7dff370 con 0x5652d7b1b4d0 2024-05-04T03:20:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89b207c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5652d7ee5a50 con 0x5652d7b132f0 2024-05-04T03:20:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89b207c740 10 client.0 _ll_register_callbacks cb 0x5652d7b53a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:55596/0 (socket says 172.21.15.39:55596) 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.447+0000 7f89a3fff640 1 -- 172.21.15.39:0/3795029143 learned_addr learned my addr 172.21.15.39:0/3795029143 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.448+0000 7f89a9764640 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b132f0 0x5652d7ee7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.448+0000 7f89a9764640 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 0x5652d7eeeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.448+0000 7f89a3fff640 1 -- 172.21.15.39:0/3795029143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b132f0 msgr2=0x5652d7ee7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.448+0000 7f89a3fff640 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b132f0 0x5652d7ee7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.448+0000 7f89a3fff640 1 -- 172.21.15.39:0/3795029143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 msgr2=0x5652d7eeeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.448+0000 7f89a3fff640 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 0x5652d7eeeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.448+0000 7f89a3fff640 1 -- 172.21.15.39:0/3795029143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d7c20620 con 0x5652d7b1b4d0 2024-05-04T03:20:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.448+0000 7f89a9764640 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 0x5652d7eeeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:20:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.449+0000 7f8983fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:20:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.449+0000 7f89a3fff640 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee7bc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8994012700 tx=0x7f8994012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae85381a9e406d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.450+0000 7f8983fff640 1 -- 172.21.15.39:0/3795029143 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899401f060 con 0x5652d7b1b4d0 2024-05-04T03:20:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.450+0000 7f8983fff640 1 -- 172.21.15.39:0/3795029143 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8994010dc0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.450+0000 7f89b207c740 1 -- 172.21.15.39:0/3795029143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f899c037040 con 0x5652d7b1b4d0 2024-05-04T03:20:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.450+0000 7f8983fff640 1 -- 172.21.15.39:0/3795029143 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f899401a430 con 0x5652d7b1b4d0 2024-05-04T03:20:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.450+0000 7f89b207c740 10 client.26003 fetch_fsmap learned FSMap version 32 2024-05-04T03:20:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.450+0000 7f89b207c740 1 -- 172.21.15.39:0/3795029143 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5652d7ef1880 con 0x5652d7b1b4d0 2024-05-04T03:20:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.451+0000 7f8983fff640 1 -- 172.21.15.39:0/3795029143 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f899c037040 con 0x5652d7b1b4d0 2024-05-04T03:20:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.451+0000 7f8983fff640 1 -- 172.21.15.39:0/3795029143 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f899400e4d0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f89b207c740 10 client.26003 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:20:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f89b207c740 10 client.26003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f89b207c740 20 client.26003 populate_metadata read hostname 'smithi039' 2024-05-04T03:20:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f89b207c740 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5652d7afad80 0x5652d7ef4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f89b207c740 4 client.26003 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f89b207c740 1 -- 172.21.15.39:0/3795029143 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652d7e501e0 con 0x5652d7afad80 2024-05-04T03:20:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f89a9764640 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5652d7afad80 0x5652d7ef4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f89a9764640 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5652d7afad80 0x5652d7ef4ff0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753495f8c9f38572 server_cookie=c58712a537992dcc in_seq=0 out_seq=0 2024-05-04T03:20:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.452+0000 7f8983fff640 10 client.26003 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.453+0000 7f8983fff640 1 -- 172.21.15.39:0/3795029143 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652d7e501e0 con 0x5652d7afad80 2024-05-04T03:20:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.453+0000 7f8983fff640 10 client.26003 handle_command_reply: tid=0 2024-05-04T03:20:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.453+0000 7f89b207c740 1 client.26003 shutdown 2024-05-04T03:20:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.453+0000 7f89b207c740 1 -- 172.21.15.39:0/3795029143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 msgr2=0x5652d7ee7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.453+0000 7f89b207c740 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee7bc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8994012700 tx=0x7f8994012bc0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f89b207c740 1 -- 172.21.15.39:0/3795029143 shutdown_connections 2024-05-04T03:20:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f89b207c740 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5652d7afad80 0x5652d7ef4ff0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f89b207c740 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 0x5652d7eeeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f89b207c740 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee7bc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f89b207c740 1 --2- 172.21.15.39:0/3795029143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b132f0 0x5652d7ee7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f89b207c740 1 -- 172.21.15.39:0/3795029143 >> 172.21.15.39:0/3795029143 conn(0x5652d7b30300 msgr2=0x5652d7ed83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f8983fff640 0 client.26003 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f89b207c740 1 -- 172.21.15.39:0/3795029143 shutdown_connections 2024-05-04T03:20:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.454+0000 7f89b207c740 1 -- 172.21.15.39:0/3795029143 wait complete. 2024-05-04T03:20:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.455+0000 7f89b207c740 20 client.26003 trim_cache size 0 max 16384 2024-05-04T03:20:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89b207c740 1 Processor -- start 2024-05-04T03:20:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89b207c740 1 -- start start 2024-05-04T03:20:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89b207c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 0x5652d7ee0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89b207c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7eeea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89b207c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b3c1b0 0x5652d7ed8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89b207c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89a41ad4f0 con 0x5652d7b132f0 2024-05-04T03:20:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89b207c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89a41af030 con 0x5652d7b3c1b0 2024-05-04T03:20:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89b207c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89a41adbc0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89a9764640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 0x5652d7ee0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89a9764640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 0x5652d7ee0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33064/0 (socket says 172.21.15.39:33064) 2024-05-04T03:20:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89a9764640 1 -- 172.21.15.39:0/577226351 learned_addr learned my addr 172.21.15.39:0/577226351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.487+0000 7f89a3fff640 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7eeea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.488+0000 7f89a9764640 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b3c1b0 0x5652d7ed8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.488+0000 7f89a3fff640 1 -- 172.21.15.39:0/577226351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b3c1b0 msgr2=0x5652d7ed8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.488+0000 7f89a3fff640 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b3c1b0 0x5652d7ed8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.488+0000 7f89a3fff640 1 -- 172.21.15.39:0/577226351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 msgr2=0x5652d7ee0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.488+0000 7f89a3fff640 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 0x5652d7ee0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.488+0000 7f89a3fff640 1 -- 172.21.15.39:0/577226351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d7edec60 con 0x5652d7b1b4d0 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.488+0000 7f89a3fff640 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7eeea40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8994012700 tx=0x7f8994009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd91a213120c47e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.488+0000 7f898a7fc640 1 -- 172.21.15.39:0/577226351 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994033070 con 0x5652d7b1b4d0 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.489+0000 7f898a7fc640 1 -- 172.21.15.39:0/577226351 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f899400ece0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.489+0000 7f898a7fc640 1 -- 172.21.15.39:0/577226351 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899402a040 con 0x5652d7b1b4d0 2024-05-04T03:20:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.491+0000 7f89b207c740 1 -- 172.21.15.39:0/577226351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 msgr2=0x5652d7eeea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.491+0000 7f89b207c740 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7eeea40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8994012700 tx=0x7f8994009d20 comp rx=0 tx=0).stop 2024-05-04T03:20:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.491+0000 7f89b207c740 1 -- 172.21.15.39:0/577226351 shutdown_connections 2024-05-04T03:20:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.491+0000 7f89b207c740 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b3c1b0 0x5652d7ed8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.491+0000 7f89b207c740 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7eeea40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.491+0000 7f89b207c740 1 --2- 172.21.15.39:0/577226351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b132f0 0x5652d7ee0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.491+0000 7f89b207c740 1 -- 172.21.15.39:0/577226351 >> 172.21.15.39:0/577226351 conn(0x5652d7b30300 msgr2=0x5652d7ee1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.492+0000 7f89b207c740 1 -- 172.21.15.39:0/577226351 shutdown_connections 2024-05-04T03:20:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.492+0000 7f89b207c740 1 -- 172.21.15.39:0/577226351 wait complete. 2024-05-04T03:20:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.492+0000 7f89b207c740 1 -- start start 2024-05-04T03:20:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.493+0000 7f89b207c740 1 Processor -- start 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89b207c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b132f0 0x5652d7ee9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89b207c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89a9764640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b132f0 0x5652d7ee9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89a9764640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b132f0 0x5652d7ee9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:55622/0 (socket says 172.21.15.39:55622) 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89a9764640 1 -- 172.21.15.39:0/3468529264 learned_addr learned my addr 172.21.15.39:0/3468529264 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89b207c740 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 0x5652d7f5b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89a41ae960 con 0x5652d7b3c1b0 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89a41ae290 con 0x5652d7b132f0 2024-05-04T03:20:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89a41ab4d0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89a3fff640 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.494+0000 7f89b207c740 10 client.0 _ll_register_callbacks cb 0x7f8994009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89a3fff640 1 -- 172.21.15.39:0/3468529264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b132f0 msgr2=0x5652d7ee9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89a3fff640 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b132f0 0x5652d7ee9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89a3fff640 1 -- 172.21.15.39:0/3468529264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 msgr2=0x5652d7f5b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89a3fff640 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 0x5652d7f5b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89a3fff640 1 -- 172.21.15.39:0/3468529264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d7eeeb20 con 0x5652d7b1b4d0 2024-05-04T03:20:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89a9764640 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b132f0 0x5652d7ee9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89a3fff640 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee9b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5652d7ee5200 tx=0x7f8994010ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a172151e0f8729 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f8983fff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f8983fff640 1 -- 172.21.15.39:0/3468529264 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899401f070 con 0x5652d7b1b4d0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f8983fff640 1 -- 172.21.15.39:0/3468529264 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8994005290 con 0x5652d7b1b4d0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f899400ece0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f8983fff640 1 -- 172.21.15.39:0/3468529264 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f899403f050 con 0x5652d7b1b4d0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89b207c740 10 client.26029 fetch_fsmap learned FSMap version 32 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.495+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5652d7f5c6d0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f8983fff640 1 -- 172.21.15.39:0/3468529264 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f899400ece0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f8983fff640 1 -- 172.21.15.39:0/3468529264 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f89940035f0 con 0x5652d7b1b4d0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f89b207c740 10 client.26029 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f89b207c740 10 client.26029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f89b207c740 20 client.26029 populate_metadata read hostname 'smithi039' 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f89b207c740 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5652d7ad01c0 0x5652d7f5f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f89b207c740 4 client.26029 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652d7e4a050 con 0x5652d7ad01c0 2024-05-04T03:20:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.496+0000 7f89a9764640 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5652d7ad01c0 0x5652d7f5f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.497+0000 7f8983fff640 10 client.26029 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.497+0000 7f89a9764640 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5652d7ad01c0 0x5652d7f5f020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea4ef2e02147e0a server_cookie=9ccfc9eea868187b in_seq=0 out_seq=0 2024-05-04T03:20:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f8983fff640 1 -- 172.21.15.39:0/3468529264 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652d7e4a050 con 0x5652d7ad01c0 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f8983fff640 10 client.26029 handle_command_reply: tid=0 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 client.26029 shutdown 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 msgr2=0x5652d7ee9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee9b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5652d7ee5200 tx=0x7f8994010ec0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 shutdown_connections 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5652d7ad01c0 0x5652d7f5f020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5652d7b3c1b0 0x5652d7f5b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652d7b1b4d0 0x5652d7ee9b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 --2- 172.21.15.39:0/3468529264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5652d7b132f0 0x5652d7ee9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 >> 172.21.15.39:0/3468529264 conn(0x5652d7b30300 msgr2=0x5652d7ed8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.505+0000 7f8983fff640 0 client.26029 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.506+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 shutdown_connections 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.506+0000 7f89b207c740 1 -- 172.21.15.39:0/3468529264 wait complete. 2024-05-04T03:20:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.506+0000 7f89b207c740 20 client.26029 trim_cache size 0 max 16384 2024-05-04T03:20:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:20:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:20:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:20:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 -- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f898c0b2c30 msgr2=0x7f898c0b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f898c0b2c30 0x7f898c0b5140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f89980a8fe0 tx=0x7f8998010c50 comp rx=0 tx=0).stop 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 -- 172.21.15.39:0/1882915264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89a4103c80 msgr2=0x7f89a419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89a4103c80 0x7f89a419a300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f899800e980 tx=0x7f899800ee40 comp rx=0 tx=0).stop 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 -- 172.21.15.39:0/1882915264 shutdown_connections 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f898c0b2c30 0x7f898c0b5140 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89a4109bc0 0x7f89a41a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89a4104ec0 0x7f89a419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 --2- 172.21.15.39:0/1882915264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89a4103c80 0x7f89a419a300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.507+0000 7f89ab1c7640 1 -- 172.21.15.39:0/1882915264 >> 172.21.15.39:0/1882915264 conn(0x7f89a40fee70 msgr2=0x7f89a4105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.508+0000 7f89ab1c7640 1 -- 172.21.15.39:0/1882915264 shutdown_connections 2024-05-04T03:20:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:20:47.508+0000 7f89ab1c7640 1 -- 172.21.15.39:0/1882915264 wait complete. 2024-05-04T03:20:48.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:48 smithi167 ceph-mon[33614]: pgmap v686: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-05-04T03:20:48.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:48 smithi039 ceph-mon[29715]: pgmap v686: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-05-04T03:20:48.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:48 smithi066 ceph-mon[33264]: pgmap v686: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-05-04T03:20:50.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:50 smithi167 ceph-mon[33614]: pgmap v687: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:50.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:50 smithi039 ceph-mon[29715]: pgmap v687: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:50.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:50 smithi066 ceph-mon[33264]: pgmap v687: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 1.47k op/s 2024-05-04T03:20:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 420.0 s, 1342.9 tps, lat 74.403 ms stddev 20.884 2024-05-04T03:20:52.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:52 smithi167 ceph-mon[33614]: pgmap v688: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-05-04T03:20:52.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:52 smithi039 ceph-mon[29715]: pgmap v688: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-05-04T03:20:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:52 smithi066 ceph-mon[33264]: pgmap v688: 97 pgs: 97 active+clean; 1.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-05-04T03:20:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:54 smithi039 ceph-mon[29715]: pgmap v689: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.42k op/s 2024-05-04T03:20:54.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:54 smithi066 ceph-mon[33264]: pgmap v689: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.42k op/s 2024-05-04T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:54 smithi167 ceph-mon[33614]: pgmap v689: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 1.42k op/s 2024-05-04T03:20:56.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:56 smithi039 ceph-mon[29715]: pgmap v690: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-05-04T03:20:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 425.0 s, 1372.8 tps, lat 72.974 ms stddev 21.582 2024-05-04T03:20:56.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:56 smithi066 ceph-mon[33264]: pgmap v690: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-05-04T03:20:56.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:56 smithi167 ceph-mon[33614]: pgmap v690: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 1.33k op/s 2024-05-04T03:20:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:58 smithi039 ceph-mon[29715]: pgmap v691: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:20:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:58 smithi066 ceph-mon[33264]: pgmap v691: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:20:58.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:58 smithi167 ceph-mon[33614]: pgmap v691: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:20:59.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:20:59 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:20:59.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:20:59 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:20:59.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:20:59 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:21:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.674+0000 7fc0661c7640 1 -- 172.21.15.39:0/2336617273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06010bdd0 msgr2=0x7fc06010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.674+0000 7fc0661c7640 1 --2- 172.21.15.39:0/2336617273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06010bdd0 0x7fc06010e1f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc058009be0 tx=0x7fc05802a6c0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.675+0000 7fc0661c7640 1 -- 172.21.15.39:0/2336617273 shutdown_connections 2024-05-04T03:21:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.675+0000 7fc0661c7640 1 --2- 172.21.15.39:0/2336617273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc06010bdd0 0x7fc06010e1f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.675+0000 7fc0661c7640 1 --2- 172.21.15.39:0/2336617273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0601070d0 0x7fc0601074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.675+0000 7fc0661c7640 1 --2- 172.21.15.39:0/2336617273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc060105e90 0x7fc060106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.675+0000 7fc0661c7640 1 -- 172.21.15.39:0/2336617273 >> 172.21.15.39:0/2336617273 conn(0x7fc060101060 msgr2=0x7fc060103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.675+0000 7fc0661c7640 1 -- 172.21.15.39:0/2336617273 shutdown_connections 2024-05-04T03:21:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.675+0000 7fc0661c7640 1 -- 172.21.15.39:0/2336617273 wait complete. 2024-05-04T03:21:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.676+0000 7fc0661c7640 1 Processor -- start 2024-05-04T03:21:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.676+0000 7fc0661c7640 1 -- start start 2024-05-04T03:21:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.676+0000 7fc0661c7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc060105e90 0x7fc0601a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.676+0000 7fc0661c7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0601070d0 0x7fc0601a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.676+0000 7fc0661c7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc06010bdd0 0x7fc0601a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.676+0000 7fc0661c7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc060111840 con 0x7fc06010bdd0 2024-05-04T03:21:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.676+0000 7fc0661c7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601116c0 con 0x7fc060105e90 2024-05-04T03:21:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.676+0000 7fc0661c7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601119c0 con 0x7fc0601070d0 2024-05-04T03:21:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc060105e90 0x7fc0601a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc060105e90 0x7fc0601a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:47592/0 (socket says 172.21.15.39:47592) 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 -- 172.21.15.39:0/1933667879 learned_addr learned my addr 172.21.15.39:0/1933667879 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05effd640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0601070d0 0x7fc0601a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05ffff640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc06010bdd0 0x7fc0601a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 -- 172.21.15.39:0/1933667879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0601070d0 msgr2=0x7fc0601a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0601070d0 0x7fc0601a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 -- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc06010bdd0 msgr2=0x7fc0601a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc06010bdd0 0x7fc0601a7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 -- 172.21.15.39:0/1933667879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0601a8310 con 0x7fc060105e90 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.677+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc060105e90 0x7fc0601a07d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc05000e980 tx=0x7fc05000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.678+0000 7fc05cff9640 1 -- 172.21.15.39:0/1933667879 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05000ccc0 con 0x7fc060105e90 2024-05-04T03:21:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.678+0000 7fc05cff9640 1 -- 172.21.15.39:0/1933667879 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0500108c0 con 0x7fc060105e90 2024-05-04T03:21:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.678+0000 7fc0661c7640 1 -- 172.21.15.39:0/1933667879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0601a8600 con 0x7fc060105e90 2024-05-04T03:21:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.678+0000 7fc0661c7640 1 -- 172.21.15.39:0/1933667879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0601a8ae0 con 0x7fc060105e90 2024-05-04T03:21:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.678+0000 7fc05cff9640 1 -- 172.21.15.39:0/1933667879 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500187e0 con 0x7fc060105e90 2024-05-04T03:21:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.682+0000 7fc05cff9640 1 -- 172.21.15.39:0/1933667879 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc05001f020 con 0x7fc060105e90 2024-05-04T03:21:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.683+0000 7fc05cff9640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0480b28e0 0x7fc0480b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc05cff9640 1 -- 172.21.15.39:0/1933667879 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc0500e30c0 con 0x7fc060105e90 2024-05-04T03:21:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc06d079740 1 Processor -- start 2024-05-04T03:21:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc06d079740 1 -- start start 2024-05-04T03:21:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc05effd640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0480b28e0 0x7fc0480b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc06d079740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98292a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc06d079740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a98292fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc06d079740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a98299ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc06d079740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556a981f75d0 con 0x556a97ec82f0 2024-05-04T03:21:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc06d079740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556a981b0890 con 0x556a97ed04d0 2024-05-04T03:21:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.684+0000 7fc06d079740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a981f3590 con 0x556a97ef11b0 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98292a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98292a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:59762/0 (socket says 172.21.15.39:59762) 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05f7fe640 1 -- 172.21.15.39:0/4090884955 learned_addr learned my addr 172.21.15.39:0/4090884955 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05ffff640 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a98299ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05effd640 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a98292fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05f7fe640 1 -- 172.21.15.39:0/4090884955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 msgr2=0x556a98299ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a98299ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05f7fe640 1 -- 172.21.15.39:0/4090884955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 msgr2=0x556a98292fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a98292fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.685+0000 7fc05f7fe640 1 -- 172.21.15.39:0/4090884955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a97f09210 con 0x556a97ef11b0 2024-05-04T03:21:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.686+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98292a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc0500a6710 tx=0x7fc0500aaed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347d581dd0538bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.686+0000 7fc05effd640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0480b28e0 0x7fc0480b4df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc0601a1870 tx=0x7fc054009820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.686+0000 7fc045ffb640 1 -- 172.21.15.39:0/4090884955 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500b34f0 con 0x556a97ef11b0 2024-05-04T03:21:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.686+0000 7fc045ffb640 1 -- 172.21.15.39:0/4090884955 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0500e0c80 con 0x556a97ef11b0 2024-05-04T03:21:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.686+0000 7fc045ffb640 1 -- 172.21.15.39:0/4090884955 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500b32c0 con 0x556a97ef11b0 2024-05-04T03:21:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.687+0000 7fc06d079740 1 -- 172.21.15.39:0/4090884955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 msgr2=0x556a98292a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.687+0000 7fc06d079740 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98292a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc0500a6710 tx=0x7fc0500aaed0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.688+0000 7fc06d079740 1 -- 172.21.15.39:0/4090884955 shutdown_connections 2024-05-04T03:21:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.688+0000 7fc06d079740 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a98299ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.688+0000 7fc06d079740 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a98292fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.688+0000 7fc06d079740 1 --2- 172.21.15.39:0/4090884955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98292a60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.688+0000 7fc06d079740 1 -- 172.21.15.39:0/4090884955 >> 172.21.15.39:0/4090884955 conn(0x556a97ee5300 msgr2=0x556a9828e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.688+0000 7fc06d079740 1 -- 172.21.15.39:0/4090884955 shutdown_connections 2024-05-04T03:21:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.688+0000 7fc06d079740 1 -- 172.21.15.39:0/4090884955 wait complete. 2024-05-04T03:21:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.688+0000 7fc06d079740 1 -- start start 2024-05-04T03:21:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.689+0000 7fc06d079740 1 Processor -- start 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.689+0000 7fc06d079740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ec82f0 0x556a9829c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.689+0000 7fc06d079740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ed04d0 0x556a9829ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.689+0000 7fc06d079740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.689+0000 7fc05f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ec82f0 0x556a9829c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.689+0000 7fc05f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ec82f0 0x556a9829c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:47606/0 (socket says 172.21.15.39:47606) 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.690+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.690+0000 7fc05effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ed04d0 0x556a9829ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.690+0000 7fc05effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ed04d0 0x556a9829ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38154/0 (socket says 172.21.15.39:38154) 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.690+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:59764/0 (socket says 172.21.15.39:59764) 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.690+0000 7fc05f7fe640 1 -- 172.21.15.39:0/994645110 learned_addr learned my addr 172.21.15.39:0/994645110 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.690+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556a9829a870 con 0x556a97ed04d0 2024-05-04T03:21:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.690+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556a981b4370 con 0x556a97ec82f0 2024-05-04T03:21:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.690+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a9829a9f0 con 0x556a97ef11b0 2024-05-04T03:21:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.691+0000 7fc06d079740 10 client.0 _ll_register_callbacks cb 0x556a97f08a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.691+0000 7fc05effd640 1 -- 172.21.15.39:0/994645110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 msgr2=0x556a982a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.691+0000 7fc05effd640 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.691+0000 7fc05effd640 1 -- 172.21.15.39:0/994645110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ec82f0 msgr2=0x556a9829c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.691+0000 7fc05effd640 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ec82f0 0x556a9829c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.691+0000 7fc05effd640 1 -- 172.21.15.39:0/994645110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a97fd5620 con 0x556a97ed04d0 2024-05-04T03:21:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.691+0000 7fc05effd640 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ed04d0 0x556a9829ca90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x556a98293e40 tx=0x7fc054004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608e96c5e06050e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.691+0000 7fc03effd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:21:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.692+0000 7fc03effd640 1 -- 172.21.15.39:0/994645110 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05401f450 con 0x556a97ed04d0 2024-05-04T03:21:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.692+0000 7fc03effd640 1 -- 172.21.15.39:0/994645110 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc05401f5f0 con 0x556a97ed04d0 2024-05-04T03:21:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.692+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0500e0c80 con 0x556a97ed04d0 2024-05-04T03:21:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.693+0000 7fc03effd640 1 -- 172.21.15.39:0/994645110 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc05401f830 con 0x556a97ed04d0 2024-05-04T03:21:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.693+0000 7fc06d079740 10 client.16425 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.693+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556a982a6980 con 0x556a97ed04d0 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.693+0000 7fc03effd640 1 -- 172.21.15.39:0/994645110 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc054010430 con 0x556a97ed04d0 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.694+0000 7fc03effd640 1 -- 172.21.15.39:0/994645110 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc0540273e0 con 0x556a97ed04d0 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.694+0000 7fc06d079740 10 client.16425 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.694+0000 7fc06d079740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.694+0000 7fc06d079740 20 client.16425 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.694+0000 7fc06d079740 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556a97eafd80 0x556a982aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.694+0000 7fc06d079740 4 client.16425 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.694+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc0500e0c80 con 0x556a97eafd80 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.694+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556a97eafd80 0x556a982aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.695+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556a97eafd80 0x556a982aa5f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd3a94eeed2be15 server_cookie=aa778d8c74004a9 in_seq=0 out_seq=0 2024-05-04T03:21:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.695+0000 7fc03effd640 10 client.16425 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.696+0000 7fc03effd640 1 -- 172.21.15.39:0/994645110 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc0500e0c80 con 0x556a97eafd80 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.696+0000 7fc03effd640 10 client.16425 handle_command_reply: tid=0 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.696+0000 7fc06d079740 1 client.16425 shutdown 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ed04d0 msgr2=0x556a9829ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ed04d0 0x556a9829ca90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x556a98293e40 tx=0x7fc054004730 comp rx=0 tx=0).stop 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 shutdown_connections 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556a97eafd80 0x556a982aa5f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ed04d0 0x556a9829ca90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 --2- 172.21.15.39:0/994645110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ec82f0 0x556a9829c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 >> 172.21.15.39:0/994645110 conn(0x556a97ee5300 msgr2=0x556a9828d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc03effd640 0 client.16425 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 shutdown_connections 2024-05-04T03:21:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.697+0000 7fc06d079740 1 -- 172.21.15.39:0/994645110 wait complete. 2024-05-04T03:21:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.698+0000 7fc06d079740 20 client.16425 trim_cache size 0 max 16384 2024-05-04T03:21:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.731+0000 7fc06d079740 1 Processor -- start 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc06d079740 1 -- start start 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc06d079740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a9828e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc06d079740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc06d079740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc06d079740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c00e430 con 0x556a97ec82f0 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc06d079740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c00ff70 con 0x556a97ed04d0 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc06d079740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c00eb00 con 0x556a97ef11b0 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:59766/0 (socket says 172.21.15.39:59766) 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc05ffff640 1 -- 172.21.15.39:0/2647395841 learned_addr learned my addr 172.21.15.39:0/2647395841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.732+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a9828e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.733+0000 7fc05effd640 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.733+0000 7fc05ffff640 1 -- 172.21.15.39:0/2647395841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 msgr2=0x556a982a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.733+0000 7fc05ffff640 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.733+0000 7fc05ffff640 1 -- 172.21.15.39:0/2647395841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 msgr2=0x556a9828e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.733+0000 7fc05ffff640 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a9828e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.733+0000 7fc05ffff640 1 -- 172.21.15.39:0/2647395841 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a982a4390 con 0x556a97ef11b0 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.733+0000 7fc05ffff640 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a3e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc05802acb0 tx=0x7fc0580095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5edc674a349262f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.734+0000 7fc0457fa640 1 -- 172.21.15.39:0/2647395841 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058036040 con 0x556a97ef11b0 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.734+0000 7fc0457fa640 1 -- 172.21.15.39:0/2647395841 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc058037040 con 0x556a97ef11b0 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.734+0000 7fc0457fa640 1 -- 172.21.15.39:0/2647395841 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0580043c0 con 0x556a97ef11b0 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 -- 172.21.15.39:0/2647395841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 msgr2=0x556a982a3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a3e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc05802acb0 tx=0x7fc0580095b0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 -- 172.21.15.39:0/2647395841 shutdown_connections 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a982a3e50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 --2- 172.21.15.39:0/2647395841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a9828e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 -- 172.21.15.39:0/2647395841 >> 172.21.15.39:0/2647395841 conn(0x556a97ee5300 msgr2=0x556a9828dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 -- 172.21.15.39:0/2647395841 shutdown_connections 2024-05-04T03:21:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.735+0000 7fc06d079740 1 -- 172.21.15.39:0/2647395841 wait complete. 2024-05-04T03:21:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.736+0000 7fc06d079740 1 -- start start 2024-05-04T03:21:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.736+0000 7fc06d079740 1 Processor -- start 2024-05-04T03:21:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc06d079740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a982a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc06d079740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc06d079740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98310990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc05f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a982a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc05f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a982a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38178/0 (socket says 172.21.15.39:38178) 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc05effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc05effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:47638/0 (socket says 172.21.15.39:47638) 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc06d079740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c00f8a0 con 0x556a97ec82f0 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc06d079740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c00f1d0 con 0x556a97ed04d0 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.737+0000 7fc06d079740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c00c410 con 0x556a97ef11b0 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.738+0000 7fc05f7fe640 1 -- 172.21.15.39:0/1944300154 learned_addr learned my addr 172.21.15.39:0/1944300154 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.738+0000 7fc06d079740 10 client.0 _ll_register_callbacks cb 0x7fc054009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.738+0000 7fc05ffff640 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98310990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.738+0000 7fc05effd640 1 -- 172.21.15.39:0/1944300154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 msgr2=0x556a98310990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.738+0000 7fc05effd640 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98310990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.738+0000 7fc05effd640 1 -- 172.21.15.39:0/1944300154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 msgr2=0x556a982a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.738+0000 7fc05effd640 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a982a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.738+0000 7fc05effd640 1 -- 172.21.15.39:0/1944300154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a98310ed0 con 0x556a97ed04d0 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.739+0000 7fc05effd640 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a1250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0540278d0 tx=0x7fc054009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a078f006482e80 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.739+0000 7fc03effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.739+0000 7fc03effd640 1 -- 172.21.15.39:0/1944300154 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054039070 con 0x556a97ed04d0 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.739+0000 7fc03effd640 1 -- 172.21.15.39:0/1944300154 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc0540275e0 con 0x556a97ed04d0 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.739+0000 7fc06d079740 1 -- 172.21.15.39:0/1944300154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc058037040 con 0x556a97ed04d0 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.739+0000 7fc03effd640 1 -- 172.21.15.39:0/1944300154 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc05401f660 con 0x556a97ed04d0 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.739+0000 7fc06d079740 10 client.26027 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.739+0000 7fc06d079740 1 -- 172.21.15.39:0/1944300154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556a98312030 con 0x556a97ed04d0 2024-05-04T03:21:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.740+0000 7fc03effd640 1 -- 172.21.15.39:0/1944300154 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc058037040 con 0x556a97ed04d0 2024-05-04T03:21:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.740+0000 7fc03effd640 1 -- 172.21.15.39:0/1944300154 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc05401fc40 con 0x556a97ed04d0 2024-05-04T03:21:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.740+0000 7fc06d079740 10 client.26027 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.740+0000 7fc06d079740 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.740+0000 7fc06d079740 20 client.26027 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.740+0000 7fc06d079740 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556a97e851c0 0x556a98314a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.740+0000 7fc06d079740 4 client.26027 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.740+0000 7fc06d079740 1 -- 172.21.15.39:0/1944300154 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a982051e0 con 0x556a97e851c0 2024-05-04T03:21:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.741+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556a97e851c0 0x556a98314a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.741+0000 7fc05f7fe640 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556a97e851c0 0x556a98314a20 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3448e5746b316e server_cookie=58532063119b66d0 in_seq=0 out_seq=0 2024-05-04T03:21:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.741+0000 7fc03effd640 10 client.26027 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.750+0000 7fc03effd640 1 -- 172.21.15.39:0/1944300154 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a982051e0 con 0x556a97e851c0 2024-05-04T03:21:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.750+0000 7fc03effd640 10 client.26027 handle_command_reply: tid=0 2024-05-04T03:21:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.750+0000 7fc06d079740 1 client.26027 shutdown 2024-05-04T03:21:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.751+0000 7fc06d079740 1 -- 172.21.15.39:0/1944300154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 msgr2=0x556a982a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.751+0000 7fc06d079740 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a1250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0540278d0 tx=0x7fc054009f20 comp rx=0 tx=0).stop 2024-05-04T03:21:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.751+0000 7fc06d079740 1 -- 172.21.15.39:0/1944300154 shutdown_connections 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.751+0000 7fc06d079740 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556a97e851c0 0x556a98314a20 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.751+0000 7fc03effd640 0 client.26027 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.751+0000 7fc06d079740 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a97ef11b0 0x556a98310990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.752+0000 7fc06d079740 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556a97ed04d0 0x556a982a1250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.752+0000 7fc06d079740 1 --2- 172.21.15.39:0/1944300154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556a97ec82f0 0x556a982a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.752+0000 7fc06d079740 1 -- 172.21.15.39:0/1944300154 >> 172.21.15.39:0/1944300154 conn(0x556a97ee5300 msgr2=0x556a982911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.752+0000 7fc06d079740 1 -- 172.21.15.39:0/1944300154 shutdown_connections 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.752+0000 7fc06d079740 1 -- 172.21.15.39:0/1944300154 wait complete. 2024-05-04T03:21:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.752+0000 7fc06d079740 20 client.26027 trim_cache size 0 max 16384 2024-05-04T03:21:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:21:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:21:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:21:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:21:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 -- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0480b28e0 msgr2=0x7fc0480b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0480b28e0 0x7fc0480b4df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc0601a1870 tx=0x7fc054009820 comp rx=0 tx=0).stop 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 -- 172.21.15.39:0/1933667879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc060105e90 msgr2=0x7fc0601a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc060105e90 0x7fc0601a07d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc05000e980 tx=0x7fc05000ed80 comp rx=0 tx=0).stop 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 -- 172.21.15.39:0/1933667879 shutdown_connections 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc0480b28e0 0x7fc0480b4df0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc06010bdd0 0x7fc0601a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0601070d0 0x7fc0601a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 --2- 172.21.15.39:0/1933667879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc060105e90 0x7fc0601a07d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.753+0000 7fc0661c7640 1 -- 172.21.15.39:0/1933667879 >> 172.21.15.39:0/1933667879 conn(0x7fc060101060 msgr2=0x7fc060107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.754+0000 7fc0661c7640 1 -- 172.21.15.39:0/1933667879 shutdown_connections 2024-05-04T03:21:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:00.754+0000 7fc0661c7640 1 -- 172.21.15.39:0/1933667879 wait complete. 2024-05-04T03:21:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:00 smithi066 ceph-mon[33264]: pgmap v692: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:21:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:00 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:00 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:00 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:00 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:00 smithi167 ceph-mon[33614]: pgmap v692: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:21:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:00 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:00 smithi039 ceph-mon[29715]: pgmap v692: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:21:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:00 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:00 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:00 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:00 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 430.0 s, 1247.8 tps, lat 80.043 ms stddev 49.141 2024-05-04T03:21:01.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:01 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:21:01.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:01 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:21:01.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:01 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:01.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:01 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:21:01.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:01 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:21:01.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:01 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:01.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:01 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:21:01.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:01 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:21:01.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:01 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:21:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:02 smithi066 ceph-mon[33264]: pgmap v693: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-04T03:21:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:02 smithi167 ceph-mon[33614]: pgmap v693: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-04T03:21:02.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:02 smithi039 ceph-mon[29715]: pgmap v693: 97 pgs: 97 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-04T03:21:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:04 smithi066 ceph-mon[33264]: pgmap v694: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:21:04.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:04 smithi167 ceph-mon[33614]: pgmap v694: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:21:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:04 smithi039 ceph-mon[29715]: pgmap v694: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-05-04T03:21:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 435.0 s, 1440.6 tps, lat 69.436 ms stddev 93.437 2024-05-04T03:21:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:06 smithi066 ceph-mon[33264]: pgmap v695: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-04T03:21:06.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:06 smithi167 ceph-mon[33614]: pgmap v695: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-04T03:21:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:06 smithi039 ceph-mon[29715]: pgmap v695: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-04T03:21:08.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:08 smithi066 ceph-mon[33264]: pgmap v696: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:21:08.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:08 smithi167 ceph-mon[33614]: pgmap v696: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:21:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:08 smithi039 ceph-mon[29715]: pgmap v696: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:21:10.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:10 smithi066 ceph-mon[33264]: pgmap v697: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:21:10.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:10 smithi167 ceph-mon[33614]: pgmap v697: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:21:10.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:10 smithi039 ceph-mon[29715]: pgmap v697: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:21:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 440.0 s, 1310.2 tps, lat 47.306 ms stddev 39.946 2024-05-04T03:21:12.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:12 smithi066 ceph-mon[33264]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:21:12.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:12 smithi167 ceph-mon[33614]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:21:12.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:12 smithi039 ceph-mon[29715]: pgmap v698: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:21:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 -- 172.21.15.39:0/1151699548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 msgr2=0x7f0e44105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1151699548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 0x7f0e44105120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009860 tx=0x7f0e3802a340 comp rx=0 tx=0).stop 2024-05-04T03:21:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 -- 172.21.15.39:0/1151699548 shutdown_connections 2024-05-04T03:21:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1151699548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e44109a10 0x7f0e4410be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1151699548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 0x7f0e44105120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1151699548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e44103ad0 0x7f0e441047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 -- 172.21.15.39:0/1151699548 >> 172.21.15.39:0/1151699548 conn(0x7f0e440fecc0 msgr2=0x7f0e44101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 -- 172.21.15.39:0/1151699548 shutdown_connections 2024-05-04T03:21:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.915+0000 7f0e494d9640 1 -- 172.21.15.39:0/1151699548 wait complete. 2024-05-04T03:21:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.916+0000 7f0e494d9640 1 Processor -- start 2024-05-04T03:21:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.917+0000 7f0e494d9640 1 -- start start 2024-05-04T03:21:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.917+0000 7f0e494d9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e44103ad0 0x7f0e4419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.917+0000 7f0e494d9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 0x7f0e4419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e494d9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e44109a10 0x7f0e441a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e494d9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4410f480 con 0x7f0e44109a10 2024-05-04T03:21:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e494d9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4410f300 con 0x7f0e44104d10 2024-05-04T03:21:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e494d9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4410f600 con 0x7f0e44103ad0 2024-05-04T03:21:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e427fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 0x7f0e4419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e427fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 0x7f0e4419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:52222/0 (socket says 172.21.15.39:52222) 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e427fc640 1 -- 172.21.15.39:0/1608845109 learned_addr learned my addr 172.21.15.39:0/1608845109 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e437fe640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e44109a10 0x7f0e441a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e44103ad0 0x7f0e4419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e437fe640 1 -- 172.21.15.39:0/1608845109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e44103ad0 msgr2=0x7f0e4419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e437fe640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e44103ad0 0x7f0e4419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e437fe640 1 -- 172.21.15.39:0/1608845109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 msgr2=0x7f0e4419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e437fe640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 0x7f0e4419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e437fe640 1 -- 172.21.15.39:0/1608845109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e441a1c30 con 0x7f0e44109a10 2024-05-04T03:21:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.918+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e44103ad0 0x7f0e4419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:21:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.919+0000 7f0e437fe640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e44109a10 0x7f0e441a15c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009fd0 tx=0x7f0e3c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.919+0000 7f0e2bfff640 1 -- 172.21.15.39:0/1608845109 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00ce20 con 0x7f0e44109a10 2024-05-04T03:21:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.919+0000 7f0e2bfff640 1 -- 172.21.15.39:0/1608845109 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e3c005020 con 0x7f0e44109a10 2024-05-04T03:21:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.919+0000 7f0e494d9640 1 -- 172.21.15.39:0/1608845109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e441a1f20 con 0x7f0e44109a10 2024-05-04T03:21:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.919+0000 7f0e2bfff640 1 -- 172.21.15.39:0/1608845109 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c010690 con 0x7f0e44109a10 2024-05-04T03:21:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.919+0000 7f0e494d9640 1 -- 172.21.15.39:0/1608845109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e441a22d0 con 0x7f0e44109a10 2024-05-04T03:21:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.923+0000 7f0e5038f740 1 Processor -- start 2024-05-04T03:21:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.925+0000 7f0e2bfff640 1 -- 172.21.15.39:0/1608845109 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0e3c010830 con 0x7f0e44109a10 2024-05-04T03:21:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.925+0000 7f0e5038f740 1 -- start start 2024-05-04T03:21:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.925+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa637c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.925+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa6381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.925+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 0x5614fa63f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.925+0000 7f0e5038f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5614fa59c5d0 con 0x5614fa2961b0 2024-05-04T03:21:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.925+0000 7f0e5038f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614fa555890 con 0x5614fa2754d0 2024-05-04T03:21:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.925+0000 7f0e5038f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5614fa598590 con 0x5614fa26d2f0 2024-05-04T03:21:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e427fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa637c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e427fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa637c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59552/0 (socket says 172.21.15.39:59552) 2024-05-04T03:21:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e427fc640 1 -- 172.21.15.39:0/3428294178 learned_addr learned my addr 172.21.15.39:0/3428294178 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e427fc640 1 --2- 172.21.15.39:0/3428294178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 0x5614fa63f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa6381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3428294178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 msgr2=0x5614fa63f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3428294178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 0x5614fa63f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3428294178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 msgr2=0x5614fa637c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3428294178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa637c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3428294178 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614fa2ae210 con 0x5614fa26d2f0 2024-05-04T03:21:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.926+0000 7f0e2bfff640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e2c0b2860 0x7f0e2c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.927+0000 7f0e427fc640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e2c0b2860 0x7f0e2c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.927+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3428294178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa6381b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0e441033a0 tx=0x7f0e34010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ac76bcda9f7e17 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.927+0000 7f0e2bfff640 1 -- 172.21.15.39:0/1608845109 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0e3c0e2a20 con 0x7f0e44109a10 2024-05-04T03:21:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.927+0000 7f0e297fa640 1 -- 172.21.15.39:0/3428294178 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34009e40 con 0x5614fa26d2f0 2024-05-04T03:21:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.927+0000 7f0e297fa640 1 -- 172.21.15.39:0/3428294178 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e340128e0 con 0x5614fa26d2f0 2024-05-04T03:21:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.927+0000 7f0e297fa640 1 -- 172.21.15.39:0/3428294178 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34011760 con 0x5614fa26d2f0 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e427fc640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e2c0b2860 0x7f0e2c0b4d70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5614fa638e00 tx=0x7f0e38005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 -- 172.21.15.39:0/3428294178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 msgr2=0x5614fa6381b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3428294178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa6381b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0e441033a0 tx=0x7f0e34010740 comp rx=0 tx=0).stop 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 -- 172.21.15.39:0/3428294178 shutdown_connections 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3428294178 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 0x5614fa63f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3428294178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa6381b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3428294178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa637c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 -- 172.21.15.39:0/3428294178 >> 172.21.15.39:0/3428294178 conn(0x5614fa28a300 msgr2=0x5614fa6335c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 -- 172.21.15.39:0/3428294178 shutdown_connections 2024-05-04T03:21:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.929+0000 7f0e5038f740 1 -- 172.21.15.39:0/3428294178 wait complete. 2024-05-04T03:21:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 1 -- start start 2024-05-04T03:21:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 1 Processor -- start 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa64a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 0x5614fa64a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa6514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5614fa63fb00 con 0x5614fa2961b0 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5614fa559370 con 0x5614fa2754d0 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5614fa63fc80 con 0x5614fa26d2f0 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e5038f740 10 client.0 _ll_register_callbacks cb 0x5614fa2ada00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa64a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa64a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:50432/0 (socket says 172.21.15.39:50432) 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.930+0000 7f0e42ffd640 1 -- 172.21.15.39:0/4133101372 learned_addr learned my addr 172.21.15.39:0/4133101372 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e437fe640 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa6514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e42ffd640 1 -- 172.21.15.39:0/4133101372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 msgr2=0x5614fa64a590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:21:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 0x5614fa64a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e42ffd640 1 -- 172.21.15.39:0/4133101372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 msgr2=0x5614fa6514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa6514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e42ffd640 1 -- 172.21.15.39:0/4133101372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614fa37a620 con 0x5614fa26d2f0 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e437fe640 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa6514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa64a050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e34000c00 tx=0x7f0e34012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4abef38ad1d398cd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.931+0000 7f0e26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.932+0000 7f0e26ff5640 1 -- 172.21.15.39:0/4133101372 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34005040 con 0x5614fa26d2f0 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.932+0000 7f0e26ff5640 1 -- 172.21.15.39:0/4133101372 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0e340051e0 con 0x5614fa26d2f0 2024-05-04T03:21:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.932+0000 7f0e5038f740 1 -- 172.21.15.39:0/4133101372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e340128e0 con 0x5614fa26d2f0 2024-05-04T03:21:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.932+0000 7f0e26ff5640 1 -- 172.21.15.39:0/4133101372 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e340053e0 con 0x5614fa26d2f0 2024-05-04T03:21:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.933+0000 7f0e5038f740 10 client.26056 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.933+0000 7f0e5038f740 1 -- 172.21.15.39:0/4133101372 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5614fa643ec0 con 0x5614fa26d2f0 2024-05-04T03:21:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.933+0000 7f0e26ff5640 1 -- 172.21.15.39:0/4133101372 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e340128e0 con 0x5614fa26d2f0 2024-05-04T03:21:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.933+0000 7f0e26ff5640 1 -- 172.21.15.39:0/4133101372 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0e3400f040 con 0x5614fa26d2f0 2024-05-04T03:21:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.934+0000 7f0e5038f740 10 client.26056 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.934+0000 7f0e5038f740 10 client.26056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.934+0000 7f0e5038f740 20 client.26056 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.934+0000 7f0e5038f740 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5614fa254d80 0x5614fa647330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.934+0000 7f0e5038f740 4 client.26056 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.934+0000 7f0e5038f740 1 -- 172.21.15.39:0/4133101372 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614fa5aa1e0 con 0x5614fa254d80 2024-05-04T03:21:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.934+0000 7f0e427fc640 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5614fa254d80 0x5614fa647330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.935+0000 7f0e427fc640 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5614fa254d80 0x5614fa647330 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e3b7d022e446fc server_cookie=c32a68009d7daa53 in_seq=0 out_seq=0 2024-05-04T03:21:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.935+0000 7f0e26ff5640 10 client.26056 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.936+0000 7f0e26ff5640 1 -- 172.21.15.39:0/4133101372 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614fa5aa1e0 con 0x5614fa254d80 2024-05-04T03:21:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.936+0000 7f0e26ff5640 10 client.26056 handle_command_reply: tid=0 2024-05-04T03:21:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.936+0000 7f0e5038f740 1 client.26056 shutdown 2024-05-04T03:21:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 -- 172.21.15.39:0/4133101372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 msgr2=0x5614fa64a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa64a050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e34000c00 tx=0x7f0e34012680 comp rx=0 tx=0).stop 2024-05-04T03:21:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 -- 172.21.15.39:0/4133101372 shutdown_connections 2024-05-04T03:21:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5614fa254d80 0x5614fa647330 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa2961b0 0x5614fa6514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2754d0 0x5614fa64a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 --2- 172.21.15.39:0/4133101372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa26d2f0 0x5614fa64a050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 -- 172.21.15.39:0/4133101372 >> 172.21.15.39:0/4133101372 conn(0x5614fa28a300 msgr2=0x5614fa6364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e26ff5640 0 client.26056 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 -- 172.21.15.39:0/4133101372 shutdown_connections 2024-05-04T03:21:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.937+0000 7f0e5038f740 1 -- 172.21.15.39:0/4133101372 wait complete. 2024-05-04T03:21:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.938+0000 7f0e5038f740 20 client.26056 trim_cache size 0 max 16384 2024-05-04T03:21:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.970+0000 7f0e5038f740 1 Processor -- start 2024-05-04T03:21:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.970+0000 7f0e5038f740 1 -- start start 2024-05-04T03:21:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa6337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa6337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa6337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59572/0 (socket says 172.21.15.39:59572) 2024-05-04T03:21:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa651410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 0x5614fa64a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e5038f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441a8e80 con 0x5614fa26d2f0 2024-05-04T03:21:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e5038f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441aa9c0 con 0x5614fa2961b0 2024-05-04T03:21:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e5038f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441a9550 con 0x5614fa2754d0 2024-05-04T03:21:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3934990192 learned_addr learned my addr 172.21.15.39:0/3934990192 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e437fe640 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 0x5614fa64a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e427fc640 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa651410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3934990192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 msgr2=0x5614fa651410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa651410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3934990192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 msgr2=0x5614fa64a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 0x5614fa64a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3934990192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614fa637740 con 0x5614fa26d2f0 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.971+0000 7f0e427fc640 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa651410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.972+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa6337d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0e34010780 tx=0x7f0e34009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243cb0bf07fd7cae server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.972+0000 7f0e28ff9640 1 -- 172.21.15.39:0/3934990192 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34005580 con 0x5614fa26d2f0 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.972+0000 7f0e28ff9640 1 -- 172.21.15.39:0/3934990192 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e34029070 con 0x5614fa26d2f0 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.972+0000 7f0e28ff9640 1 -- 172.21.15.39:0/3934990192 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34036480 con 0x5614fa26d2f0 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.973+0000 7f0e5038f740 1 -- 172.21.15.39:0/3934990192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 msgr2=0x5614fa6337d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.973+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa6337d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0e34010780 tx=0x7f0e34009460 comp rx=0 tx=0).stop 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.973+0000 7f0e5038f740 1 -- 172.21.15.39:0/3934990192 shutdown_connections 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.973+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 0x5614fa64a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.973+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa651410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.973+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3934990192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa6337d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.973+0000 7f0e5038f740 1 -- 172.21.15.39:0/3934990192 >> 172.21.15.39:0/3934990192 conn(0x5614fa28a300 msgr2=0x5614fa649ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.974+0000 7f0e5038f740 1 -- 172.21.15.39:0/3934990192 shutdown_connections 2024-05-04T03:21:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.974+0000 7f0e5038f740 1 -- 172.21.15.39:0/3934990192 wait complete. 2024-05-04T03:21:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.974+0000 7f0e5038f740 1 -- start start 2024-05-04T03:21:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.974+0000 7f0e5038f740 1 Processor -- start 2024-05-04T03:21:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.974+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa64e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.974+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa64ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e5038f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 0x5614fa64fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa64e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e5038f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441aa2f0 con 0x5614fa26d2f0 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e5038f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441a9c20 con 0x5614fa2961b0 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e5038f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441a6e60 con 0x5614fa2754d0 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa64e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59578/0 (socket says 172.21.15.39:59578) 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3326016481 learned_addr learned my addr 172.21.15.39:0/3326016481 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e5038f740 10 client.0 _ll_register_callbacks cb 0x7f0e34005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e427fc640 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa64ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e437fe640 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 0x5614fa64fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3326016481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 msgr2=0x5614fa64ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa64ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3326016481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 msgr2=0x5614fa64fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 0x5614fa64fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 -- 172.21.15.39:0/3326016481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614fa6500f0 con 0x5614fa26d2f0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e42ffd640 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa64e6c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0e34011680 tx=0x7f0e34009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5ccd6b3bf4e476 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.975+0000 7f0e26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.976+0000 7f0e26ff5640 1 -- 172.21.15.39:0/3326016481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34012660 con 0x5614fa26d2f0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.976+0000 7f0e26ff5640 1 -- 172.21.15.39:0/3326016481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f0e34010f40 con 0x5614fa26d2f0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.976+0000 7f0e5038f740 1 -- 172.21.15.39:0/3326016481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e34029070 con 0x5614fa26d2f0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.976+0000 7f0e26ff5640 1 -- 172.21.15.39:0/3326016481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e34011140 con 0x5614fa26d2f0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.976+0000 7f0e5038f740 10 client.16449 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.976+0000 7f0e5038f740 1 -- 172.21.15.39:0/3326016481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5614fa651380 con 0x5614fa26d2f0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.976+0000 7f0e26ff5640 1 -- 172.21.15.39:0/3326016481 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e34029070 con 0x5614fa26d2f0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.977+0000 7f0e26ff5640 1 -- 172.21.15.39:0/3326016481 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0e34036480 con 0x5614fa26d2f0 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.977+0000 7f0e5038f740 10 client.16449 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.977+0000 7f0e5038f740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.977+0000 7f0e5038f740 20 client.16449 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.977+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5614fa22a1c0 0x5614fa6ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.977+0000 7f0e5038f740 4 client.16449 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.977+0000 7f0e5038f740 1 -- 172.21.15.39:0/3326016481 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614fa5a4050 con 0x5614fa22a1c0 2024-05-04T03:21:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.977+0000 7f0e427fc640 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5614fa22a1c0 0x5614fa6ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.978+0000 7f0e427fc640 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5614fa22a1c0 0x5614fa6ca990 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908f594fa425df4e server_cookie=b2ec9691c97300ed in_seq=0 out_seq=0 2024-05-04T03:21:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.978+0000 7f0e26ff5640 10 client.16449 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e26ff5640 1 -- 172.21.15.39:0/3326016481 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614fa5a4050 con 0x5614fa22a1c0 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e26ff5640 10 client.16449 handle_command_reply: tid=0 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 client.16449 shutdown 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 -- 172.21.15.39:0/3326016481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 msgr2=0x5614fa64e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa64e6c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0e34011680 tx=0x7f0e34009830 comp rx=0 tx=0).stop 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 -- 172.21.15.39:0/3326016481 shutdown_connections 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5614fa22a1c0 0x5614fa6ca990 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e26ff5640 0 client.16449 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5614fa2961b0 0x5614fa64fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5614fa2754d0 0x5614fa64ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 --2- 172.21.15.39:0/3326016481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614fa26d2f0 0x5614fa64e6c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.987+0000 7f0e5038f740 1 -- 172.21.15.39:0/3326016481 >> 172.21.15.39:0/3326016481 conn(0x5614fa28a300 msgr2=0x5614fa636df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.988+0000 7f0e5038f740 1 -- 172.21.15.39:0/3326016481 shutdown_connections 2024-05-04T03:21:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.988+0000 7f0e5038f740 1 -- 172.21.15.39:0/3326016481 wait complete. 2024-05-04T03:21:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.988+0000 7f0e5038f740 20 client.16449 trim_cache size 0 max 16384 2024-05-04T03:21:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:21:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:21:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:21:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:21:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 -- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e2c0b2860 msgr2=0x7f0e2c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e2c0b2860 0x7f0e2c0b4d70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5614fa638e00 tx=0x7f0e38005e30 comp rx=0 tx=0).stop 2024-05-04T03:21:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 -- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e44109a10 msgr2=0x7f0e441a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e44109a10 0x7f0e441a15c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009fd0 tx=0x7f0e3c00ef10 comp rx=0 tx=0).stop 2024-05-04T03:21:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 -- 172.21.15.39:0/1608845109 shutdown_connections 2024-05-04T03:21:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f0e2c0b2860 0x7f0e2c0b4d70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e44109a10 0x7f0e441a15c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e44104d10 0x7f0e4419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 --2- 172.21.15.39:0/1608845109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e44103ad0 0x7f0e4419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 -- 172.21.15.39:0/1608845109 >> 172.21.15.39:0/1608845109 conn(0x7f0e440fecc0 msgr2=0x7f0e441059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 -- 172.21.15.39:0/1608845109 shutdown_connections 2024-05-04T03:21:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:13.989+0000 7f0e494d9640 1 -- 172.21.15.39:0/1608845109 wait complete. 2024-05-04T03:21:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:14 smithi066 ceph-mon[33264]: pgmap v699: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:21:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:14 smithi167 ceph-mon[33614]: pgmap v699: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:21:14.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:14 smithi039 ceph-mon[29715]: pgmap v699: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:21:16.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:16 smithi039 ceph-mon[29715]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-05-04T03:21:16.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 445.0 s, 1414.8 tps, lat 97.573 ms stddev 396.160 2024-05-04T03:21:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:16 smithi066 ceph-mon[33264]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-05-04T03:21:16.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:16 smithi167 ceph-mon[33614]: pgmap v700: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.48k op/s 2024-05-04T03:21:17.831 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 12.7 2024-05-04T03:21:17.832 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:21:18.216 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:21:18.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:18 smithi039 ceph-mon[29715]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:21:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:18 smithi066 ceph-mon[33264]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:21:18.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:18 smithi167 ceph-mon[33614]: pgmap v701: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:21:20.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:20 smithi066 ceph-mon[33264]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.52k op/s 2024-05-04T03:21:20.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:20 smithi167 ceph-mon[33614]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.52k op/s 2024-05-04T03:21:20.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:20 smithi039 ceph-mon[29715]: pgmap v702: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.52k op/s 2024-05-04T03:21:21.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 450.0 s, 1174.8 tps, lat 84.466 ms stddev 32.294 2024-05-04T03:21:21.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:21 smithi039 ceph-mon[29715]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:21:22.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:21 smithi066 ceph-mon[33264]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:21:22.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:21 smithi167 ceph-mon[33614]: pgmap v703: 97 pgs: 97 active+clean; 1.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:21:24.568 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:24 smithi066 ceph-mon[33264]: pgmap v704: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-05-04T03:21:24.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.577+0000 7f6caf7fe700 1 -- 172.21.15.39:0/2840793104 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4017400 con 0x7f6cb0108f20 2024-05-04T03:21:24.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:24 smithi039 ceph-mon[29715]: pgmap v704: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-05-04T03:21:24.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.578+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2840793104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb0108f20 msgr2=0x7f6cb0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:24.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.578+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2840793104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb0108f20 0x7f6cb0109330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40098f0 tx=0x7f6ca4009c00 comp rx=0 tx=0).stop 2024-05-04T03:21:24.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.578+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2840793104 shutdown_connections 2024-05-04T03:21:24.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.578+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2840793104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cb01045f0 0x7f6cb0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:24.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.578+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2840793104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cb0109870 0x7f6cb01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:24.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.578+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2840793104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb0108f20 0x7f6cb0109330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:24.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.578+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2840793104 >> 172.21.15.39:0/2840793104 conn(0x7f6cb0100040 msgr2=0x7f6cb0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:24.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.579+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2840793104 shutdown_connections 2024-05-04T03:21:24.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.579+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2840793104 wait complete. 2024-05-04T03:21:24.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.580+0000 7f6cb70e3700 1 Processor -- start 2024-05-04T03:21:24.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.580+0000 7f6cb70e3700 1 -- start start 2024-05-04T03:21:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.580+0000 7f6cb70e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cb01045f0 0x7f6cb019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.580+0000 7f6cb70e3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb0108f20 0x7f6cb019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.580+0000 7f6cb70e3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cb0109870 0x7f6cb01a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.580+0000 7f6cb70e3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00773e0 con 0x7f6cb01045f0 2024-05-04T03:21:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.580+0000 7f6cb70e3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0077260 con 0x7f6cb0108f20 2024-05-04T03:21:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.580+0000 7f6cb70e3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0077560 con 0x7f6cb0109870 2024-05-04T03:21:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb5680700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cb0109870 0x7f6cb01a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb4e7f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cb01045f0 0x7f6cb019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:24.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb5680700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cb0109870 0x7f6cb01a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:38712/0 (socket says 172.21.15.39:38712) 2024-05-04T03:21:24.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb5680700 1 -- 172.21.15.39:0/2943196293 learned_addr learned my addr 172.21.15.39:0/2943196293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:24.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6caffff700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb0108f20 0x7f6cb019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb4e7f700 1 -- 172.21.15.39:0/2943196293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cb0109870 msgr2=0x7f6cb01a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb4e7f700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cb0109870 0x7f6cb01a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb4e7f700 1 -- 172.21.15.39:0/2943196293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb0108f20 msgr2=0x7f6cb019d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb4e7f700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb0108f20 0x7f6cb019d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb4e7f700 1 -- 172.21.15.39:0/2943196293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb01a1e70 con 0x7f6cb01045f0 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.581+0000 7f6cb5680700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cb0109870 0x7f6cb01a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.582+0000 7f6cb4e7f700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cb01045f0 0x7f6cb019cfa0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00eb90 tx=0x7f6c9c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.582+0000 7f6cadffb700 1 -- 172.21.15.39:0/2943196293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00cc90 con 0x7f6cb01045f0 2024-05-04T03:21:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.582+0000 7f6cadffb700 1 -- 172.21.15.39:0/2943196293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c9c00ce30 con 0x7f6cb01045f0 2024-05-04T03:21:24.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.583+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb01a2160 con 0x7f6cb01045f0 2024-05-04T03:21:24.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.583+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb01a9a80 con 0x7f6cb01045f0 2024-05-04T03:21:24.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.583+0000 7f6cadffb700 1 -- 172.21.15.39:0/2943196293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c003de0 con 0x7f6cb01045f0 2024-05-04T03:21:24.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.584+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb00512c0 con 0x7f6cb01045f0 2024-05-04T03:21:24.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.587+0000 7f6cadffb700 1 -- 172.21.15.39:0/2943196293 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6c9c003f80 con 0x7f6cb01045f0 2024-05-04T03:21:24.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.588+0000 7f6cadffb700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ca00b2980 0x7f6ca00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.588+0000 7f6caffff700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ca00b2980 0x7f6ca00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.589+0000 7f6cadffb700 1 -- 172.21.15.39:0/2943196293 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6c9c0e2650 con 0x7f6cb01045f0 2024-05-04T03:21:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.589+0000 7f6caffff700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ca00b2980 0x7f6ca00b4e90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40098c0 tx=0x7f6ca4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:24.596+0000 7f6cadffb700 1 -- 172.21.15.39:0/2943196293 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c9c0aa3b0 con 0x7f6cb01045f0 2024-05-04T03:21:24.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:24 smithi167 ceph-mon[33614]: pgmap v704: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-05-04T03:21:25.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.073+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cb019e300 con 0x7f6cb01045f0 2024-05-04T03:21:25.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.074+0000 7f6cadffb700 1 -- 172.21.15.39:0/2943196293 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f6c9c0a9cb0 con 0x7f6cb01045f0 2024-05-04T03:21:25.077 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:21:25.077 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:21:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ca00b2980 msgr2=0x7f6ca00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ca00b2980 0x7f6ca00b4e90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40098c0 tx=0x7f6ca4019040 comp rx=0 tx=0).stop 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cb01045f0 msgr2=0x7f6cb019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cb01045f0 0x7f6cb019cfa0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00eb90 tx=0x7f6c9c00eea0 comp rx=0 tx=0).stop 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 shutdown_connections 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6ca00b2980 0x7f6ca00b4e90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cb01045f0 0x7f6cb019cfa0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cb0109870 0x7f6cb01a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 --2- 172.21.15.39:0/2943196293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb0108f20 0x7f6cb019d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 >> 172.21.15.39:0/2943196293 conn(0x7f6cb0100040 msgr2=0x7f6cb010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 shutdown_connections 2024-05-04T03:21:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:25.078+0000 7f6cb70e3700 1 -- 172.21.15.39:0/2943196293 wait complete. 2024-05-04T03:21:25.082 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:21:25.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:25 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2943196293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:21:25.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:25 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2943196293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:21:25.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:25 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2943196293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:21:25.848 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 72.7 --await-for 42.0' 2024-05-04T03:21:25.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 72.7 --await-for 42.0 2024-05-04T03:21:26.259 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:21:26.672 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:26 smithi039 ceph-mon[29715]: pgmap v705: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:21:26.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:26 smithi167 ceph-mon[33614]: pgmap v705: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:21:26.719 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:26 smithi066 ceph-mon[33264]: pgmap v705: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:21:26.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 455.0 s, 1223.6 tps, lat 82.248 ms stddev 26.287 2024-05-04T03:21:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.148+0000 7fdcc5b81640 1 -- 172.21.15.39:0/3987194796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 msgr2=0x7fdcc01069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.148+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/3987194796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 0x7fdcc01069c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009ca0 tx=0x7fdcb002a8c0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.149+0000 7fdcc5b81640 1 -- 172.21.15.39:0/3987194796 shutdown_connections 2024-05-04T03:21:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.149+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/3987194796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc010bc00 0x7fdcc010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.149+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/3987194796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcc0106f00 0x7fdcc0107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.149+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/3987194796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 0x7fdcc01069c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.149+0000 7fdcc5b81640 1 -- 172.21.15.39:0/3987194796 >> 172.21.15.39:0/3987194796 conn(0x7fdcc0100ed0 msgr2=0x7fdcc0103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.149+0000 7fdcc5b81640 1 -- 172.21.15.39:0/3987194796 shutdown_connections 2024-05-04T03:21:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.149+0000 7fdcc5b81640 1 -- 172.21.15.39:0/3987194796 wait complete. 2024-05-04T03:21:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.150+0000 7fdcc5b81640 1 Processor -- start 2024-05-04T03:21:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.150+0000 7fdcc5b81640 1 -- start start 2024-05-04T03:21:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.150+0000 7fdcc5b81640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 0x7fdcc019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.150+0000 7fdcc5b81640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc0106f00 0x7fdcc019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.150+0000 7fdcc5b81640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcc010bc00 0x7fdcc01a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5b81640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01116a0 con 0x7fdcc0105cc0 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5b81640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111520 con 0x7fdcc0106f00 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5b81640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111820 con 0x7fdcc010bc00 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc4b7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 0x7fdcc019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc4b7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 0x7fdcc019c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42576/0 (socket says 172.21.15.39:42576) 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc0106f00 0x7fdcc019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc4b7f640 1 -- 172.21.15.39:0/2618048465 learned_addr learned my addr 172.21.15.39:0/2618048465 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5380640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcc010bc00 0x7fdcc01a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5380640 1 -- 172.21.15.39:0/2618048465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc0106f00 msgr2=0x7fdcc019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5380640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc0106f00 0x7fdcc019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5380640 1 -- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 msgr2=0x7fdcc019c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5380640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 0x7fdcc019c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcc5380640 1 -- 172.21.15.39:0/2618048465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01a3e10 con 0x7fdcc010bc00 2024-05-04T03:21:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.151+0000 7fdcbffff640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc0106f00 0x7fdcc019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.152+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 0x7fdcc019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:21:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.152+0000 7fdcc5380640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcc010bc00 0x7fdcc01a37a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800e970 tx=0x7fdcb800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.152+0000 7fdcbdffb640 1 -- 172.21.15.39:0/2618048465 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800ccc0 con 0x7fdcc010bc00 2024-05-04T03:21:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.152+0000 7fdcc5b81640 1 -- 172.21.15.39:0/2618048465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc01a4100 con 0x7fdcc010bc00 2024-05-04T03:21:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.152+0000 7fdcbdffb640 1 -- 172.21.15.39:0/2618048465 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcb8005020 con 0x7fdcc010bc00 2024-05-04T03:21:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.152+0000 7fdcbdffb640 1 -- 172.21.15.39:0/2618048465 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800ce60 con 0x7fdcc010bc00 2024-05-04T03:21:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.152+0000 7fdcc5b81640 1 -- 172.21.15.39:0/2618048465 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a4650 con 0x7fdcc010bc00 2024-05-04T03:21:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.156+0000 7fdccdd6b740 1 Processor -- start 2024-05-04T03:21:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdcbdffb640 1 -- 172.21.15.39:0/2618048465 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdcb8004a00 con 0x7fdcc010bc00 2024-05-04T03:21:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdccdd6b740 1 -- start start 2024-05-04T03:21:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7e81b0 0x5599ceb89c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7bf2f0 0x5599ceb8a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7c74d0 0x5599ceb910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdccdd6b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5599ceaee5d0 con 0x5599ce7bf2f0 2024-05-04T03:21:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdccdd6b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599ceaa7890 con 0x5599ce7c74d0 2024-05-04T03:21:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdccdd6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5599ceaea590 con 0x5599ce7e81b0 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.157+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7e81b0 0x5599ceb89c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.158+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7e81b0 0x5599ceb89c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:59484/0 (socket says 172.21.15.39:59484) 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.158+0000 7fdcbffff640 1 -- 172.21.15.39:0/2692007432 learned_addr learned my addr 172.21.15.39:0/2692007432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.158+0000 7fdcbffff640 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7c74d0 0x5599ceb910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.158+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7bf2f0 0x5599ceb8a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.158+0000 7fdcbffff640 1 -- 172.21.15.39:0/2692007432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7c74d0 msgr2=0x5599ceb910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.158+0000 7fdcbdffb640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdcac0b2860 0x7fdcac0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.158+0000 7fdcbffff640 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7c74d0 0x5599ceb910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.159+0000 7fdcbffff640 1 -- 172.21.15.39:0/2692007432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7bf2f0 msgr2=0x5599ceb8a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.159+0000 7fdcbffff640 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7bf2f0 0x5599ceb8a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.159+0000 7fdcbffff640 1 -- 172.21.15.39:0/2692007432 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ce800210 con 0x5599ce7e81b0 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.159+0000 7fdcbdffb640 1 -- 172.21.15.39:0/2618048465 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fdcb80e1fe0 con 0x7fdcc010bc00 2024-05-04T03:21:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.159+0000 7fdcc5380640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdcac0b2860 0x7fdcac0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.159+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7bf2f0 0x5599ceb8a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:21:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.159+0000 7fdcbffff640 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7e81b0 0x5599ceb89c30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdca8011c00 tx=0x7fdca8009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bfc74e406cc3d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.160+0000 7fdcc5380640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdcac0b2860 0x7fdcac0b4d70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80a9640 tx=0x7fdcb80057a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.160+0000 7fdcb6ffd640 1 -- 172.21.15.39:0/2692007432 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca800fc50 con 0x5599ce7e81b0 2024-05-04T03:21:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.160+0000 7fdcb6ffd640 1 -- 172.21.15.39:0/2692007432 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdca800fdf0 con 0x5599ce7e81b0 2024-05-04T03:21:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.161+0000 7fdcb6ffd640 1 -- 172.21.15.39:0/2692007432 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca800e640 con 0x5599ce7e81b0 2024-05-04T03:21:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.161+0000 7fdccdd6b740 1 -- 172.21.15.39:0/2692007432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7e81b0 msgr2=0x5599ceb89c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.161+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7e81b0 0x5599ceb89c30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdca8011c00 tx=0x7fdca8009ba0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.162+0000 7fdccdd6b740 1 -- 172.21.15.39:0/2692007432 shutdown_connections 2024-05-04T03:21:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.162+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7c74d0 0x5599ceb910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.162+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7bf2f0 0x5599ceb8a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.162+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/2692007432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7e81b0 0x5599ceb89c30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.162+0000 7fdccdd6b740 1 -- 172.21.15.39:0/2692007432 >> 172.21.15.39:0/2692007432 conn(0x5599ce7dc300 msgr2=0x5599ceb855c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.162+0000 7fdccdd6b740 1 -- 172.21.15.39:0/2692007432 shutdown_connections 2024-05-04T03:21:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.163+0000 7fdccdd6b740 1 -- 172.21.15.39:0/2692007432 wait complete. 2024-05-04T03:21:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.163+0000 7fdccdd6b740 1 -- start start 2024-05-04T03:21:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.164+0000 7fdccdd6b740 1 Processor -- start 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.164+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.164+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cebb4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48160/0 (socket says 172.21.15.39:48160) 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdcbffff640 1 -- 172.21.15.39:0/921935117 learned_addr learned my addr 172.21.15.39:0/921935117 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5599ceba2b60 con 0x5599ce7e81b0 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599ceaab370 con 0x5599ce7bf2f0 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5599ceba2ce0 con 0x5599ce7c74d0 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdccdd6b740 10 client.0 _ll_register_callbacks cb 0x5599ce7ffa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.165+0000 7fdcbffff640 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cebb4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdcbffff640 1 -- 172.21.15.39:0/921935117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 msgr2=0x5599cebad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdcbffff640 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdcbffff640 1 -- 172.21.15.39:0/921935117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 msgr2=0x5599cebb4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdcbffff640 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cebb4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdcbffff640 1 -- 172.21.15.39:0/921935117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ce8cc620 con 0x5599ce7bf2f0 2024-05-04T03:21:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdcbffff640 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebad190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009b70 tx=0x7fdca8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e0bdb3cc0a0044b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdc9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:21:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdc9ffff640 1 -- 172.21.15.39:0/921935117 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca801f6e0 con 0x5599ce7bf2f0 2024-05-04T03:21:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdc9ffff640 1 -- 172.21.15.39:0/921935117 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fdca801f880 con 0x5599ce7bf2f0 2024-05-04T03:21:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.166+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca800fdf0 con 0x5599ce7bf2f0 2024-05-04T03:21:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdc9ffff640 1 -- 172.21.15.39:0/921935117 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdca801fa80 con 0x5599ce7bf2f0 2024-05-04T03:21:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdccdd6b740 10 client.26060 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5599cebb7540 con 0x5599ce7bf2f0 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdc9ffff640 1 -- 172.21.15.39:0/921935117 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdca80044c0 con 0x5599ce7bf2f0 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdc9ffff640 1 -- 172.21.15.39:0/921935117 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fdca802a020 con 0x5599ce7bf2f0 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdccdd6b740 10 client.26060 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdccdd6b740 10 client.26060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdccdd6b740 20 client.26060 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599ce7a6d80 0x5599cebba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdccdd6b740 4 client.26060 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.167+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdca800fdf0 con 0x5599ce7a6d80 2024-05-04T03:21:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.168+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599ce7a6d80 0x5599cebba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.168+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599ce7a6d80 0x5599cebba990 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a03f7de8d34424 server_cookie=8c249defbf699106 in_seq=0 out_seq=0 2024-05-04T03:21:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.168+0000 7fdc9ffff640 10 client.26060 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.169+0000 7fdc9ffff640 1 -- 172.21.15.39:0/921935117 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fdca800fdf0 con 0x5599ce7a6d80 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.169+0000 7fdc9ffff640 10 client.26060 handle_command_reply: tid=0 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.169+0000 7fdccdd6b740 1 client.26060 shutdown 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 msgr2=0x5599cebad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebad190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009b70 tx=0x7fdca8005850 comp rx=0 tx=0).stop 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 shutdown_connections 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599ce7a6d80 0x5599cebba990 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cebb4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/921935117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebad190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 >> 172.21.15.39:0/921935117 conn(0x5599ce7dc300 msgr2=0x5599ceb84580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.170+0000 7fdc9ffff640 0 client.26060 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.171+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 shutdown_connections 2024-05-04T03:21:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.171+0000 7fdccdd6b740 1 -- 172.21.15.39:0/921935117 wait complete. 2024-05-04T03:21:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.171+0000 7fdccdd6b740 20 client.26060 trim_cache size 0 max 16384 2024-05-04T03:21:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.201+0000 7fdccdd6b740 1 Processor -- start 2024-05-04T03:21:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.201+0000 7fdccdd6b740 1 -- start start 2024-05-04T03:21:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.201+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599ceb855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.201+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cebb4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdccdd6b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0070d40 con 0x5599ce7e81b0 2024-05-04T03:21:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdccdd6b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0072880 con 0x5599ce7bf2f0 2024-05-04T03:21:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdccdd6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0071410 con 0x5599ce7c74d0 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599ceb855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599ceb855e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48162/0 (socket says 172.21.15.39:48162) 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcbffff640 1 -- 172.21.15.39:0/3939480155 learned_addr learned my addr 172.21.15.39:0/3939480155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/3939480155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcbffff640 1 -- 172.21.15.39:0/3939480155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 msgr2=0x5599cebb45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcbffff640 1 --2- 172.21.15.39:0/3939480155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcbffff640 1 -- 172.21.15.39:0/3939480155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 msgr2=0x5599cebb4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcbffff640 1 --2- 172.21.15.39:0/3939480155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cebb4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.202+0000 7fdcbffff640 1 -- 172.21.15.39:0/3939480155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ceb80b70 con 0x5599ce7bf2f0 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.203+0000 7fdcbffff640 1 --2- 172.21.15.39:0/3939480155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599ceb855e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009d70 tx=0x7fdca8011500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6164604684848f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.203+0000 7fdcb67fc640 1 -- 172.21.15.39:0/3939480155 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8009e90 con 0x5599ce7bf2f0 2024-05-04T03:21:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.203+0000 7fdcb67fc640 1 -- 172.21.15.39:0/3939480155 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdca8005560 con 0x5599ce7bf2f0 2024-05-04T03:21:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.203+0000 7fdcb67fc640 1 -- 172.21.15.39:0/3939480155 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8003810 con 0x5599ce7bf2f0 2024-05-04T03:21:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.204+0000 7fdccdd6b740 1 -- 172.21.15.39:0/3939480155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 msgr2=0x5599ceb855e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.204+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/3939480155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599ceb855e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009d70 tx=0x7fdca8011500 comp rx=0 tx=0).stop 2024-05-04T03:21:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.205+0000 7fdccdd6b740 1 -- 172.21.15.39:0/3939480155 shutdown_connections 2024-05-04T03:21:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.205+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/3939480155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cebb4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.205+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/3939480155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.205+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/3939480155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599ceb855e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.205+0000 7fdccdd6b740 1 -- 172.21.15.39:0/3939480155 >> 172.21.15.39:0/3939480155 conn(0x5599ce7dc300 msgr2=0x5599ceb84ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.205+0000 7fdccdd6b740 1 -- 172.21.15.39:0/3939480155 shutdown_connections 2024-05-04T03:21:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.205+0000 7fdccdd6b740 1 -- 172.21.15.39:0/3939480155 wait complete. 2024-05-04T03:21:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.207+0000 7fdccdd6b740 1 -- start start 2024-05-04T03:21:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.207+0000 7fdccdd6b740 1 Processor -- start 2024-05-04T03:21:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.208+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebb1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.208+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.209+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebb1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.209+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebb1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48164/0 (socket says 172.21.15.39:48164) 2024-05-04T03:21:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.209+0000 7fdcc4b7f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.208+0000 7fdccdd6b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cec20d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.209+0000 7fdcc4b7f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:59528/0 (socket says 172.21.15.39:59528) 2024-05-04T03:21:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.209+0000 7fdcc4b7f640 1 -- 172.21.15.39:0/1144390766 learned_addr learned my addr 172.21.15.39:0/1144390766 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.209+0000 7fdccdd6b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc00721b0 con 0x5599ce7e81b0 2024-05-04T03:21:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.209+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0071ae0 con 0x5599ce7bf2f0 2024-05-04T03:21:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.209+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc006ed20 con 0x5599ce7c74d0 2024-05-04T03:21:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdcbffff640 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cec20d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdcbffff640 1 -- 172.21.15.39:0/1144390766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 msgr2=0x5599cebb1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdcbffff640 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdcbffff640 1 -- 172.21.15.39:0/1144390766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 msgr2=0x5599cec20d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdcbffff640 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cec20d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdcbffff640 1 -- 172.21.15.39:0/1144390766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cebb4e80 con 0x5599ce7bf2f0 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdccdd6b740 10 client.0 _ll_register_callbacks cb 0x7fdca80116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdcbffff640 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebb1980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdca8021010 tx=0x7fdca8037600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67a86d7026b1a4f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.210+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca8005560 con 0x5599ce7bf2f0 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.211+0000 7fdc9ffff640 10 client.26066 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.211+0000 7fdc9ffff640 1 -- 172.21.15.39:0/1144390766 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8009e90 con 0x5599ce7bf2f0 2024-05-04T03:21:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.211+0000 7fdc9ffff640 1 -- 172.21.15.39:0/1144390766 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fdca801f440 con 0x5599ce7bf2f0 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.211+0000 7fdc9ffff640 1 -- 172.21.15.39:0/1144390766 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdca801f6a0 con 0x5599ce7bf2f0 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.211+0000 7fdccdd6b740 10 client.26066 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.211+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5599cec21e80 con 0x5599ce7bf2f0 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.212+0000 7fdc9ffff640 1 -- 172.21.15.39:0/1144390766 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fdca802a040 con 0x5599ce7bf2f0 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.212+0000 7fdccdd6b740 10 client.26066 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.212+0000 7fdccdd6b740 10 client.26066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.213+0000 7fdccdd6b740 20 client.26066 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.213+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599ce77c1c0 0x5599cec24620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.213+0000 7fdccdd6b740 4 client.26066 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.213+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599ceafc1e0 con 0x5599ce77c1c0 2024-05-04T03:21:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.213+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599ce77c1c0 0x5599cec24620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.214+0000 7fdc9ffff640 10 client.26066 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.214+0000 7fdcc4b7f640 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599ce77c1c0 0x5599cec24620 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b526b350aaf4cdc6 server_cookie=17c2b70a01f360ea in_seq=0 out_seq=0 2024-05-04T03:21:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.223+0000 7fdc9ffff640 1 -- 172.21.15.39:0/1144390766 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599ceafc1e0 con 0x5599ce77c1c0 2024-05-04T03:21:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdc9ffff640 10 client.26066 handle_command_reply: tid=0 2024-05-04T03:21:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 client.26066 shutdown 2024-05-04T03:21:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 msgr2=0x5599cebb1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebb1980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdca8021010 tx=0x7fdca8037600 comp rx=0 tx=0).stop 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 shutdown_connections 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599ce77c1c0 0x5599cec24620 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599ce7e81b0 0x5599cec20d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599ce7c74d0 0x5599cebb1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 --2- 172.21.15.39:0/1144390766 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599ce7bf2f0 0x5599cebb1980 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdc9ffff640 0 client.26066 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 >> 172.21.15.39:0/1144390766 conn(0x5599ce7dc300 msgr2=0x5599ceba30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 shutdown_connections 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.224+0000 7fdccdd6b740 1 -- 172.21.15.39:0/1144390766 wait complete. 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.225+0000 7fdccdd6b740 20 client.26066 trim_cache size 0 max 16384 2024-05-04T03:21:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:21:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:21:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:21:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:21:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 -- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdcac0b2860 msgr2=0x7fdcac0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdcac0b2860 0x7fdcac0b4d70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80a9640 tx=0x7fdcb80057a0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 -- 172.21.15.39:0/2618048465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcc010bc00 msgr2=0x7fdcc01a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcc010bc00 0x7fdcc01a37a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800e970 tx=0x7fdcb800ee30 comp rx=0 tx=0).stop 2024-05-04T03:21:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 -- 172.21.15.39:0/2618048465 shutdown_connections 2024-05-04T03:21:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdcac0b2860 0x7fdcac0b4d70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcc010bc00 0x7fdcc01a37a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc0106f00 0x7fdcc019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 --2- 172.21.15.39:0/2618048465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0105cc0 0x7fdcc019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.226+0000 7fdcc5b81640 1 -- 172.21.15.39:0/2618048465 >> 172.21.15.39:0/2618048465 conn(0x7fdcc0100ed0 msgr2=0x7fdcc0107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.227+0000 7fdcc5b81640 1 -- 172.21.15.39:0/2618048465 shutdown_connections 2024-05-04T03:21:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:27.227+0000 7fdcc5b81640 1 -- 172.21.15.39:0/2618048465 wait complete. 2024-05-04T03:21:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.359+0000 7f2833d9e700 1 -- 172.21.15.39:0/2839517600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106580 msgr2=0x7f282c106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.359+0000 7f2833d9e700 1 --2- 172.21.15.39:0/2839517600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106580 0x7f282c106990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f281c009990 tx=0x7f281c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.360+0000 7f2833d9e700 1 -- 172.21.15.39:0/2839517600 shutdown_connections 2024-05-04T03:21:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.360+0000 7f2833d9e700 1 --2- 172.21.15.39:0/2839517600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f282c106ed0 0x7f282c113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.360+0000 7f2833d9e700 1 --2- 172.21.15.39:0/2839517600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f282c10c5c0 0x7f282c10c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.360+0000 7f2833d9e700 1 --2- 172.21.15.39:0/2839517600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106580 0x7f282c106990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.360+0000 7f2833d9e700 1 -- 172.21.15.39:0/2839517600 >> 172.21.15.39:0/2839517600 conn(0x7f282c102080 msgr2=0x7f282c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:28.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.361+0000 7f2833d9e700 1 -- 172.21.15.39:0/2839517600 shutdown_connections 2024-05-04T03:21:28.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.361+0000 7f2833d9e700 1 -- 172.21.15.39:0/2839517600 wait complete. 2024-05-04T03:21:28.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.362+0000 7f2833d9e700 1 Processor -- start 2024-05-04T03:21:28.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.362+0000 7f2833d9e700 1 -- start start 2024-05-04T03:21:28.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.362+0000 7f2833d9e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f282c106580 0x7f282c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.362+0000 7f2833d9e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106ed0 0x7f282c19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.362+0000 7f2833d9e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f282c10c5c0 0x7f282c1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.362+0000 7f2833d9e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1165f0 con 0x7f282c10c5c0 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106ed0 0x7f282c19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106ed0 0x7f282c19d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:59542/0 (socket says 172.21.15.39:59542) 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 -- 172.21.15.39:0/3205839943 learned_addr learned my addr 172.21.15.39:0/3205839943 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.362+0000 7f2833d9e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f282c116470 con 0x7f282c106580 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f282c116770 con 0x7f282c106ed0 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f283233b700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f282c10c5c0 0x7f282c1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831b3a700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f282c106580 0x7f282c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 -- 172.21.15.39:0/3205839943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f282c106580 msgr2=0x7f282c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f282c106580 0x7f282c19cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 -- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f282c10c5c0 msgr2=0x7f282c1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f282c10c5c0 0x7f282c1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 -- 172.21.15.39:0/3205839943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1a1e30 con 0x7f282c106ed0 2024-05-04T03:21:28.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.363+0000 7f2831339700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106ed0 0x7f282c19d500 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f281c0098e0 tx=0x7f281c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:28.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.364+0000 7f2822ffd700 1 -- 172.21.15.39:0/3205839943 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c026070 con 0x7f282c106ed0 2024-05-04T03:21:28.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.364+0000 7f2822ffd700 1 -- 172.21.15.39:0/3205839943 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f281c021410 con 0x7f282c106ed0 2024-05-04T03:21:28.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.364+0000 7f2822ffd700 1 -- 172.21.15.39:0/3205839943 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c021e60 con 0x7f282c106ed0 2024-05-04T03:21:28.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.364+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1a20c0 con 0x7f282c106ed0 2024-05-04T03:21:28.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.364+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1a2490 con 0x7f282c106ed0 2024-05-04T03:21:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.370+0000 7f2822ffd700 1 -- 172.21.15.39:0/3205839943 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f281c029070 con 0x7f282c106ed0 2024-05-04T03:21:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.370+0000 7f28427b2b80 1 Processor -- start 2024-05-04T03:21:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.371+0000 7f28427b2b80 1 -- start start 2024-05-04T03:21:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.371+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af85a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f283233b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af85a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af90430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f283233b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af85a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42636/0 (socket says 172.21.15.39:42636) 2024-05-04T03:21:28.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f28427b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56106aea89f0 con 0x56106af85660 2024-05-04T03:21:28.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f2831b3a700 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af90430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f28427b2b80 1 -- 172.21.15.39:0/3874408148 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56106ae47950 con 0x56106af85fb0 2024-05-04T03:21:28.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f28427b2b80 1 -- 172.21.15.39:0/3874408148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56106aea49b0 con 0x56106af90970 2024-05-04T03:21:28.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.372+0000 7f283233b700 1 -- 172.21.15.39:0/3874408148 learned_addr learned my addr 172.21.15.39:0/3874408148 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:28.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.373+0000 7f2822ffd700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f28180b29c0 0x7f28180b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.373+0000 7f2831339700 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f2831339700 1 -- 172.21.15.39:0/3874408148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 msgr2=0x56106af90430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:28.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f2831339700 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af90430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f2831339700 1 -- 172.21.15.39:0/3874408148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 msgr2=0x56106af85a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:28.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f2831339700 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af85a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f2831339700 1 -- 172.21.15.39:0/3874408148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106adf5a60 con 0x56106af90970 2024-05-04T03:21:28.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f2831b3a700 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af90430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:21:28.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f283233b700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f28180b29c0 0x7f28180b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f2822ffd700 1 -- 172.21.15.39:0/3205839943 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f281c0e2ec0 con 0x7f282c106ed0 2024-05-04T03:21:28.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.374+0000 7f2831339700 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af92d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f281c0a7790 tx=0x7f281c02e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb36d78d2e601d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:28.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.375+0000 7f2817fff700 1 -- 172.21.15.39:0/3874408148 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c0aa030 con 0x56106af90970 2024-05-04T03:21:28.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.375+0000 7f2817fff700 1 -- 172.21.15.39:0/3874408148 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f281c00ae80 con 0x56106af90970 2024-05-04T03:21:28.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.375+0000 7f2817fff700 1 -- 172.21.15.39:0/3874408148 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c0aa520 con 0x56106af90970 2024-05-04T03:21:28.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 -- 172.21.15.39:0/3874408148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 msgr2=0x56106af92d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af92d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f281c0a7790 tx=0x7f281c02e000 comp rx=0 tx=0).stop 2024-05-04T03:21:28.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f283233b700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f28180b29c0 0x7f28180b4ed0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x56106af88340 tx=0x7f2824004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:28.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 -- 172.21.15.39:0/3874408148 shutdown_connections 2024-05-04T03:21:28.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af85a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af92d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 --2- 172.21.15.39:0/3874408148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af90430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 -- 172.21.15.39:0/3874408148 >> 172.21.15.39:0/3874408148 conn(0x56106ae08fd0 msgr2=0x56106af81690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:28.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 -- 172.21.15.39:0/3874408148 shutdown_connections 2024-05-04T03:21:28.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.377+0000 7f28427b2b80 1 -- 172.21.15.39:0/3874408148 wait complete. 2024-05-04T03:21:28.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.378+0000 7f28427b2b80 1 -- start start 2024-05-04T03:21:28.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.378+0000 7f28427b2b80 1 Processor -- start 2024-05-04T03:21:28.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.379+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af8d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.379+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af8de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.379+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af8e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.379+0000 7f28427b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56106aea49b0 con 0x56106af85660 2024-05-04T03:21:28.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.379+0000 7f28427b2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56106ae0edc0 con 0x56106af85fb0 2024-05-04T03:21:28.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.379+0000 7f28427b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56106af934c0 con 0x56106af90970 2024-05-04T03:21:28.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.379+0000 7f28427b2b80 10 client.0 _ll_register_callbacks cb 0x56106ae40ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.380+0000 7f283233b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af8e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.380+0000 7f283233b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af8e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:59560/0 (socket says 172.21.15.39:59560) 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f283233b700 1 -- 172.21.15.39:0/4117452250 learned_addr learned my addr 172.21.15.39:0/4117452250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f2831b3a700 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af8d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f2831339700 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af8de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f283233b700 1 -- 172.21.15.39:0/4117452250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 msgr2=0x56106af8de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f283233b700 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af8de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f283233b700 1 -- 172.21.15.39:0/4117452250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 msgr2=0x56106af8d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f283233b700 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af8d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f283233b700 1 -- 172.21.15.39:0/4117452250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106ae44820 con 0x56106af90970 2024-05-04T03:21:28.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.381+0000 7f283233b700 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af8e3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2824007b10 tx=0x7f2824004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15ffd9ca4867ec41 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:28.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.382+0000 7f28157fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:21:28.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.382+0000 7f28157fa700 1 -- 172.21.15.39:0/4117452250 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824029060 con 0x56106af90970 2024-05-04T03:21:28.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.382+0000 7f28157fa700 1 -- 172.21.15.39:0/4117452250 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2824019890 con 0x56106af90970 2024-05-04T03:21:28.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.382+0000 7f28427b2b80 1 -- 172.21.15.39:0/4117452250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281c00ae80 con 0x56106af90970 2024-05-04T03:21:28.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.382+0000 7f28157fa700 1 -- 172.21.15.39:0/4117452250 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f28240093b0 con 0x56106af90970 2024-05-04T03:21:28.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.382+0000 7f28427b2b80 10 client.26098 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:28.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.382+0000 7f28427b2b80 1 -- 172.21.15.39:0/4117452250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56106afa6750 con 0x56106af90970 2024-05-04T03:21:28.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.383+0000 7f28157fa700 1 -- 172.21.15.39:0/4117452250 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f281c00ae80 con 0x56106af90970 2024-05-04T03:21:28.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.383+0000 7f28157fa700 1 -- 172.21.15.39:0/4117452250 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f28240045c0 con 0x56106af90970 2024-05-04T03:21:28.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.383+0000 7f28427b2b80 10 client.26098 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:28.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.383+0000 7f28427b2b80 10 client.26098 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:21:28.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.383+0000 7f28427b2b80 20 client.26098 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:28.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.383+0000 7f28427b2b80 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56106afa8d90 0x56106afab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.383+0000 7f28427b2b80 4 client.26098 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:28.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.383+0000 7f28427b2b80 1 -- 172.21.15.39:0/4117452250 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56106aeb63d0 con 0x56106afa8d90 2024-05-04T03:21:28.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.384+0000 7f2831b3a700 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56106afa8d90 0x56106afab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.384+0000 7f2831b3a700 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56106afa8d90 0x56106afab1f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cbeed366298e50 server_cookie=d17dd79f73ba59fb in_seq=0 out_seq=0 2024-05-04T03:21:28.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.384+0000 7f28157fa700 10 client.26098 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:28.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.385+0000 7f28157fa700 1 -- 172.21.15.39:0/4117452250 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56106aeb63d0 con 0x56106afa8d90 2024-05-04T03:21:28.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.385+0000 7f28157fa700 10 client.26098 handle_command_reply: tid=0 2024-05-04T03:21:28.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.385+0000 7f28427b2b80 1 client.26098 shutdown 2024-05-04T03:21:28.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 -- 172.21.15.39:0/4117452250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 msgr2=0x56106af8e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af8e3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2824007b10 tx=0x7f2824004ad0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 -- 172.21.15.39:0/4117452250 shutdown_connections 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56106afa8d90 0x56106afab1f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85660 0x56106af8d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106af8e3c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 --2- 172.21.15.39:0/4117452250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85fb0 0x56106af8de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 -- 172.21.15.39:0/4117452250 >> 172.21.15.39:0/4117452250 conn(0x56106ae08fd0 msgr2=0x56106af87250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28157fa700 0 client.26098 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.386+0000 7f28427b2b80 1 -- 172.21.15.39:0/4117452250 shutdown_connections 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.387+0000 7f28427b2b80 1 -- 172.21.15.39:0/4117452250 wait complete. 2024-05-04T03:21:28.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.387+0000 7f28427b2b80 20 client.26098 trim_cache size 0 max 16384 2024-05-04T03:21:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.421+0000 7f28427b2b80 1 Processor -- start 2024-05-04T03:21:28.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.421+0000 7f28427b2b80 1 -- start start 2024-05-04T03:21:28.424 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:28 smithi039 ceph-mon[29715]: pgmap v706: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-05-04T03:21:28.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.421+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af8deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85fb0 0x56106afb09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106afb0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f28427b2b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1ac900 con 0x56106af85fb0 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f28427b2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1ae440 con 0x56106af85660 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f28427b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1acfd0 con 0x56106af90970 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f283233b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106afb0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f2831b3a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af8deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f2831b3a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af8deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48218/0 (socket says 172.21.15.39:48218) 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f2831b3a700 1 -- 172.21.15.39:0/596896067 learned_addr learned my addr 172.21.15.39:0/596896067 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:28.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f2831b3a700 1 -- 172.21.15.39:0/596896067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 msgr2=0x56106afb0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f2831b3a700 1 --2- 172.21.15.39:0/596896067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106afb0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f2831b3a700 1 -- 172.21.15.39:0/596896067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85fb0 msgr2=0x56106afb09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f2831b3a700 1 --2- 172.21.15.39:0/596896067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85fb0 0x56106afb09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.422+0000 7f2831b3a700 1 -- 172.21.15.39:0/596896067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106af85470 con 0x56106af85660 2024-05-04T03:21:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.423+0000 7f2831b3a700 1 --2- 172.21.15.39:0/596896067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af8deb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2828000c00 tx=0x7f282800d480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=139832760bd14ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.423+0000 7f28177fe700 1 -- 172.21.15.39:0/596896067 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828013b80 con 0x56106af85660 2024-05-04T03:21:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.423+0000 7f28177fe700 1 -- 172.21.15.39:0/596896067 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2828013d20 con 0x56106af85660 2024-05-04T03:21:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.423+0000 7f28177fe700 1 -- 172.21.15.39:0/596896067 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828004b40 con 0x56106af85660 2024-05-04T03:21:28.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.424+0000 7f28427b2b80 1 -- 172.21.15.39:0/596896067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 msgr2=0x56106af8deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.424+0000 7f28427b2b80 1 --2- 172.21.15.39:0/596896067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af8deb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2828000c00 tx=0x7f282800d480 comp rx=0 tx=0).stop 2024-05-04T03:21:28.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.424+0000 7f28427b2b80 1 -- 172.21.15.39:0/596896067 shutdown_connections 2024-05-04T03:21:28.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.425+0000 7f28427b2b80 1 --2- 172.21.15.39:0/596896067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af90970 0x56106afb0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.425+0000 7f28427b2b80 1 --2- 172.21.15.39:0/596896067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af8deb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.425+0000 7f28427b2b80 1 --2- 172.21.15.39:0/596896067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af85fb0 0x56106afb09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.425+0000 7f28427b2b80 1 -- 172.21.15.39:0/596896067 >> 172.21.15.39:0/596896067 conn(0x56106ae08fd0 msgr2=0x56106af809b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:28.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.425+0000 7f28427b2b80 1 -- 172.21.15.39:0/596896067 shutdown_connections 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.425+0000 7f28427b2b80 1 -- 172.21.15.39:0/596896067 wait complete. 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.425+0000 7f28427b2b80 1 -- start start 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.425+0000 7f28427b2b80 1 Processor -- start 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.426+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.426+0000 7f28427b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af85fb0 0x56106af95c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.426+0000 7f2831b3a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.426+0000 7f2831b3a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af95720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48234/0 (socket says 172.21.15.39:48234) 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.426+0000 7f2831b3a700 1 -- 172.21.15.39:0/2576151351 learned_addr learned my addr 172.21.15.39:0/2576151351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:28.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.426+0000 7f28427b2b80 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af90970 0x56106af961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.426+0000 7f2831339700 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af85fb0 0x56106af95c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1add70 con 0x56106af90970 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1ad6a0 con 0x56106af85660 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f283233b700 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af90970 0x56106af961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f282c07cd90 con 0x56106af85fb0 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f28427b2b80 10 client.0 _ll_register_callbacks cb 0x56106ae40ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f2831b3a700 1 -- 172.21.15.39:0/2576151351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af85fb0 msgr2=0x56106af95c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f2831b3a700 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af85fb0 0x56106af95c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f2831b3a700 1 -- 172.21.15.39:0/2576151351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af90970 msgr2=0x56106af961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f2831b3a700 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af90970 0x56106af961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.427+0000 7f2831b3a700 1 -- 172.21.15.39:0/2576151351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106b0135f0 con 0x56106af85660 2024-05-04T03:21:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.428+0000 7f2831b3a700 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af95720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2828006010 tx=0x7f282801b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f1ad37a582e168 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.428+0000 7f28157fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:21:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.428+0000 7f28157fa700 1 -- 172.21.15.39:0/2576151351 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282801ba20 con 0x56106af85660 2024-05-04T03:21:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.428+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2828013d20 con 0x56106af85660 2024-05-04T03:21:28.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.428+0000 7f28157fa700 1 -- 172.21.15.39:0/2576151351 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f282802d440 con 0x56106af85660 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.429+0000 7f28157fa700 1 -- 172.21.15.39:0/2576151351 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f282802d640 con 0x56106af85660 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.429+0000 7f28427b2b80 10 client.26075 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.429+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=32}) -- 0x56106b0146c0 con 0x56106af85660 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.429+0000 7f28157fa700 1 -- 172.21.15.39:0/2576151351 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f282802b020 con 0x56106af85660 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.429+0000 7f28427b2b80 10 client.26075 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.430+0000 7f28427b2b80 10 client.26075 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.430+0000 7f28427b2b80 20 client.26075 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.430+0000 7f28427b2b80 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56106b014be0 0x56106b017000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:28.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.430+0000 7f28427b2b80 4 client.26075 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 72.7, "await_for": 42.0}] 2024-05-04T03:21:28.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.430+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 72.7, "await_for": 42.0}) -- 0x7f2828013d20 con 0x56106b014be0 2024-05-04T03:21:28.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.430+0000 7f2831339700 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56106b014be0 0x56106b017000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:28.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.431+0000 7f28157fa700 10 client.26075 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:28.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.431+0000 7f2831339700 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56106b014be0 0x56106b017000 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25bbbebcfae083a3 server_cookie=9570cef6ec869d3b in_seq=0 out_seq=0 2024-05-04T03:21:28.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:28 smithi167 ceph-mon[33614]: pgmap v706: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28157fa700 1 -- 172.21.15.39:0/2576151351 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f2828013d20 con 0x56106b014be0 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28157fa700 10 client.26075 handle_command_reply: tid=0 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 client.26075 shutdown 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 msgr2=0x56106af95720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af95720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2828006010 tx=0x7f282801b3c0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 shutdown_connections 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56106b014be0 0x56106b017000 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56106af90970 0x56106af961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56106af85660 0x56106af95720 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28157fa700 0 client.26075 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 --2- 172.21.15.39:0/2576151351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56106af85fb0 0x56106af95c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 >> 172.21.15.39:0/2576151351 conn(0x56106ae08fd0 msgr2=0x56106af813e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 shutdown_connections 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.787+0000 7f28427b2b80 1 -- 172.21.15.39:0/2576151351 wait complete. 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f28427b2b80 20 client.26075 trim_cache size 0 max 16384 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 26, 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stdout: "8a6f08e7": { 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 26, 2024-05-04T03:21:28.791 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 72.7, 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:28.792 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:28.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:28.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:28.793 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:21:28.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f28180b29c0 msgr2=0x7f28180b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f28180b29c0 0x7f28180b4ed0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x56106af88340 tx=0x7f2824004b40 comp rx=0 tx=0).stop 2024-05-04T03:21:28.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106ed0 msgr2=0x7f282c19d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:28.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106ed0 0x7f282c19d500 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f281c0098e0 tx=0x7f281c0057c0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 shutdown_connections 2024-05-04T03:21:28.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f28180b29c0 0x7f28180b4ed0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f282c10c5c0 0x7f282c1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f282c106580 0x7f282c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 --2- 172.21.15.39:0/3205839943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c106ed0 0x7f282c19d500 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:28.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 >> 172.21.15.39:0/3205839943 conn(0x7f282c102080 msgr2=0x7f282c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:28.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 shutdown_connections 2024-05-04T03:21:28.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:28.788+0000 7f2833d9e700 1 -- 172.21.15.39:0/3205839943 wait complete. 2024-05-04T03:21:28.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:28 smithi066 ceph-mon[33264]: pgmap v706: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-05-04T03:21:29.449 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8a6f08e7', quiesce took 11.6 seconds. Will release after: 12.7 2024-05-04T03:21:30.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:30 smithi167 ceph-mon[33614]: pgmap v707: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-05-04T03:21:30.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:30 smithi039 ceph-mon[29715]: pgmap v707: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-05-04T03:21:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:30 smithi066 ceph-mon[33264]: pgmap v707: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-05-04T03:21:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 460.0 s, 401.9 tps, lat 84.830 ms stddev 25.119 2024-05-04T03:21:32.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:32 smithi167 ceph-mon[33614]: pgmap v708: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-05-04T03:21:32.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:32 smithi039 ceph-mon[29715]: pgmap v708: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-05-04T03:21:32.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:32 smithi066 ceph-mon[33264]: pgmap v708: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-05-04T03:21:34.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:34 smithi167 ceph-mon[33614]: pgmap v709: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.20k op/s 2024-05-04T03:21:34.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:34 smithi039 ceph-mon[29715]: pgmap v709: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.20k op/s 2024-05-04T03:21:34.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:34 smithi066 ceph-mon[33264]: pgmap v709: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 1.20k op/s 2024-05-04T03:21:36.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:36 smithi167 ceph-mon[33614]: pgmap v710: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 900 op/s 2024-05-04T03:21:36.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:36 smithi039 ceph-mon[29715]: pgmap v710: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 900 op/s 2024-05-04T03:21:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 465.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:21:36.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:36 smithi066 ceph-mon[33264]: pgmap v710: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 900 op/s 2024-05-04T03:21:38.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:38 smithi167 ceph-mon[33614]: pgmap v711: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-05-04T03:21:38.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:38 smithi039 ceph-mon[29715]: pgmap v711: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-05-04T03:21:38.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:38 smithi066 ceph-mon[33264]: pgmap v711: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-05-04T03:21:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 -- 172.21.15.39:0/1614927488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4103c70 msgr2=0x7f79f4104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 --2- 172.21.15.39:0/1614927488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4103c70 0x7f79f4104970 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009900 tx=0x7f79e402a410 comp rx=0 tx=0).stop 2024-05-04T03:21:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 -- 172.21.15.39:0/1614927488 shutdown_connections 2024-05-04T03:21:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 --2- 172.21.15.39:0/1614927488 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79f4109bb0 0x7f79f410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 --2- 172.21.15.39:0/1614927488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f4104eb0 0x7f79f41052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 --2- 172.21.15.39:0/1614927488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4103c70 0x7f79f4104970 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 -- 172.21.15.39:0/1614927488 >> 172.21.15.39:0/1614927488 conn(0x7f79f40fee40 msgr2=0x7f79f4101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 -- 172.21.15.39:0/1614927488 shutdown_connections 2024-05-04T03:21:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.349+0000 7f79f8d14640 1 -- 172.21.15.39:0/1614927488 wait complete. 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f8d14640 1 Processor -- start 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f8d14640 1 -- start start 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f8d14640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f4103c70 0x7f79f419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f8d14640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79f4104eb0 0x7f79f419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f8d14640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4109bb0 0x7f79f41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f8d14640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410f620 con 0x7f79f4109bb0 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f8d14640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410f4a0 con 0x7f79f4104eb0 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f8d14640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410f7a0 con 0x7f79f4103c70 2024-05-04T03:21:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4109bb0 0x7f79f41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4109bb0 0x7f79f41a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47408/0 (socket says 172.21.15.39:47408) 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f2d9d640 1 -- 172.21.15.39:0/3489746173 learned_addr learned my addr 172.21.15.39:0/3489746173 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79f4104eb0 0x7f79f419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.350+0000 7f79f259c640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f4103c70 0x7f79f419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79f2d9d640 1 -- 172.21.15.39:0/3489746173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f4103c70 msgr2=0x7f79f419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79f2d9d640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f4103c70 0x7f79f419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79f2d9d640 1 -- 172.21.15.39:0/3489746173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79f4104eb0 msgr2=0x7f79f419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79f2d9d640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79f4104eb0 0x7f79f419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79f2d9d640 1 -- 172.21.15.39:0/3489746173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f41a1e00 con 0x7f79f4109bb0 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79f2d9d640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4109bb0 0x7f79f41a1790 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00eaa0 tx=0x7f79ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79e37fe640 1 -- 172.21.15.39:0/3489746173 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00cdb0 con 0x7f79f4109bb0 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79e37fe640 1 -- 172.21.15.39:0/3489746173 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79ec005020 con 0x7f79f4109bb0 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79f8d14640 1 -- 172.21.15.39:0/3489746173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f41a20f0 con 0x7f79f4109bb0 2024-05-04T03:21:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79e37fe640 1 -- 172.21.15.39:0/3489746173 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0116d0 con 0x7f79f4109bb0 2024-05-04T03:21:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.351+0000 7f79f8d14640 1 -- 172.21.15.39:0/3489746173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f41a25f0 con 0x7f79f4109bb0 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.354+0000 7f79ffbc7740 1 Processor -- start 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.354+0000 7f79e37fe640 1 -- 172.21.15.39:0/3489746173 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f79ec0051d0 con 0x7f79f4109bb0 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.354+0000 7f79ffbc7740 1 -- start start 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d718e1b0 0x55c4d752fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d716d4d0 0x55c4d7536ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c4d74945d0 con 0x55c4d718e1b0 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c4d744d890 con 0x55c4d716d4d0 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c4d7490590 con 0x55c4d71652f0 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d718e1b0 0x55c4d752fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d718e1b0 0x55c4d752fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47412/0 (socket says 172.21.15.39:47412) 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:60344/0 (socket says 172.21.15.39:60344) 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 -- 172.21.15.39:0/3661212114 learned_addr learned my addr 172.21.15.39:0/3661212114 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79e37fe640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f79dc0b27e0 0x7f79dc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d716d4d0 0x55c4d7536ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f2d9d640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f79dc0b27e0 0x7f79dc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 -- 172.21.15.39:0/3661212114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 msgr2=0x55c4d752ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 -- 172.21.15.39:0/3661212114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d716d4d0 msgr2=0x55c4d7536ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d716d4d0 0x55c4d7536ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f1d9b640 1 -- 172.21.15.39:0/3661212114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d71a6210 con 0x55c4d718e1b0 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79f259c640 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.355+0000 7f79e37fe640 1 -- 172.21.15.39:0/3489746173 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f79ec0e23f0 con 0x7f79f4109bb0 2024-05-04T03:21:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.356+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d718e1b0 0x55c4d752fa60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f79e800b620 tx=0x7f79e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e24adc9db8af7bd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.356+0000 7f79f2d9d640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f79dc0b27e0 0x7f79dc0b4cf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0a6c90 tx=0x7f79ec091040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.359+0000 7f79e0ff9640 1 -- 172.21.15.39:0/3661212114 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8003e80 con 0x55c4d718e1b0 2024-05-04T03:21:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.359+0000 7f79e0ff9640 1 -- 172.21.15.39:0/3661212114 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79e80054b0 con 0x55c4d718e1b0 2024-05-04T03:21:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.359+0000 7f79e0ff9640 1 -- 172.21.15.39:0/3661212114 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8004da0 con 0x55c4d718e1b0 2024-05-04T03:21:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3661212114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d718e1b0 msgr2=0x55c4d752fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d718e1b0 0x55c4d752fa60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f79e800b620 tx=0x7f79e8004270 comp rx=0 tx=0).stop 2024-05-04T03:21:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3661212114 shutdown_connections 2024-05-04T03:21:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d716d4d0 0x55c4d7536ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3661212114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d718e1b0 0x55c4d752fa60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3661212114 >> 172.21.15.39:0/3661212114 conn(0x55c4d7182300 msgr2=0x55c4d752b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3661212114 shutdown_connections 2024-05-04T03:21:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3661212114 wait complete. 2024-05-04T03:21:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 -- start start 2024-05-04T03:21:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.360+0000 7f79ffbc7740 1 Processor -- start 2024-05-04T03:21:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d75396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7539be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c4d75378d0 con 0x55c4d716d4d0 2024-05-04T03:21:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c4d7451370 con 0x55c4d718e1b0 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c4d7537a50 con 0x55c4d71652f0 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79ffbc7740 10 client.0 _ll_register_callbacks cb 0x55c4d71a5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7539be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7539be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47416/0 (socket says 172.21.15.39:47416) 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 -- 172.21.15.39:0/3627634376 learned_addr learned my addr 172.21.15.39:0/3627634376 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d75396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 -- 172.21.15.39:0/3627634376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 msgr2=0x55c4d75396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d75396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 -- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 msgr2=0x55c4d7540b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 -- 172.21.15.39:0/3627634376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d7272620 con 0x55c4d716d4d0 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:21:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79f259c640 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7539be0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f79e4006010 tx=0x7f79e4031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99211405b0d1445a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.361+0000 7f79d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79d9ffb640 1 -- 172.21.15.39:0/3627634376 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4035040 con 0x55c4d716d4d0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79d9ffb640 1 -- 172.21.15.39:0/3627634376 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f79e4004240 con 0x55c4d716d4d0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3627634376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e80054b0 con 0x55c4d716d4d0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79d9ffb640 1 -- 172.21.15.39:0/3627634376 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79e4004440 con 0x55c4d716d4d0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 10 client.16494 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3627634376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4d7543ae0 con 0x55c4d716d4d0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79d9ffb640 1 -- 172.21.15.39:0/3627634376 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79e80054b0 con 0x55c4d716d4d0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79d9ffb640 1 -- 172.21.15.39:0/3627634376 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f79e403d5e0 con 0x55c4d716d4d0 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 10 client.16494 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 20 client.16494 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c4d714cd80 0x55c4d75471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 4 client.16494 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.362+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3627634376 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d74a21e0 con 0x55c4d714cd80 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.363+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c4d714cd80 0x55c4d75471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.363+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c4d714cd80 0x55c4d75471b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74996a3a35b66442 server_cookie=fecea61166bacd10 in_seq=0 out_seq=0 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.363+0000 7f79d9ffb640 10 client.16494 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79d9ffb640 1 -- 172.21.15.39:0/3627634376 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4d74a21e0 con 0x55c4d714cd80 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79d9ffb640 10 client.16494 handle_command_reply: tid=0 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 client.16494 shutdown 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3627634376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 msgr2=0x55c4d7539be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7539be0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f79e4006010 tx=0x7f79e4031870 comp rx=0 tx=0).stop 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3627634376 shutdown_connections 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c4d714cd80 0x55c4d75471b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79d9ffb640 0 client.16494 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7539be0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/3627634376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d75396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3627634376 >> 172.21.15.39:0/3627634376 conn(0x55c4d7182300 msgr2=0x55c4d752e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3627634376 shutdown_connections 2024-05-04T03:21:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.364+0000 7f79ffbc7740 1 -- 172.21.15.39:0/3627634376 wait complete. 2024-05-04T03:21:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.365+0000 7f79ffbc7740 20 client.16494 trim_cache size 0 max 16384 2024-05-04T03:21:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79ffbc7740 1 Processor -- start 2024-05-04T03:21:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79ffbc7740 1 -- start start 2024-05-04T03:21:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7540a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79f41ab2d0 con 0x55c4d716d4d0 2024-05-04T03:21:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f79f41ace10 con 0x55c4d718e1b0 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79f41ab9a0 con 0x55c4d71652f0 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7540a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7540a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47432/0 (socket says 172.21.15.39:47432) 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f259c640 1 -- 172.21.15.39:0/2515991344 learned_addr learned my addr 172.21.15.39:0/2515991344 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/2515991344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f259c640 1 -- 172.21.15.39:0/2515991344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 msgr2=0x55c4d752e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f259c640 1 --2- 172.21.15.39:0/2515991344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f259c640 1 -- 172.21.15.39:0/2515991344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 msgr2=0x55c4d7540fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f259c640 1 --2- 172.21.15.39:0/2515991344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.380+0000 7f79f259c640 1 -- 172.21.15.39:0/2515991344 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d752f5f0 con 0x55c4d716d4d0 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.381+0000 7f79f259c640 1 --2- 172.21.15.39:0/2515991344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7540a60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009850 tx=0x7f79e4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a82e13a03307f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.381+0000 7f79dbfff640 1 -- 172.21.15.39:0/2515991344 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4035070 con 0x55c4d716d4d0 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.381+0000 7f79dbfff640 1 -- 172.21.15.39:0/2515991344 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79e4041070 con 0x55c4d716d4d0 2024-05-04T03:21:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.381+0000 7f79dbfff640 1 -- 172.21.15.39:0/2515991344 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4045c20 con 0x55c4d716d4d0 2024-05-04T03:21:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.381+0000 7f79ffbc7740 1 -- 172.21.15.39:0/2515991344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 msgr2=0x55c4d7540a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.381+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/2515991344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7540a60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009850 tx=0x7f79e4005f00 comp rx=0 tx=0).stop 2024-05-04T03:21:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.381+0000 7f79ffbc7740 1 -- 172.21.15.39:0/2515991344 shutdown_connections 2024-05-04T03:21:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.381+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/2515991344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d718e1b0 0x55c4d7540fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/2515991344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d7540a60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/2515991344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d71652f0 0x55c4d752e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 -- 172.21.15.39:0/2515991344 >> 172.21.15.39:0/2515991344 conn(0x55c4d7182300 msgr2=0x55c4d752e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 -- 172.21.15.39:0/2515991344 shutdown_connections 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 -- 172.21.15.39:0/2515991344 wait complete. 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 -- start start 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 Processor -- start 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d71652f0 0x55c4d753ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d753f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d718e1b0 0x55c4d7540020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79f41ac740 con 0x55c4d716d4d0 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f79f41ac070 con 0x55c4d71652f0 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79f41a92b0 con 0x55c4d718e1b0 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79ffbc7740 10 client.0 _ll_register_callbacks cb 0x7f79e4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d753f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d753f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47438/0 (socket says 172.21.15.39:47438) 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.382+0000 7f79f259c640 1 -- 172.21.15.39:0/209399003 learned_addr learned my addr 172.21.15.39:0/209399003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d71652f0 0x55c4d753ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d718e1b0 0x55c4d7540020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f259c640 1 -- 172.21.15.39:0/209399003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d718e1b0 msgr2=0x55c4d7540020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f259c640 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d718e1b0 0x55c4d7540020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f259c640 1 -- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d71652f0 msgr2=0x55c4d753ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f259c640 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d71652f0 0x55c4d753ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f259c640 1 -- 172.21.15.39:0/209399003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d75405e0 con 0x55c4d716d4d0 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d71652f0 0x55c4d753ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79f259c640 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d753f140 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f79e403db40 tx=0x7f79e4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd18fb2cf1d7b37 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:21:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79d9ffb640 1 -- 172.21.15.39:0/209399003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e405c020 con 0x55c4d716d4d0 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79d9ffb640 1 -- 172.21.15.39:0/209399003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f79e4045ba0 con 0x55c4d716d4d0 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79ffbc7740 1 -- 172.21.15.39:0/209399003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e4041070 con 0x55c4d716d4d0 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79d9ffb640 1 -- 172.21.15.39:0/209399003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79e406a050 con 0x55c4d716d4d0 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79ffbc7740 10 client.16500 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79ffbc7740 1 -- 172.21.15.39:0/209399003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4d75af610 con 0x55c4d716d4d0 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.383+0000 7f79d9ffb640 1 -- 172.21.15.39:0/209399003 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79e4041070 con 0x55c4d716d4d0 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79d9ffb640 1 -- 172.21.15.39:0/209399003 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f79e403c450 con 0x55c4d716d4d0 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79ffbc7740 10 client.16500 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79ffbc7740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79ffbc7740 20 client.16500 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c4d71221c0 0x55c4d75b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79ffbc7740 4 client.16500 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79ffbc7740 1 -- 172.21.15.39:0/209399003 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4d749c050 con 0x55c4d71221c0 2024-05-04T03:21:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c4d71221c0 0x55c4d75b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79f1d9b640 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c4d71221c0 0x55c4d75b1fd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c39a9ecead78ab server_cookie=afbbdd936eb34896 in_seq=0 out_seq=0 2024-05-04T03:21:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.384+0000 7f79d9ffb640 10 client.16500 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79d9ffb640 1 -- 172.21.15.39:0/209399003 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4d749c050 con 0x55c4d71221c0 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79d9ffb640 10 client.16500 handle_command_reply: tid=0 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 client.16500 shutdown 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 -- 172.21.15.39:0/209399003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 msgr2=0x55c4d753f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d753f140 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f79e403db40 tx=0x7f79e4005720 comp rx=0 tx=0).stop 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 -- 172.21.15.39:0/209399003 shutdown_connections 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c4d71221c0 0x55c4d75b1fd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c4d718e1b0 0x55c4d7540020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c4d716d4d0 0x55c4d753f140 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 --2- 172.21.15.39:0/209399003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c4d71652f0 0x55c4d753ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79d9ffb640 0 client.16500 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 -- 172.21.15.39:0/209399003 >> 172.21.15.39:0/209399003 conn(0x55c4d7182300 msgr2=0x55c4d754b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 -- 172.21.15.39:0/209399003 shutdown_connections 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 1 -- 172.21.15.39:0/209399003 wait complete. 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79ffbc7740 20 client.16500 trim_cache size 0 max 16384 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:21:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:21:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79f8d14640 1 -- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f79dc0b27e0 msgr2=0x7f79dc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79f8d14640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f79dc0b27e0 0x7f79dc0b4cf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0a6c90 tx=0x7f79ec091040 comp rx=0 tx=0).stop 2024-05-04T03:21:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79f8d14640 1 -- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4109bb0 msgr2=0x7f79f41a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.394+0000 7f79f8d14640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4109bb0 0x7f79f41a1790 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00eaa0 tx=0x7f79ec00ef60 comp rx=0 tx=0).stop 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.395+0000 7f79f8d14640 1 -- 172.21.15.39:0/3489746173 shutdown_connections 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.395+0000 7f79f8d14640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f79dc0b27e0 0x7f79dc0b4cf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.395+0000 7f79f8d14640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79f4109bb0 0x7f79f41a1790 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.395+0000 7f79f8d14640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79f4104eb0 0x7f79f419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.395+0000 7f79f8d14640 1 --2- 172.21.15.39:0/3489746173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79f4103c70 0x7f79f419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.395+0000 7f79f8d14640 1 -- 172.21.15.39:0/3489746173 >> 172.21.15.39:0/3489746173 conn(0x7f79f40fee40 msgr2=0x7f79f4105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.395+0000 7f79f8d14640 1 -- 172.21.15.39:0/3489746173 shutdown_connections 2024-05-04T03:21:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:40.395+0000 7f79f8d14640 1 -- 172.21.15.39:0/3489746173 wait complete. 2024-05-04T03:21:40.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:40 smithi167 ceph-mon[33614]: pgmap v712: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 488 op/s 2024-05-04T03:21:40.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:40 smithi039 ceph-mon[29715]: pgmap v712: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 488 op/s 2024-05-04T03:21:40.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:40 smithi066 ceph-mon[33264]: pgmap v712: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 488 op/s 2024-05-04T03:21:41.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 470.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:21:42.150 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:21:42.418 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:21:42.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:42 smithi039 ceph-mon[29715]: pgmap v713: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-05-04T03:21:42.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:42 smithi167 ceph-mon[33614]: pgmap v713: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-05-04T03:21:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:42 smithi066 ceph-mon[33264]: pgmap v713: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 -- 172.21.15.39:0/4228633871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 msgr2=0x7f4cdc102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/4228633871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 0x7f4cdc102cf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009990 tx=0x7f4cd0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 -- 172.21.15.39:0/4228633871 shutdown_connections 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/4228633871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc103230 0x7f4cdc111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/4228633871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cdc1092d0 0x7f4cdc1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/4228633871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 0x7f4cdc102cf0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 -- 172.21.15.39:0/4228633871 >> 172.21.15.39:0/4228633871 conn(0x7f4cdc0fe6f0 msgr2=0x7f4cdc100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 -- 172.21.15.39:0/4228633871 shutdown_connections 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.150+0000 7f4ce16c0700 1 -- 172.21.15.39:0/4228633871 wait complete. 2024-05-04T03:21:44.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.151+0000 7f4ce16c0700 1 Processor -- start 2024-05-04T03:21:44.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.151+0000 7f4ce16c0700 1 -- start start 2024-05-04T03:21:44.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.151+0000 7f4ce16c0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 0x7f4cdc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:44.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.151+0000 7f4ce16c0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cdc103230 0x7f4cdc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.151+0000 7f4ce16c0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc1092d0 0x7f4cdc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.151+0000 7f4ce16c0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc114440 con 0x7f4cdc1028e0 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.151+0000 7f4ce16c0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1142c0 con 0x7f4cdc103230 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.151+0000 7f4ce16c0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1145c0 con 0x7f4cdc1092d0 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 0x7f4cdc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 0x7f4cdc19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47452/0 (socket says 172.21.15.39:47452) 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 -- 172.21.15.39:0/446044685 learned_addr learned my addr 172.21.15.39:0/446044685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cda7fc700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cdc103230 0x7f4cdc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdb7fe700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc1092d0 0x7f4cdc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 -- 172.21.15.39:0/446044685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc1092d0 msgr2=0x7f4cdc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc1092d0 0x7f4cdc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 -- 172.21.15.39:0/446044685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cdc103230 msgr2=0x7f4cdc19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cdc103230 0x7f4cdc19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 -- 172.21.15.39:0/446044685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc19fcc0 con 0x7f4cdc1028e0 2024-05-04T03:21:44.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cdaffd700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 0x7f4cdc19adf0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e970 tx=0x7f4ccc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:44.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cc3fff700 1 -- 172.21.15.39:0/446044685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00cbd0 con 0x7f4cdc1028e0 2024-05-04T03:21:44.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc19ffb0 con 0x7f4cdc1028e0 2024-05-04T03:21:44.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cc3fff700 1 -- 172.21.15.39:0/446044685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ccc00cd70 con 0x7f4cdc1028e0 2024-05-04T03:21:44.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4cc3fff700 1 -- 172.21.15.39:0/446044685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc005570 con 0x7f4cdc1028e0 2024-05-04T03:21:44.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.152+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc1a78d0 con 0x7f4cdc1028e0 2024-05-04T03:21:44.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.153+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cdc0512c0 con 0x7f4cdc1028e0 2024-05-04T03:21:44.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.155+0000 7f4cc3fff700 1 -- 172.21.15.39:0/446044685 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4ccc004750 con 0x7f4cdc1028e0 2024-05-04T03:21:44.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.156+0000 7f4cc3fff700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4cc40b2880 0x7f4cc40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:44.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.156+0000 7f4cda7fc700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4cc40b2880 0x7f4cc40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:44.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.156+0000 7f4cc3fff700 1 -- 172.21.15.39:0/446044685 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4ccc0e2610 con 0x7f4cdc1028e0 2024-05-04T03:21:44.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.156+0000 7f4cda7fc700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4cc40b2880 0x7f4cc40b4d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000b2e0 tx=0x7f4cd0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:44.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.159+0000 7f4cc3fff700 1 -- 172.21.15.39:0/446044685 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ccc054df0 con 0x7f4cdc1028e0 2024-05-04T03:21:44.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.428+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cdc19c150 con 0x7f4cdc1028e0 2024-05-04T03:21:44.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.429+0000 7f4cc3fff700 1 -- 172.21.15.39:0/446044685 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f4ccc091090 con 0x7f4cdc1028e0 2024-05-04T03:21:44.431 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:21:44.433 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:21:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4cc40b2880 msgr2=0x7f4cc40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:44.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4cc40b2880 0x7f4cc40b4d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000b2e0 tx=0x7f4cd0005fb0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 msgr2=0x7f4cdc19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 0x7f4cdc19adf0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e970 tx=0x7f4ccc00ec80 comp rx=0 tx=0).stop 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 shutdown_connections 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4cc40b2880 0x7f4cc40b4d90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc1028e0 0x7f4cdc19adf0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cdc1092d0 0x7f4cdc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 --2- 172.21.15.39:0/446044685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4cdc103230 0x7f4cdc19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 >> 172.21.15.39:0/446044685 conn(0x7f4cdc0fe6f0 msgr2=0x7f4cdc10a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 shutdown_connections 2024-05-04T03:21:44.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:44.431+0000 7f4ce16c0700 1 -- 172.21.15.39:0/446044685 wait complete. 2024-05-04T03:21:44.435 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:21:44.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:44 smithi167 ceph-mon[33614]: pgmap v714: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-04T03:21:44.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:44 smithi039 ceph-mon[29715]: pgmap v714: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-04T03:21:44.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:44 smithi066 ceph-mon[33264]: pgmap v714: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-04T03:21:45.230 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 8a6f08e7 --release --await' 2024-05-04T03:21:45.231 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id 8a6f08e7 --release --await 2024-05-04T03:21:45.484 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:21:45.606 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:45 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/446044685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:21:45.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:45 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/446044685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:21:45.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:45 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/446044685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:21:46.518 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:46 smithi039 ceph-mon[29715]: pgmap v715: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-05-04T03:21:46.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:46 smithi167 ceph-mon[33614]: pgmap v715: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-05-04T03:21:46.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 475.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:21:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:46 smithi066 ceph-mon[33264]: pgmap v715: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-05-04T03:21:47.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.473+0000 7f88f5fb5700 1 -- 172.21.15.39:0/1896196351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 msgr2=0x7f88f0103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.473+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/1896196351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 0x7f88f0103930 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f88e00099c0 tx=0x7f88e0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.473+0000 7f88f5fb5700 1 -- 172.21.15.39:0/1896196351 shutdown_connections 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.473+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/1896196351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 0x7f88f0103930 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.473+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/1896196351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0108750 0x7f88f01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.474+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/1896196351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88f0107e00 0x7f88f0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.474+0000 7f88f5fb5700 1 -- 172.21.15.39:0/1896196351 >> 172.21.15.39:0/1896196351 conn(0x7f88f00fef90 msgr2=0x7f88f01013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.474+0000 7f88f5fb5700 1 -- 172.21.15.39:0/1896196351 shutdown_connections 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.474+0000 7f88f5fb5700 1 -- 172.21.15.39:0/1896196351 wait complete. 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.474+0000 7f88f5fb5700 1 Processor -- start 2024-05-04T03:21:47.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88f5fb5700 1 -- start start 2024-05-04T03:21:47.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88f5fb5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 0x7f88f019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88f5fb5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88f0107e00 0x7f88f019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88f5fb5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0108750 0x7f88f019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88f5fb5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88f0114380 con 0x7f88f0103520 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88f5fb5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f88f0114200 con 0x7f88f0107e00 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88f5fb5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88f0114500 con 0x7f88f0108750 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 0x7f88f019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 0x7f88f019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55094/0 (socket says 172.21.15.39:55094) 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88ef7fe700 1 -- 172.21.15.39:0/2119128261 learned_addr learned my addr 172.21.15.39:0/2119128261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88effff700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0108750 0x7f88f019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.475+0000 7f88eeffd700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88f0107e00 0x7f88f019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ef7fe700 1 -- 172.21.15.39:0/2119128261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0108750 msgr2=0x7f88f019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0108750 0x7f88f019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ef7fe700 1 -- 172.21.15.39:0/2119128261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88f0107e00 msgr2=0x7f88f019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88f0107e00 0x7f88f019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ef7fe700 1 -- 172.21.15.39:0/2119128261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f019fc00 con 0x7f88f0103520 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88effff700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0108750 0x7f88f019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 0x7f88f019ad30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f88e0006010 tx=0x7f88e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ecff9700 1 -- 172.21.15.39:0/2119128261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0026070 con 0x7f88f0103520 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ecff9700 1 -- 172.21.15.39:0/2119128261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88e0004630 con 0x7f88f0103520 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88f5fb5700 1 -- 172.21.15.39:0/2119128261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f019fe90 con 0x7f88f0103520 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88ecff9700 1 -- 172.21.15.39:0/2119128261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e000faa0 con 0x7f88f0103520 2024-05-04T03:21:47.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.476+0000 7f88f5fb5700 1 -- 172.21.15.39:0/2119128261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f01a0370 con 0x7f88f0103520 2024-05-04T03:21:47.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.479+0000 7f88ecff9700 1 -- 172.21.15.39:0/2119128261 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f88e0029070 con 0x7f88f0103520 2024-05-04T03:21:47.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.480+0000 7f88ecff9700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f88d80b2880 0x7f88d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.480+0000 7f88eeffd700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f88d80b2880 0x7f88d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.480+0000 7f88ecff9700 1 -- 172.21.15.39:0/2119128261 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f88e00e2b20 con 0x7f88f0103520 2024-05-04T03:21:47.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.480+0000 7f89049c9b80 1 Processor -- start 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.480+0000 7f89049c9b80 1 -- start start 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.480+0000 7f88eeffd700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f88d80b2880 0x7f88d80b4d90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f88f01067b0 tx=0x7f88e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dc420 0x555fa92e6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f89049c9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555fa91fea40 con 0x555fa92e6c40 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f89049c9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555fa9163ea0 con 0x555fa92dbad0 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f89049c9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555fa91faa00 con 0x555fa92dc420 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88effff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88effff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55096/0 (socket says 172.21.15.39:55096) 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88effff700 1 -- 172.21.15.39:0/2949854088 learned_addr learned my addr 172.21.15.39:0/2949854088 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88eeffd700 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dc420 0x555fa92e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88effff700 1 -- 172.21.15.39:0/2949854088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dc420 msgr2=0x555fa92e6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:47.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88effff700 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dc420 0x555fa92e6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88effff700 1 -- 172.21.15.39:0/2949854088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 msgr2=0x555fa92dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:47.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88effff700 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92dbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88effff700 1 -- 172.21.15.39:0/2949854088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa914b9d0 con 0x555fa92e6c40 2024-05-04T03:21:47.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88eeffd700 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dc420 0x555fa92e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:47.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.481+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:47.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.482+0000 7f88effff700 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e9060 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f88e8003eb0 tx=0x7f88e8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1b13a744e2c46d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:47.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.482+0000 7f88d5ffb700 1 -- 172.21.15.39:0/2949854088 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8019020 con 0x555fa92e6c40 2024-05-04T03:21:47.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.482+0000 7f88d5ffb700 1 -- 172.21.15.39:0/2949854088 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88e80092d0 con 0x555fa92e6c40 2024-05-04T03:21:47.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.482+0000 7f88d5ffb700 1 -- 172.21.15.39:0/2949854088 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e800e750 con 0x555fa92e6c40 2024-05-04T03:21:47.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 -- 172.21.15.39:0/2949854088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 msgr2=0x555fa92e9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e9060 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f88e8003eb0 tx=0x7f88e8009920 comp rx=0 tx=0).stop 2024-05-04T03:21:47.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 -- 172.21.15.39:0/2949854088 shutdown_connections 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e9060 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 --2- 172.21.15.39:0/2949854088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dc420 0x555fa92e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 -- 172.21.15.39:0/2949854088 >> 172.21.15.39:0/2949854088 conn(0x555fa915ef40 msgr2=0x555fa92d79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 -- 172.21.15.39:0/2949854088 shutdown_connections 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 -- 172.21.15.39:0/2949854088 wait complete. 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.483+0000 7f89049c9b80 1 -- start start 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f89049c9b80 1 Processor -- start 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dbad0 0x555fa92e3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dc420 0x555fa92e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f89049c9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555fa92ede20 con 0x555fa92e6c40 2024-05-04T03:21:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f88effff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f88effff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55102/0 (socket says 172.21.15.39:55102) 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f88effff700 1 -- 172.21.15.39:0/218366804 learned_addr learned my addr 172.21.15.39:0/218366804 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555fa91b7fd0 con 0x555fa92dc420 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555fa92edfa0 con 0x555fa92dbad0 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f89049c9b80 10 client.0 _ll_register_callbacks cb 0x555fa9196a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f88eeffd700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dc420 0x555fa92e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.484+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dbad0 0x555fa92e3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88eeffd700 1 -- 172.21.15.39:0/218366804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dbad0 msgr2=0x555fa92e3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88eeffd700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dbad0 0x555fa92e3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88eeffd700 1 -- 172.21.15.39:0/218366804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 msgr2=0x555fa92e47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88eeffd700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88eeffd700 1 -- 172.21.15.39:0/218366804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa91b5860 con 0x555fa92dc420 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88effff700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dbad0 0x555fa92e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:47.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88eeffd700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dc420 0x555fa92e42a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x555fa92d9050 tx=0x7f88e40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef83d118301d80c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88ceffd700 1 -- 172.21.15.39:0/218366804 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4004240 con 0x555fa92dc420 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e80092d0 con 0x555fa92dc420 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88ceffd700 1 -- 172.21.15.39:0/218366804 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f88e4009040 con 0x555fa92dc420 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f88ceffd700 1 -- 172.21.15.39:0/218366804 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88e40043e0 con 0x555fa92dc420 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f89049c9b80 10 client.26102 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.485+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555fa92f8c50 con 0x555fa92dc420 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f88ceffd700 1 -- 172.21.15.39:0/218366804 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88e80092d0 con 0x555fa92dc420 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f88ceffd700 1 -- 172.21.15.39:0/218366804 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f88e401c6a0 con 0x555fa92dc420 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f89049c9b80 10 client.26102 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f89049c9b80 10 client.26102 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f89049c9b80 20 client.26102 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f89049c9b80 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555fa92fb1e0 0x555fa92fd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f89049c9b80 4 client.26102 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fa920c420 con 0x555fa92fb1e0 2024-05-04T03:21:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.486+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555fa92fb1e0 0x555fa92fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.487+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555fa92fb1e0 0x555fa92fd600 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0374f61bda0229 server_cookie=2b36318239008533 in_seq=0 out_seq=0 2024-05-04T03:21:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.487+0000 7f88ceffd700 10 client.26102 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.487+0000 7f88ceffd700 1 -- 172.21.15.39:0/218366804 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555fa920c420 con 0x555fa92fb1e0 2024-05-04T03:21:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.487+0000 7f88ceffd700 10 client.26102 handle_command_reply: tid=0 2024-05-04T03:21:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.487+0000 7f89049c9b80 1 client.26102 shutdown 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dc420 msgr2=0x555fa92e42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dc420 0x555fa92e42a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x555fa92d9050 tx=0x7f88e40047c0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 shutdown_connections 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555fa92fb1e0 0x555fa92fd600 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92e6c40 0x555fa92e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92dbad0 0x555fa92e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f88ceffd700 0 client.26102 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 --2- 172.21.15.39:0/218366804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dc420 0x555fa92e42a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 >> 172.21.15.39:0/218366804 conn(0x555fa915ef40 msgr2=0x555fa92dd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 shutdown_connections 2024-05-04T03:21:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 1 -- 172.21.15.39:0/218366804 wait complete. 2024-05-04T03:21:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.488+0000 7f89049c9b80 20 client.26102 trim_cache size 0 max 16384 2024-05-04T03:21:47.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f89049c9b80 1 Processor -- start 2024-05-04T03:21:47.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f89049c9b80 1 -- start start 2024-05-04T03:21:47.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa9302dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa9303300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 0x555fa92e1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f89049c9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01ac880 con 0x555fa92dc420 2024-05-04T03:21:47.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f89049c9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01ae3c0 con 0x555fa92dbad0 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f89049c9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01acf50 con 0x555fa92e6c40 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa9302dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa9302dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46458/0 (socket says 172.21.15.39:46458) 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f88eeffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa9303300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.523+0000 7f88ef7fe700 1 -- 172.21.15.39:0/1154303280 learned_addr learned my addr 172.21.15.39:0/1154303280 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88effff700 1 --2- 172.21.15.39:0/1154303280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 0x555fa92e1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88ef7fe700 1 -- 172.21.15.39:0/1154303280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 msgr2=0x555fa92e1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/1154303280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 0x555fa92e1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88ef7fe700 1 -- 172.21.15.39:0/1154303280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 msgr2=0x555fa9303300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/1154303280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa9303300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88ef7fe700 1 -- 172.21.15.39:0/1154303280 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa92e22f0 con 0x555fa92dbad0 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/1154303280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa9302dc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88e00e09a0 tx=0x7f88e0092040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3c66fe662cd164 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88d57fa700 1 -- 172.21.15.39:0/1154303280 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e00ae590 con 0x555fa92dbad0 2024-05-04T03:21:47.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88d57fa700 1 -- 172.21.15.39:0/1154303280 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88e00b2fb0 con 0x555fa92dbad0 2024-05-04T03:21:47.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.524+0000 7f88d57fa700 1 -- 172.21.15.39:0/1154303280 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e00a5e20 con 0x555fa92dbad0 2024-05-04T03:21:47.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 -- 172.21.15.39:0/1154303280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 msgr2=0x555fa9302dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 --2- 172.21.15.39:0/1154303280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa9302dc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88e00e09a0 tx=0x7f88e0092040 comp rx=0 tx=0).stop 2024-05-04T03:21:47.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 -- 172.21.15.39:0/1154303280 shutdown_connections 2024-05-04T03:21:47.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 --2- 172.21.15.39:0/1154303280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 0x555fa92e1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 --2- 172.21.15.39:0/1154303280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa9302dc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 --2- 172.21.15.39:0/1154303280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa9303300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 -- 172.21.15.39:0/1154303280 >> 172.21.15.39:0/1154303280 conn(0x555fa915ef40 msgr2=0x555fa92d7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 -- 172.21.15.39:0/1154303280 shutdown_connections 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.525+0000 7f89049c9b80 1 -- 172.21.15.39:0/1154303280 wait complete. 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.526+0000 7f89049c9b80 1 -- start start 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.526+0000 7f89049c9b80 1 Processor -- start 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.526+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.526+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa92f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.526+0000 7f89049c9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 0x555fa92f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.526+0000 7f89049c9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01adcf0 con 0x555fa92dc420 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.526+0000 7f89049c9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01ad620 con 0x555fa92dbad0 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.526+0000 7f89049c9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01aa820 con 0x555fa92e6c40 2024-05-04T03:21:47.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f89049c9b80 10 client.0 _ll_register_callbacks cb 0x555fa9196a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46474/0 (socket says 172.21.15.39:46474) 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 -- 172.21.15.39:0/3233995275 learned_addr learned my addr 172.21.15.39:0/3233995275 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88eeffd700 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa92f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88effff700 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 0x555fa92f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 -- 172.21.15.39:0/3233995275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 msgr2=0x555fa92f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 0x555fa92f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 -- 172.21.15.39:0/3233995275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 msgr2=0x555fa92f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa92f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 -- 172.21.15.39:0/3233995275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa92f2a30 con 0x555fa92dbad0 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88eeffd700 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa92f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ef7fe700 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92f38d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f88e00a62d0 tx=0x7f88e000f5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689f01769bb7c350 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.527+0000 7f88ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f88ceffd700 1 -- 172.21.15.39:0/3233995275 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e006ab00 con 0x555fa92dbad0 2024-05-04T03:21:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f88ceffd700 1 -- 172.21.15.39:0/3233995275 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f88e00e0af0 con 0x555fa92dbad0 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 1 -- 172.21.15.39:0/3233995275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e00b2fb0 con 0x555fa92dbad0 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f88ceffd700 1 -- 172.21.15.39:0/3233995275 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f88e00e0cf0 con 0x555fa92dbad0 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 10 client.26108 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 1 -- 172.21.15.39:0/3233995275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555fa93669f0 con 0x555fa92dbad0 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f88ceffd700 1 -- 172.21.15.39:0/3233995275 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88e00b2fb0 con 0x555fa92dbad0 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f88ceffd700 1 -- 172.21.15.39:0/3233995275 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f88e00c4a40 con 0x555fa92dbad0 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 10 client.26108 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 10 client.26108 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 20 client.26108 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555fa9367100 0x555fa9369520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 4 client.26108 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8a6f08e7", "release": true, "await": true}] 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.528+0000 7f89049c9b80 1 -- 172.21.15.39:0/3233995275 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8a6f08e7", "release": true, "await": true}) -- 0x555fa92064c0 con 0x555fa9367100 2024-05-04T03:21:47.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.529+0000 7f88eeffd700 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555fa9367100 0x555fa9369520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:47.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.529+0000 7f88ceffd700 10 client.26108 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:47.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.529+0000 7f88eeffd700 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555fa9367100 0x555fa9369520 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222029cbd2afdda0 server_cookie=6f7fa2ff0b4f5553 in_seq=0 out_seq=0 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.534+0000 7f88ceffd700 1 -- 172.21.15.39:0/3233995275 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555fa92064c0 con 0x555fa9367100 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.534+0000 7f88ceffd700 10 client.26108 handle_command_reply: tid=0 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.534+0000 7f89049c9b80 1 client.26108 shutdown 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.534+0000 7f89049c9b80 1 -- 172.21.15.39:0/3233995275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 msgr2=0x555fa92f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.534+0000 7f89049c9b80 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92f38d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f88e00a62d0 tx=0x7f88e000f5b0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.534+0000 7f89049c9b80 1 -- 172.21.15.39:0/3233995275 shutdown_connections 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f89049c9b80 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555fa9367100 0x555fa9369520 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f89049c9b80 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555fa92e6c40 0x555fa92f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f89049c9b80 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555fa92dbad0 0x555fa92f38d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f89049c9b80 1 --2- 172.21.15.39:0/3233995275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555fa92dc420 0x555fa92f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f89049c9b80 1 -- 172.21.15.39:0/3233995275 >> 172.21.15.39:0/3233995275 conn(0x555fa915ef40 msgr2=0x555fa92f5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f88ceffd700 0 client.26108 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f89049c9b80 1 -- 172.21.15.39:0/3233995275 shutdown_connections 2024-05-04T03:21:47.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f89049c9b80 1 -- 172.21.15.39:0/3233995275 wait complete. 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f89049c9b80 20 client.26108 trim_cache size 0 max 16384 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 28, 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "8a6f08e7": { 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 28, 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:21:47.538 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 72.7, 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:21:47.539 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:21:47.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f88f5fb5700 1 -- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f88d80b2880 msgr2=0x7f88d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.535+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f88d80b2880 0x7f88d80b4d90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f88f01067b0 tx=0x7f88e4009260 comp rx=0 tx=0).stop 2024-05-04T03:21:47.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 -- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 msgr2=0x7f88f019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:47.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 0x7f88f019ad30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f88e0006010 tx=0x7f88e0004270 comp rx=0 tx=0).stop 2024-05-04T03:21:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 -- 172.21.15.39:0/2119128261 shutdown_connections 2024-05-04T03:21:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f88d80b2880 0x7f88d80b4d90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88f0103520 0x7f88f019ad30 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0108750 0x7f88f019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 --2- 172.21.15.39:0/2119128261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88f0107e00 0x7f88f019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 -- 172.21.15.39:0/2119128261 >> 172.21.15.39:0/2119128261 conn(0x7f88f00fef90 msgr2=0x7f88f01096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 -- 172.21.15.39:0/2119128261 shutdown_connections 2024-05-04T03:21:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:21:47.536+0000 7f88f5fb5700 1 -- 172.21.15.39:0/2119128261 wait complete. 2024-05-04T03:21:48.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:48 smithi039 ceph-mon[29715]: pgmap v716: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 203 KiB/s wr, 36 op/s 2024-05-04T03:21:48.491 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8a6f08e7', seconds elapsed: 30.7 2024-05-04T03:21:48.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:48 smithi167 ceph-mon[33614]: pgmap v716: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 203 KiB/s wr, 36 op/s 2024-05-04T03:21:48.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:48 smithi066 ceph-mon[33264]: pgmap v716: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 203 KiB/s wr, 36 op/s 2024-05-04T03:21:50.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:50 smithi167 ceph-mon[33614]: pgmap v717: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 476 KiB/s wr, 57 op/s 2024-05-04T03:21:50.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:50 smithi039 ceph-mon[29715]: pgmap v717: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 476 KiB/s wr, 57 op/s 2024-05-04T03:21:50.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:50 smithi066 ceph-mon[33264]: pgmap v717: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 476 KiB/s wr, 57 op/s 2024-05-04T03:21:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 480.0 s, 1086.4 tps, lat 429.143 ms stddev 2569.302 2024-05-04T03:21:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:52 smithi167 ceph-mon[33614]: pgmap v718: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-05-04T03:21:52.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:52 smithi039 ceph-mon[29715]: pgmap v718: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-05-04T03:21:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:52 smithi066 ceph-mon[33264]: pgmap v718: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-05-04T03:21:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 -- 172.21.15.39:0/1057637928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f755c09eb00 msgr2=0x7f755c0a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 --2- 172.21.15.39:0/1057637928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f755c09eb00 0x7f755c0a0f20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7564069180 tx=0x7f7564097920 comp rx=0 tx=0).stop 2024-05-04T03:21:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 -- 172.21.15.39:0/1057637928 shutdown_connections 2024-05-04T03:21:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 --2- 172.21.15.39:0/1057637928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f755c09eb00 0x7f755c0a0f20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 --2- 172.21.15.39:0/1057637928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c099e00 0x7f755c09a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 --2- 172.21.15.39:0/1057637928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f755c098bc0 0x7f755c0998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 -- 172.21.15.39:0/1057637928 >> 172.21.15.39:0/1057637928 conn(0x7f755c093dd0 msgr2=0x7f755c096260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 -- 172.21.15.39:0/1057637928 shutdown_connections 2024-05-04T03:21:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.560+0000 7f756a889640 1 -- 172.21.15.39:0/1057637928 wait complete. 2024-05-04T03:21:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.561+0000 7f756a889640 1 Processor -- start 2024-05-04T03:21:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.561+0000 7f756a889640 1 -- start start 2024-05-04T03:21:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f756a889640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f755c098bc0 0x7f755c133580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f756a889640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c099e00 0x7f755c133ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f756a889640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f755c09eb00 0x7f755c13aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f756a889640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f755c0a44b0 con 0x7f755c098bc0 2024-05-04T03:21:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f756a889640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f755c0a4330 con 0x7f755c09eb00 2024-05-04T03:21:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f756a889640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f755c0a4630 con 0x7f755c099e00 2024-05-04T03:21:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f7569086640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c099e00 0x7f755c133ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f7569086640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c099e00 0x7f755c133ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53980/0 (socket says 172.21.15.39:53980) 2024-05-04T03:21:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f7569086640 1 -- 172.21.15.39:0/1805854893 learned_addr learned my addr 172.21.15.39:0/1805854893 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f756a088640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f755c09eb00 0x7f755c13aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.562+0000 7f7569887640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f755c098bc0 0x7f755c133580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f7569086640 1 -- 172.21.15.39:0/1805854893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f755c09eb00 msgr2=0x7f755c13aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f7569086640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f755c09eb00 0x7f755c13aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f7569086640 1 -- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f755c098bc0 msgr2=0x7f755c133580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f7569086640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f755c098bc0 0x7f755c133580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f7569086640 1 -- 172.21.15.39:0/1805854893 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c129cc0 con 0x7f755c099e00 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f7569086640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c099e00 0x7f755c133ac0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f755400e9f0 tx=0x7f755400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f755affd640 1 -- 172.21.15.39:0/1805854893 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755400cd80 con 0x7f755c099e00 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f755affd640 1 -- 172.21.15.39:0/1805854893 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75540108c0 con 0x7f755c099e00 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.563+0000 7f756a889640 1 -- 172.21.15.39:0/1805854893 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755c129fb0 con 0x7f755c099e00 2024-05-04T03:21:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.564+0000 7f756a889640 1 -- 172.21.15.39:0/1805854893 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755c12a500 con 0x7f755c099e00 2024-05-04T03:21:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.564+0000 7f755affd640 1 -- 172.21.15.39:0/1805854893 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540187c0 con 0x7f755c099e00 2024-05-04T03:21:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f757299d740 1 Processor -- start 2024-05-04T03:21:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f755affd640 1 -- 172.21.15.39:0/1805854893 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7554010430 con 0x7f755c099e00 2024-05-04T03:21:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f757299d740 1 -- start start 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f757299d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5c31b0 0x55599c964c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f757299d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f757299d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5a24d0 0x55599c96c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f757299d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55599c8c95d0 con 0x55599c59a2f0 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f757299d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55599c882890 con 0x55599c5a24d0 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.568+0000 7f757299d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55599c8c5590 con 0x55599c5c31b0 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.569+0000 7f756a088640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5c31b0 0x55599c964c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.569+0000 7f756a088640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5c31b0 0x55599c964c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53990/0 (socket says 172.21.15.39:53990) 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.569+0000 7f756a088640 1 -- 172.21.15.39:0/2482687083 learned_addr learned my addr 172.21.15.39:0/2482687083 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.569+0000 7f7569887640 1 --2- 172.21.15.39:0/2482687083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.569+0000 7f755affd640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f75500b2980 0x7f75500b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f7569887640 1 -- 172.21.15.39:0/2482687083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5c31b0 msgr2=0x55599c964c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f7569887640 1 --2- 172.21.15.39:0/2482687083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5c31b0 0x55599c964c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f7569887640 1 -- 172.21.15.39:0/2482687083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5a24d0 msgr2=0x55599c96c0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:21:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f7569887640 1 --2- 172.21.15.39:0/2482687083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5a24d0 0x55599c96c0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f7569887640 1 -- 172.21.15.39:0/2482687083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55599c5db210 con 0x55599c59a2f0 2024-05-04T03:21:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f755affd640 1 -- 172.21.15.39:0/1805854893 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f75540e2570 con 0x7f755c099e00 2024-05-04T03:21:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f7569887640 1 --2- 172.21.15.39:0/2482687083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9651b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f755c098490 tx=0x7f756000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6607bd2b7703b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f756a088640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f75500b2980 0x7f75500b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f754ffff640 1 -- 172.21.15.39:0/2482687083 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756000dea0 con 0x55599c59a2f0 2024-05-04T03:21:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f754ffff640 1 -- 172.21.15.39:0/2482687083 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75600035f0 con 0x55599c59a2f0 2024-05-04T03:21:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.570+0000 7f754ffff640 1 -- 172.21.15.39:0/2482687083 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560011680 con 0x55599c59a2f0 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 -- 172.21.15.39:0/2482687083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 msgr2=0x55599c9651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 --2- 172.21.15.39:0/2482687083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9651b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f755c098490 tx=0x7f756000f730 comp rx=0 tx=0).stop 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f756a088640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f75500b2980 0x7f75500b4e90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7564051620 tx=0x7f75640a0040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 -- 172.21.15.39:0/2482687083 shutdown_connections 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 --2- 172.21.15.39:0/2482687083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5a24d0 0x55599c96c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 --2- 172.21.15.39:0/2482687083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9651b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 --2- 172.21.15.39:0/2482687083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5c31b0 0x55599c964c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 -- 172.21.15.39:0/2482687083 >> 172.21.15.39:0/2482687083 conn(0x55599c5b7300 msgr2=0x55599c9605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 -- 172.21.15.39:0/2482687083 shutdown_connections 2024-05-04T03:21:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.572+0000 7f757299d740 1 -- 172.21.15.39:0/2482687083 wait complete. 2024-05-04T03:21:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.573+0000 7f757299d740 1 -- start start 2024-05-04T03:21:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f757299d740 1 Processor -- start 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f757299d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c59a2f0 0x55599c96e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f757299d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c96ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f757299d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5c31b0 0x55599c975e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f757299d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55599c96cb00 con 0x55599c5c31b0 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f757299d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55599c886370 con 0x55599c59a2f0 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f757299d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55599c96cc80 con 0x55599c5a24d0 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f757299d740 10 client.0 _ll_register_callbacks cb 0x55599c5daa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.574+0000 7f756a088640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5c31b0 0x55599c975e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f756a088640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5c31b0 0x55599c975e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55150/0 (socket says 172.21.15.39:55150) 2024-05-04T03:21:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f756a088640 1 -- 172.21.15.39:0/3658472437 learned_addr learned my addr 172.21.15.39:0/3658472437 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f7569887640 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c59a2f0 0x55599c96e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f7569086640 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c96ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f756a088640 1 -- 172.21.15.39:0/3658472437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 msgr2=0x55599c96ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f756a088640 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c96ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f756a088640 1 -- 172.21.15.39:0/3658472437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c59a2f0 msgr2=0x55599c96e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f756a088640 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c59a2f0 0x55599c96e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.575+0000 7f756a088640 1 -- 172.21.15.39:0/3658472437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55599c6a7620 con 0x55599c5c31b0 2024-05-04T03:21:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.576+0000 7f756a088640 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5c31b0 0x55599c975e60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7564065d90 tx=0x7f7564068f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ea8380cb161502 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.577+0000 7f754d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:21:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.577+0000 7f754d7fa640 1 -- 172.21.15.39:0/3658472437 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640a3040 con 0x55599c5c31b0 2024-05-04T03:21:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.577+0000 7f754d7fa640 1 -- 172.21.15.39:0/3658472437 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f756409ec10 con 0x55599c5c31b0 2024-05-04T03:21:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.577+0000 7f757299d740 1 -- 172.21.15.39:0/3658472437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75600035f0 con 0x55599c5c31b0 2024-05-04T03:21:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.578+0000 7f754d7fa640 1 -- 172.21.15.39:0/3658472437 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f75640ac480 con 0x55599c5c31b0 2024-05-04T03:21:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.578+0000 7f757299d740 10 client.16536 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.578+0000 7f757299d740 1 -- 172.21.15.39:0/3658472437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55599c978eb0 con 0x55599c5c31b0 2024-05-04T03:21:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.578+0000 7f754d7fa640 1 -- 172.21.15.39:0/3658472437 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75600035f0 con 0x55599c5c31b0 2024-05-04T03:21:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.579+0000 7f754d7fa640 1 -- 172.21.15.39:0/3658472437 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f75640bb5c0 con 0x55599c5c31b0 2024-05-04T03:21:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.579+0000 7f757299d740 10 client.16536 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.579+0000 7f757299d740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.579+0000 7f757299d740 20 client.16536 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.579+0000 7f757299d740 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55599c581d80 0x55599c97c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.579+0000 7f757299d740 4 client.16536 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.579+0000 7f757299d740 1 -- 172.21.15.39:0/3658472437 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55599c8d71e0 con 0x55599c581d80 2024-05-04T03:21:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.580+0000 7f7569887640 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55599c581d80 0x55599c97c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.580+0000 7f7569887640 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55599c581d80 0x55599c97c5a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941a3ea63da7b317 server_cookie=2f086791656e181a in_seq=0 out_seq=0 2024-05-04T03:21:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.580+0000 7f754d7fa640 10 client.16536 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.581+0000 7f754d7fa640 1 -- 172.21.15.39:0/3658472437 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55599c8d71e0 con 0x55599c581d80 2024-05-04T03:21:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.581+0000 7f754d7fa640 10 client.16536 handle_command_reply: tid=0 2024-05-04T03:21:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.581+0000 7f757299d740 1 client.16536 shutdown 2024-05-04T03:21:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.583+0000 7f757299d740 1 -- 172.21.15.39:0/3658472437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5c31b0 msgr2=0x55599c975e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.583+0000 7f757299d740 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5c31b0 0x55599c975e60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7564065d90 tx=0x7f7564068f90 comp rx=0 tx=0).stop 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 1 -- 172.21.15.39:0/3658472437 shutdown_connections 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55599c581d80 0x55599c97c5a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5c31b0 0x55599c975e60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c96ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 1 --2- 172.21.15.39:0/3658472437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c59a2f0 0x55599c96e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 1 -- 172.21.15.39:0/3658472437 >> 172.21.15.39:0/3658472437 conn(0x55599c5b7300 msgr2=0x55599c9634d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f754d7fa640 0 client.16536 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 1 -- 172.21.15.39:0/3658472437 shutdown_connections 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 1 -- 172.21.15.39:0/3658472437 wait complete. 2024-05-04T03:21:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.585+0000 7f757299d740 20 client.16536 trim_cache size 0 max 16384 2024-05-04T03:21:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.614+0000 7f757299d740 1 Processor -- start 2024-05-04T03:21:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.614+0000 7f757299d740 1 -- start start 2024-05-04T03:21:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.614+0000 7f757299d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.614+0000 7f757299d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c963900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.614+0000 7f757299d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c95fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f757299d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f755c14cec0 con 0x55599c59a2f0 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f757299d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f755c14ea00 con 0x55599c5c31b0 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f757299d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f755c14d590 con 0x55599c5a24d0 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569086640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c963900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569086640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c963900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53996/0 (socket says 172.21.15.39:53996) 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569086640 1 -- 172.21.15.39:0/3030847283 learned_addr learned my addr 172.21.15.39:0/3030847283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f756a088640 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c95fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569887640 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569086640 1 -- 172.21.15.39:0/3030847283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 msgr2=0x55599c95fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569086640 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c95fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569086640 1 -- 172.21.15.39:0/3030847283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 msgr2=0x55599c9633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569086640 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.615+0000 7f7569086640 1 -- 172.21.15.39:0/3030847283 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55599c95a6e0 con 0x55599c5a24d0 2024-05-04T03:21:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.616+0000 7f7569086640 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c963900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75540a9ad0 tx=0x7f7554010cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f8be0799257e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.616+0000 7f754f7fe640 1 -- 172.21.15.39:0/3030847283 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540aa240 con 0x55599c5a24d0 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.616+0000 7f754f7fe640 1 -- 172.21.15.39:0/3030847283 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75540a9b40 con 0x55599c5a24d0 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.616+0000 7f754f7fe640 1 -- 172.21.15.39:0/3030847283 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540a5dd0 con 0x55599c5a24d0 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.617+0000 7f757299d740 1 -- 172.21.15.39:0/3030847283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 msgr2=0x55599c963900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.617+0000 7f757299d740 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c963900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75540a9ad0 tx=0x7f7554010cc0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.618+0000 7f757299d740 1 -- 172.21.15.39:0/3030847283 shutdown_connections 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.618+0000 7f757299d740 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c95fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.618+0000 7f757299d740 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c5a24d0 0x55599c963900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.618+0000 7f757299d740 1 --2- 172.21.15.39:0/3030847283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c59a2f0 0x55599c9633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.618+0000 7f757299d740 1 -- 172.21.15.39:0/3030847283 >> 172.21.15.39:0/3030847283 conn(0x55599c5b7300 msgr2=0x55599c968d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.618+0000 7f757299d740 1 -- 172.21.15.39:0/3030847283 shutdown_connections 2024-05-04T03:21:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.618+0000 7f757299d740 1 -- 172.21.15.39:0/3030847283 wait complete. 2024-05-04T03:21:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.618+0000 7f757299d740 1 -- start start 2024-05-04T03:21:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.619+0000 7f757299d740 1 Processor -- start 2024-05-04T03:21:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.619+0000 7f757299d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c59a2f0 0x55599c974f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.619+0000 7f757299d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5a24d0 0x55599c971b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.619+0000 7f757299d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c972040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.619+0000 7f757299d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f755c14e330 con 0x55599c5a24d0 2024-05-04T03:21:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.619+0000 7f757299d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f755c14dc60 con 0x55599c5c31b0 2024-05-04T03:21:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.619+0000 7f757299d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f755c131370 con 0x55599c59a2f0 2024-05-04T03:21:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.619+0000 7f757299d740 10 client.0 _ll_register_callbacks cb 0x7f7554005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c59a2f0 0x55599c974f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f756a088640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c972040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c59a2f0 0x55599c974f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:54004/0 (socket says 172.21.15.39:54004) 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569086640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5a24d0 0x55599c971b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 -- 172.21.15.39:0/3233935998 learned_addr learned my addr 172.21.15.39:0/3233935998 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 -- 172.21.15.39:0/3233935998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 msgr2=0x55599c972040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c972040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 -- 172.21.15.39:0/3233935998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5a24d0 msgr2=0x55599c971b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5a24d0 0x55599c971b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 -- 172.21.15.39:0/3233935998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55599c975620 con 0x55599c59a2f0 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f756a088640 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c972040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.620+0000 7f7569887640 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c59a2f0 0x55599c974f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55599c966010 tx=0x7f756000d750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cdacb496d46110e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.621+0000 7f754d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:21:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.621+0000 7f754d7fa640 1 -- 172.21.15.39:0/3233935998 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560004250 con 0x55599c59a2f0 2024-05-04T03:21:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.621+0000 7f757299d740 1 -- 172.21.15.39:0/3233935998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75540a9b40 con 0x55599c59a2f0 2024-05-04T03:21:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.621+0000 7f754d7fa640 1 -- 172.21.15.39:0/3233935998 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f75600043f0 con 0x55599c59a2f0 2024-05-04T03:21:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.622+0000 7f754d7fa640 1 -- 172.21.15.39:0/3233935998 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7560005040 con 0x55599c59a2f0 2024-05-04T03:21:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.622+0000 7f757299d740 10 client.26158 fetch_fsmap learned FSMap version 32 2024-05-04T03:21:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.622+0000 7f757299d740 1 -- 172.21.15.39:0/3233935998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55599c9e3770 con 0x55599c59a2f0 2024-05-04T03:21:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f754d7fa640 1 -- 172.21.15.39:0/3233935998 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75540a9b40 con 0x55599c59a2f0 2024-05-04T03:21:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f754d7fa640 1 -- 172.21.15.39:0/3233935998 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7560004ce0 con 0x55599c59a2f0 2024-05-04T03:21:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f757299d740 10 client.26158 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:21:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f757299d740 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f757299d740 20 client.26158 populate_metadata read hostname 'smithi039' 2024-05-04T03:21:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f757299d740 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55599c5571c0 0x55599c9e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f757299d740 4 client.26158 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f757299d740 1 -- 172.21.15.39:0/3233935998 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55599c8d1050 con 0x55599c5571c0 2024-05-04T03:21:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.623+0000 7f7569086640 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55599c5571c0 0x55599c9e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.624+0000 7f754d7fa640 10 client.26158 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.624+0000 7f7569086640 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55599c5571c0 0x55599c9e5e30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0b41c129e81756 server_cookie=e0f9845cb6df5124 in_seq=0 out_seq=0 2024-05-04T03:21:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.635+0000 7f754d7fa640 1 -- 172.21.15.39:0/3233935998 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55599c8d1050 con 0x55599c5571c0 2024-05-04T03:21:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.635+0000 7f754d7fa640 10 client.26158 handle_command_reply: tid=0 2024-05-04T03:21:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 client.26158 shutdown 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 -- 172.21.15.39:0/3233935998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c59a2f0 msgr2=0x55599c974f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c59a2f0 0x55599c974f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55599c966010 tx=0x7f756000d750 comp rx=0 tx=0).stop 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 -- 172.21.15.39:0/3233935998 shutdown_connections 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55599c5571c0 0x55599c9e5e30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55599c5c31b0 0x55599c972040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55599c5a24d0 0x55599c971b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 --2- 172.21.15.39:0/3233935998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55599c59a2f0 0x55599c974f80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f757299d740 1 -- 172.21.15.39:0/3233935998 >> 172.21.15.39:0/3233935998 conn(0x55599c5b7300 msgr2=0x55599c9633c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.636+0000 7f754d7fa640 0 client.26158 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.637+0000 7f757299d740 1 -- 172.21.15.39:0/3233935998 shutdown_connections 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.637+0000 7f757299d740 1 -- 172.21.15.39:0/3233935998 wait complete. 2024-05-04T03:21:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.637+0000 7f757299d740 20 client.26158 trim_cache size 0 max 16384 2024-05-04T03:21:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:21:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:21:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:21:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:21:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 -- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f75500b2980 msgr2=0x7f75500b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f75500b2980 0x7f75500b4e90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7564051620 tx=0x7f75640a0040 comp rx=0 tx=0).stop 2024-05-04T03:21:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 -- 172.21.15.39:0/1805854893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c099e00 msgr2=0x7f755c133ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c099e00 0x7f755c133ac0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f755400e9f0 tx=0x7f755400eeb0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 -- 172.21.15.39:0/1805854893 shutdown_connections 2024-05-04T03:21:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f75500b2980 0x7f75500b4e90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f755c09eb00 0x7f755c13aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f755c099e00 0x7f755c133ac0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 --2- 172.21.15.39:0/1805854893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f755c098bc0 0x7f755c133580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.638+0000 7f756a889640 1 -- 172.21.15.39:0/1805854893 >> 172.21.15.39:0/1805854893 conn(0x7f755c093dd0 msgr2=0x7f755c09aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.639+0000 7f756a889640 1 -- 172.21.15.39:0/1805854893 shutdown_connections 2024-05-04T03:21:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:21:53.639+0000 7f756a889640 1 -- 172.21.15.39:0/1805854893 wait complete. 2024-05-04T03:21:54.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:54 smithi167 ceph-mon[33614]: pgmap v719: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 420 op/s 2024-05-04T03:21:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:54 smithi039 ceph-mon[29715]: pgmap v719: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 420 op/s 2024-05-04T03:21:54.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:54 smithi066 ceph-mon[33264]: pgmap v719: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 420 op/s 2024-05-04T03:21:56.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:56 smithi167 ceph-mon[33614]: pgmap v720: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 621 op/s 2024-05-04T03:21:56.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:56 smithi039 ceph-mon[29715]: pgmap v720: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 621 op/s 2024-05-04T03:21:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 485.0 s, 1178.5 tps, lat 84.892 ms stddev 27.431 2024-05-04T03:21:56.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:56 smithi066 ceph-mon[33264]: pgmap v720: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 621 op/s 2024-05-04T03:21:58.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:21:58 smithi039 ceph-mon[29715]: pgmap v721: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 MiB/s wr, 890 op/s 2024-05-04T03:21:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:21:58 smithi167 ceph-mon[33614]: pgmap v721: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 MiB/s wr, 890 op/s 2024-05-04T03:21:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:21:58 smithi066 ceph-mon[33264]: pgmap v721: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 MiB/s wr, 890 op/s 2024-05-04T03:22:00.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:00 smithi167 ceph-mon[33614]: pgmap v722: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-05-04T03:22:00.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:00 smithi039 ceph-mon[29715]: pgmap v722: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-05-04T03:22:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:00 smithi066 ceph-mon[33264]: pgmap v722: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-05-04T03:22:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 490.0 s, 1311.2 tps, lat 76.155 ms stddev 22.791 2024-05-04T03:22:02.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:02 smithi167 ceph-mon[33614]: pgmap v723: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 1.31k op/s 2024-05-04T03:22:02.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:22:02.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:02.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:02 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:02.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:02 smithi039 ceph-mon[29715]: pgmap v723: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 1.31k op/s 2024-05-04T03:22:02.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:22:02.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:02.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:02 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:02 smithi066 ceph-mon[33264]: pgmap v723: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 1.31k op/s 2024-05-04T03:22:02.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:22:02.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:02.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:02 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:03.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:22:03.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:22:03.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:03.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:22:03.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:22:03.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:03.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:22:03.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:22:03.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:22:04.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:04 smithi167 ceph-mon[33614]: pgmap v724: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 25 MiB/s wr, 1.54k op/s 2024-05-04T03:22:04.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:04 smithi039 ceph-mon[29715]: pgmap v724: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 25 MiB/s wr, 1.54k op/s 2024-05-04T03:22:04.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:04 smithi066 ceph-mon[33264]: pgmap v724: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 25 MiB/s wr, 1.54k op/s 2024-05-04T03:22:06.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:06 smithi167 ceph-mon[33614]: pgmap v725: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 23 MiB/s wr, 1.44k op/s 2024-05-04T03:22:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:06 smithi039 ceph-mon[29715]: pgmap v725: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 23 MiB/s wr, 1.44k op/s 2024-05-04T03:22:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 495.0 s, 1020.0 tps, lat 97.961 ms stddev 125.327 2024-05-04T03:22:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 -- 172.21.15.39:0/1085266056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b0410a0f0 msgr2=0x7f7b0410a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/1085266056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b0410a0f0 0x7f7b0410a500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009be0 tx=0x7f7af402a6c0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 -- 172.21.15.39:0/1085266056 shutdown_connections 2024-05-04T03:22:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/1085266056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b041121f0 0x7f7b04114690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/1085266056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0410f890 0x7f7b04111cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/1085266056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b0410a0f0 0x7f7b0410a500 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 -- 172.21.15.39:0/1085266056 >> 172.21.15.39:0/1085266056 conn(0x7f7b04070a90 msgr2=0x7f7b04072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 -- 172.21.15.39:0/1085266056 shutdown_connections 2024-05-04T03:22:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.797+0000 7f7b0ace9640 1 -- 172.21.15.39:0/1085266056 wait complete. 2024-05-04T03:22:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.798+0000 7f7b0ace9640 1 Processor -- start 2024-05-04T03:22:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.798+0000 7f7b0ace9640 1 -- start start 2024-05-04T03:22:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b0ace9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0410a0f0 0x7f7b041a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b0ace9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0410f890 0x7f7b041a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b0ace9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b041121f0 0x7f7b041aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b0ace9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04117f00 con 0x7f7b0410a0f0 2024-05-04T03:22:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b0ace9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04117d80 con 0x7f7b0410f890 2024-05-04T03:22:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b0ace9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04118080 con 0x7f7b041121f0 2024-05-04T03:22:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b08a85640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0410a0f0 0x7f7b041a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b09286640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b041121f0 0x7f7b041aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b08a85640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0410a0f0 0x7f7b041a2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45326/0 (socket says 172.21.15.39:45326) 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b09286640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b041121f0 0x7f7b041aa080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34054/0 (socket says 172.21.15.39:34054) 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.799+0000 7f7b08a85640 1 -- 172.21.15.39:0/2862034065 learned_addr learned my addr 172.21.15.39:0/2862034065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.800+0000 7f7b09286640 1 -- 172.21.15.39:0/2862034065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0410f890 msgr2=0x7f7b041a30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.800+0000 7f7b09286640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0410f890 0x7f7b041a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.800+0000 7f7b09286640 1 -- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0410a0f0 msgr2=0x7f7b041a2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.800+0000 7f7b09286640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0410a0f0 0x7f7b041a2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.800+0000 7f7b09286640 1 -- 172.21.15.39:0/2862034065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b041aa6f0 con 0x7f7b041121f0 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.800+0000 7f7b08a85640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0410a0f0 0x7f7b041a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.800+0000 7f7b09286640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b041121f0 0x7f7b041aa080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009fd0 tx=0x7f7af800eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.800+0000 7f7b01ffb640 1 -- 172.21.15.39:0/2862034065 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8004220 con 0x7f7b041121f0 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.801+0000 7f7b01ffb640 1 -- 172.21.15.39:0/2862034065 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7af80043c0 con 0x7f7b041121f0 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.801+0000 7f7b0ace9640 1 -- 172.21.15.39:0/2862034065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0410b6d0 con 0x7f7b041121f0 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.801+0000 7f7b01ffb640 1 -- 172.21.15.39:0/2862034065 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af80047a0 con 0x7f7b041121f0 2024-05-04T03:22:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.801+0000 7f7b0ace9640 1 -- 172.21.15.39:0/2862034065 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0410bc20 con 0x7f7b041121f0 2024-05-04T03:22:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.804+0000 7f7b01ffb640 1 -- 172.21.15.39:0/2862034065 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7af800c8b0 con 0x7f7b041121f0 2024-05-04T03:22:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b11b9c740 1 Processor -- start 2024-05-04T03:22:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b11b9c740 1 -- start start 2024-05-04T03:22:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c802270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c8096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56304c75e5d0 con 0x56304c4374d0 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56304c717890 con 0x56304c4581b0 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56304c75a590 con 0x56304c42f2f0 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c802270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c802270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43126/0 (socket says 172.21.15.39:43126) 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b03fff640 1 -- 172.21.15.39:0/4070789306 learned_addr learned my addr 172.21.15.39:0/4070789306 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b03fff640 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c8096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.805+0000 7f7b08a85640 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.806+0000 7f7b03fff640 1 -- 172.21.15.39:0/4070789306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 msgr2=0x56304c8027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.806+0000 7f7b03fff640 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.806+0000 7f7b03fff640 1 -- 172.21.15.39:0/4070789306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 msgr2=0x56304c8096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.806+0000 7f7b03fff640 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c8096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.806+0000 7f7b03fff640 1 -- 172.21.15.39:0/4070789306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304c470210 con 0x56304c4581b0 2024-05-04T03:22:06.811 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.806+0000 7f7b03fff640 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c802270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0148f0 tx=0x7f7afc00a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cdbac03c08d944 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.806+0000 7f7b01ffb640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aec0b28b0 0x7f7aec0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.806+0000 7f7aeaffd640 1 -- 172.21.15.39:0/4070789306 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc004030 con 0x56304c4581b0 2024-05-04T03:22:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.807+0000 7f7aeaffd640 1 -- 172.21.15.39:0/4070789306 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7afc005020 con 0x56304c4581b0 2024-05-04T03:22:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.807+0000 7f7b08a85640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aec0b28b0 0x7f7aec0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.807+0000 7f7b01ffb640 1 -- 172.21.15.39:0/2862034065 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7af80e20f0 con 0x7f7b041121f0 2024-05-04T03:22:06.812 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.807+0000 7f7aeaffd640 1 -- 172.21.15.39:0/4070789306 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc004a00 con 0x56304c4581b0 2024-05-04T03:22:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4070789306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 msgr2=0x56304c802270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b08a85640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aec0b28b0 0x7f7aec0b4dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7af4030800 tx=0x7f7af4031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c802270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0148f0 tx=0x7f7afc00a3f0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4070789306 shutdown_connections 2024-05-04T03:22:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c8096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/4070789306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c802270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4070789306 >> 172.21.15.39:0/4070789306 conn(0x56304c44c300 msgr2=0x56304c7f53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4070789306 shutdown_connections 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.808+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4070789306 wait complete. 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 1 -- start start 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 1 Processor -- start 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c42f2f0 0x56304c80bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c4374d0 0x56304c80c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4581b0 0x56304c813370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56304c80a130 con 0x56304c4581b0 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56304c71b370 con 0x56304c42f2f0 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56304c80a2b0 con 0x56304c4374d0 2024-05-04T03:22:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.809+0000 7f7b11b9c740 10 client.0 _ll_register_callbacks cb 0x56304c46fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c4374d0 0x56304c80c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b08a85640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c42f2f0 0x56304c80bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b08a85640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c42f2f0 0x56304c80bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43136/0 (socket says 172.21.15.39:43136) 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b08a85640 1 -- 172.21.15.39:0/2687392676 learned_addr learned my addr 172.21.15.39:0/2687392676 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b09286640 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4581b0 0x56304c813370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b08a85640 1 -- 172.21.15.39:0/2687392676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c4374d0 msgr2=0x56304c80c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b08a85640 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c4374d0 0x56304c80c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b08a85640 1 -- 172.21.15.39:0/2687392676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4581b0 msgr2=0x56304c813370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b08a85640 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4581b0 0x56304c813370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b08a85640 1 -- 172.21.15.39:0/2687392676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304c53c620 con 0x56304c42f2f0 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.810+0000 7f7b09286640 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4581b0 0x56304c813370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7b08a85640 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c42f2f0 0x56304c80bed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7af402acb0 tx=0x7f7af402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9ee33512348da7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7ae3fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:22:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7ae3fff640 1 -- 172.21.15.39:0/2687392676 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4036050 con 0x56304c42f2f0 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7ae3fff640 1 -- 172.21.15.39:0/2687392676 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7af4004450 con 0x56304c42f2f0 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7b11b9c740 1 -- 172.21.15.39:0/2687392676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc005020 con 0x56304c42f2f0 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7ae3fff640 1 -- 172.21.15.39:0/2687392676 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7af4004650 con 0x56304c42f2f0 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7b11b9c740 10 client.26132 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7b11b9c740 1 -- 172.21.15.39:0/2687392676 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56304c7fc110 con 0x56304c42f2f0 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7ae3fff640 1 -- 172.21.15.39:0/2687392676 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7afc005020 con 0x56304c42f2f0 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.811+0000 7f7ae3fff640 1 -- 172.21.15.39:0/2687392676 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7af403c040 con 0x56304c42f2f0 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.812+0000 7f7b11b9c740 10 client.26132 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.812+0000 7f7b11b9c740 10 client.26132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.812+0000 7f7b11b9c740 20 client.26132 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.812+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56304c416d80 0x56304c7ff870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.812+0000 7f7b11b9c740 4 client.26132 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.812+0000 7f7b11b9c740 1 -- 172.21.15.39:0/2687392676 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56304c76c1e0 con 0x56304c416d80 2024-05-04T03:22:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.812+0000 7f7b03fff640 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56304c416d80 0x56304c7ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.812+0000 7f7ae3fff640 10 client.26132 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.813+0000 7f7b03fff640 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56304c416d80 0x56304c7ff870 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd97a0936513600 server_cookie=72f53823369e22ed in_seq=0 out_seq=0 2024-05-04T03:22:06.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:06 smithi066 ceph-mon[33264]: pgmap v725: 97 pgs: 97 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 23 MiB/s wr, 1.44k op/s 2024-05-04T03:22:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.813+0000 7f7ae3fff640 1 -- 172.21.15.39:0/2687392676 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56304c76c1e0 con 0x56304c416d80 2024-05-04T03:22:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.813+0000 7f7ae3fff640 10 client.26132 handle_command_reply: tid=0 2024-05-04T03:22:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 client.26132 shutdown 2024-05-04T03:22:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 -- 172.21.15.39:0/2687392676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c42f2f0 msgr2=0x56304c80bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c42f2f0 0x56304c80bed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7af402acb0 tx=0x7f7af402a6c0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 -- 172.21.15.39:0/2687392676 shutdown_connections 2024-05-04T03:22:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56304c416d80 0x56304c7ff870 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4581b0 0x56304c813370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c4374d0 0x56304c80c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/2687392676 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c42f2f0 0x56304c80bed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 -- 172.21.15.39:0/2687392676 >> 172.21.15.39:0/2687392676 conn(0x56304c44c300 msgr2=0x56304c7f8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7ae3fff640 0 client.26132 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 -- 172.21.15.39:0/2687392676 shutdown_connections 2024-05-04T03:22:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 1 -- 172.21.15.39:0/2687392676 wait complete. 2024-05-04T03:22:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.814+0000 7f7b11b9c740 20 client.26132 trim_cache size 0 max 16384 2024-05-04T03:22:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.841+0000 7f7b11b9c740 1 Processor -- start 2024-05-04T03:22:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b11b9c740 1 -- start start 2024-05-04T03:22:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c829e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c7f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b1970 con 0x56304c4374d0 2024-05-04T03:22:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b34b0 con 0x56304c4581b0 2024-05-04T03:22:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b2040 con 0x56304c42f2f0 2024-05-04T03:22:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b08a85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b08a85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8298f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34076/0 (socket says 172.21.15.39:34076) 2024-05-04T03:22:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b08a85640 1 -- 172.21.15.39:0/4238654149 learned_addr learned my addr 172.21.15.39:0/4238654149 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b09286640 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c7f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.842+0000 7f7b03fff640 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c829e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.843+0000 7f7b03fff640 1 -- 172.21.15.39:0/4238654149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 msgr2=0x56304c8298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.843+0000 7f7b03fff640 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.843+0000 7f7b03fff640 1 -- 172.21.15.39:0/4238654149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 msgr2=0x56304c7f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.843+0000 7f7b03fff640 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c7f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.843+0000 7f7b03fff640 1 -- 172.21.15.39:0/4238654149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304c802e30 con 0x56304c4374d0 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.843+0000 7f7b08a85640 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.843+0000 7f7b03fff640 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c829e30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005f10 tx=0x7f7afc005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b2591a3f29bbd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.844+0000 7f7aea7fc640 1 -- 172.21.15.39:0/4238654149 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc030070 con 0x56304c4374d0 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.844+0000 7f7aea7fc640 1 -- 172.21.15.39:0/4238654149 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7afc0044d0 con 0x56304c4374d0 2024-05-04T03:22:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.844+0000 7f7aea7fc640 1 -- 172.21.15.39:0/4238654149 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc02a9e0 con 0x56304c4374d0 2024-05-04T03:22:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.845+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4238654149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 msgr2=0x56304c829e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.845+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c829e30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005f10 tx=0x7f7afc005c90 comp rx=0 tx=0).stop 2024-05-04T03:22:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.845+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4238654149 shutdown_connections 2024-05-04T03:22:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.845+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c7f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.845+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c829e30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.845+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/4238654149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c8298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.845+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4238654149 >> 172.21.15.39:0/4238654149 conn(0x56304c44c300 msgr2=0x56304c806340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.846+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4238654149 shutdown_connections 2024-05-04T03:22:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.846+0000 7f7b11b9c740 1 -- 172.21.15.39:0/4238654149 wait complete. 2024-05-04T03:22:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.846+0000 7f7b11b9c740 1 -- start start 2024-05-04T03:22:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b11b9c740 1 Processor -- start 2024-05-04T03:22:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c80f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c80fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b11b9c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c888480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b2de0 con 0x56304c4374d0 2024-05-04T03:22:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b08a85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c80f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b2710 con 0x56304c4581b0 2024-05-04T03:22:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b11b9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041af950 con 0x56304c42f2f0 2024-05-04T03:22:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b08a85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c80f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34078/0 (socket says 172.21.15.39:34078) 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b08a85640 1 -- 172.21.15.39:0/3520202708 learned_addr learned my addr 172.21.15.39:0/3520202708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.848+0000 7f7b11b9c740 10 client.0 _ll_register_callbacks cb 0x7f7af8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.848+0000 7f7b09286640 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c888480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.847+0000 7f7b03fff640 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c80fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.848+0000 7f7b09286640 1 -- 172.21.15.39:0/3520202708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 msgr2=0x56304c80f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.848+0000 7f7b09286640 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c80f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.848+0000 7f7b09286640 1 -- 172.21.15.39:0/3520202708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 msgr2=0x56304c80fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.848+0000 7f7b09286640 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c80fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.848+0000 7f7b09286640 1 -- 172.21.15.39:0/3520202708 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304c812c90 con 0x56304c4581b0 2024-05-04T03:22:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.848+0000 7f7b03fff640 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c80fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.849+0000 7f7b09286640 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c888480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7af80dfba0 tx=0x7f7af80a9e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e59381fad835c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.849+0000 7f7ae3fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.849+0000 7f7ae3fff640 1 -- 172.21.15.39:0/3520202708 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af801c040 con 0x56304c4581b0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.849+0000 7f7ae3fff640 1 -- 172.21.15.39:0/3520202708 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7af80910b0 con 0x56304c4581b0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.849+0000 7f7b11b9c740 1 -- 172.21.15.39:0/3520202708 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc0044d0 con 0x56304c4581b0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.849+0000 7f7ae3fff640 1 -- 172.21.15.39:0/3520202708 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7af80912d0 con 0x56304c4581b0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.849+0000 7f7b11b9c740 10 client.26138 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.849+0000 7f7b11b9c740 1 -- 172.21.15.39:0/3520202708 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56304c889aa0 con 0x56304c4581b0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7ae3fff640 1 -- 172.21.15.39:0/3520202708 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7afc0044d0 con 0x56304c4581b0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7ae3fff640 1 -- 172.21.15.39:0/3520202708 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7af80b2190 con 0x56304c4581b0 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7b11b9c740 10 client.26138 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7b11b9c740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7b11b9c740 20 client.26138 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56304c3ec1c0 0x56304c88c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7b11b9c740 4 client.26138 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7b11b9c740 1 -- 172.21.15.39:0/3520202708 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56304c766050 con 0x56304c3ec1c0 2024-05-04T03:22:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.850+0000 7f7b08a85640 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56304c3ec1c0 0x56304c88c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.851+0000 7f7ae3fff640 10 client.26138 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.851+0000 7f7b08a85640 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56304c3ec1c0 0x56304c88c440 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1080958011ac0994 server_cookie=ed9d001e86756a2f in_seq=0 out_seq=0 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7ae3fff640 1 -- 172.21.15.39:0/3520202708 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56304c766050 con 0x56304c3ec1c0 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7ae3fff640 10 client.26138 handle_command_reply: tid=0 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 client.26138 shutdown 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 -- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 msgr2=0x56304c888480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c888480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7af80dfba0 tx=0x7f7af80a9e80 comp rx=0 tx=0).stop 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 -- 172.21.15.39:0/3520202708 shutdown_connections 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56304c3ec1c0 0x56304c88c440 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7ae3fff640 0 client.26138 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56304c4581b0 0x56304c888480 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56304c4374d0 0x56304c80fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 --2- 172.21.15.39:0/3520202708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56304c42f2f0 0x56304c80f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.860+0000 7f7b11b9c740 1 -- 172.21.15.39:0/3520202708 >> 172.21.15.39:0/3520202708 conn(0x56304c44c300 msgr2=0x56304c7f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.861+0000 7f7b11b9c740 1 -- 172.21.15.39:0/3520202708 shutdown_connections 2024-05-04T03:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.861+0000 7f7b11b9c740 1 -- 172.21.15.39:0/3520202708 wait complete. 2024-05-04T03:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.861+0000 7f7b11b9c740 20 client.26138 trim_cache size 0 max 16384 2024-05-04T03:22:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:22:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:22:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:22:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 -- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aec0b28b0 msgr2=0x7f7aec0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aec0b28b0 0x7f7aec0b4dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7af4030800 tx=0x7f7af4031000 comp rx=0 tx=0).stop 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 -- 172.21.15.39:0/2862034065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b041121f0 msgr2=0x7f7b041aa080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b041121f0 0x7f7b041aa080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009fd0 tx=0x7f7af800eba0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 -- 172.21.15.39:0/2862034065 shutdown_connections 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aec0b28b0 0x7f7aec0b4dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b041121f0 0x7f7b041aa080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0410f890 0x7f7b041a30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 --2- 172.21.15.39:0/2862034065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0410a0f0 0x7f7b041a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 -- 172.21.15.39:0/2862034065 >> 172.21.15.39:0/2862034065 conn(0x7f7b04070a90 msgr2=0x7f7b041108d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 -- 172.21.15.39:0/2862034065 shutdown_connections 2024-05-04T03:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:06.862+0000 7f7b0ace9640 1 -- 172.21.15.39:0/2862034065 wait complete. 2024-05-04T03:22:08.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:08 smithi167 ceph-mon[33614]: pgmap v726: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-05-04T03:22:08.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:08 smithi039 ceph-mon[29715]: pgmap v726: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-05-04T03:22:08.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:08 smithi066 ceph-mon[33264]: pgmap v726: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-05-04T03:22:10.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:10 smithi167 ceph-mon[33614]: pgmap v727: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 22 MiB/s wr, 1.52k op/s 2024-05-04T03:22:10.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:10 smithi039 ceph-mon[29715]: pgmap v727: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 22 MiB/s wr, 1.52k op/s 2024-05-04T03:22:10.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:10 smithi066 ceph-mon[33264]: pgmap v727: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 22 MiB/s wr, 1.52k op/s 2024-05-04T03:22:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 500.0 s, 870.6 tps, lat 104.155 ms stddev 122.074 2024-05-04T03:22:12.078 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-04T03:22:12.078 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-04T03:22:12.078 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:22:12.418 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:12 smithi039 ceph-mon[29715]: pgmap v728: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2024-05-04T03:22:12.486 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:22:12.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:12 smithi167 ceph-mon[33614]: pgmap v728: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2024-05-04T03:22:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:12 smithi066 ceph-mon[33264]: pgmap v728: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2024-05-04T03:22:14.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:14 smithi039 ceph-mon[29715]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-05-04T03:22:14.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:14 smithi167 ceph-mon[33614]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-05-04T03:22:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:14 smithi066 ceph-mon[33264]: pgmap v729: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 -- 172.21.15.39:0/416714565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf94107df0 msgr2=0x7fdf94108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 --2- 172.21.15.39:0/416714565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf94107df0 0x7fdf94108200 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdf880099c0 tx=0x7fdf88009cd0 comp rx=0 tx=0).stop 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 -- 172.21.15.39:0/416714565 shutdown_connections 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 --2- 172.21.15.39:0/416714565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf94108740 0x7fdf941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 --2- 172.21.15.39:0/416714565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf941034c0 0x7fdf941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 --2- 172.21.15.39:0/416714565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf94107df0 0x7fdf94108200 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 -- 172.21.15.39:0/416714565 >> 172.21.15.39:0/416714565 conn(0x7fdf940fedc0 msgr2=0x7fdf941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 -- 172.21.15.39:0/416714565 shutdown_connections 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.957+0000 7fdf99522700 1 -- 172.21.15.39:0/416714565 wait complete. 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf99522700 1 Processor -- start 2024-05-04T03:22:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf99522700 1 -- start start 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf99522700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf941034c0 0x7fdf941a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf99522700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf94107df0 0x7fdf941a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf99522700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf94108740 0x7fdf941a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf99522700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94113200 con 0x7fdf94107df0 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf99522700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94113080 con 0x7fdf94108740 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf99522700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf94113380 con 0x7fdf941034c0 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf937fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf94108740 0x7fdf941a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf937fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf94108740 0x7fdf941a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43180/0 (socket says 172.21.15.39:43180) 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf937fe700 1 -- 172.21.15.39:0/1227745626 learned_addr learned my addr 172.21.15.39:0/1227745626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf927fc700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf94107df0 0x7fdf941a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.958+0000 7fdf92ffd700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf941034c0 0x7fdf941a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:14.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf937fe700 1 -- 172.21.15.39:0/1227745626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf941034c0 msgr2=0x7fdf941a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf937fe700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf941034c0 0x7fdf941a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf937fe700 1 -- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf94107df0 msgr2=0x7fdf941a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf937fe700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf94107df0 0x7fdf941a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf937fe700 1 -- 172.21.15.39:0/1227745626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf941a6210 con 0x7fdf94108740 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf937fe700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf94108740 0x7fdf941a5b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00ef30 tx=0x7fdf8c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf7bfff700 1 -- 172.21.15.39:0/1227745626 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0092e0 con 0x7fdf94108740 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf941a64a0 con 0x7fdf94108740 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf7bfff700 1 -- 172.21.15.39:0/1227745626 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf8c004bb0 con 0x7fdf94108740 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf7bfff700 1 -- 172.21.15.39:0/1227745626 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c004020 con 0x7fdf94108740 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.959+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf941a6960 con 0x7fdf94108740 2024-05-04T03:22:14.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.961+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf940512c0 con 0x7fdf94108740 2024-05-04T03:22:14.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.962+0000 7fdf7bfff700 1 -- 172.21.15.39:0/1227745626 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fdf8c01b030 con 0x7fdf94108740 2024-05-04T03:22:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.963+0000 7fdf7bfff700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf7c0b2900 0x7fdf7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.963+0000 7fdf92ffd700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf7c0b2900 0x7fdf7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.963+0000 7fdf7bfff700 1 -- 172.21.15.39:0/1227745626 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fdf8c0e1c70 con 0x7fdf94108740 2024-05-04T03:22:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.963+0000 7fdf92ffd700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf7c0b2900 0x7fdf7c0b4e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009bf0 tx=0x7fdf84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:14.968+0000 7fdf7bfff700 1 -- 172.21.15.39:0/1227745626 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf8c022020 con 0x7fdf94108740 2024-05-04T03:22:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.409+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf941ade80 con 0x7fdf94108740 2024-05-04T03:22:15.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.410+0000 7fdf7bfff700 1 -- 172.21.15.39:0/1227745626 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7fdf8c010040 con 0x7fdf94108740 2024-05-04T03:22:15.413 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:22:15.413 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:22:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf7c0b2900 msgr2=0x7fdf7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf7c0b2900 0x7fdf7c0b4e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009bf0 tx=0x7fdf84009260 comp rx=0 tx=0).stop 2024-05-04T03:22:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf94108740 msgr2=0x7fdf941a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf94108740 0x7fdf941a5b70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00ef30 tx=0x7fdf8c00c3a0 comp rx=0 tx=0).stop 2024-05-04T03:22:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 shutdown_connections 2024-05-04T03:22:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fdf7c0b2900 0x7fdf7c0b4e10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf94108740 0x7fdf941a5b70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf941034c0 0x7fdf941a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 --2- 172.21.15.39:0/1227745626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf94107df0 0x7fdf941a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.414+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 >> 172.21.15.39:0/1227745626 conn(0x7fdf940fedc0 msgr2=0x7fdf9406b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.415+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 shutdown_connections 2024-05-04T03:22:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:15.416+0000 7fdf99522700 1 -- 172.21.15.39:0/1227745626 wait complete. 2024-05-04T03:22:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:22:15.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:15 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1227745626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:15.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:15 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1227745626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:15.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:15 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1227745626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:16.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:16 smithi167 ceph-mon[33614]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 24 MiB/s wr, 1.43k op/s 2024-05-04T03:22:16.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:16 smithi039 ceph-mon[29715]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 24 MiB/s wr, 1.43k op/s 2024-05-04T03:22:16.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 505.0 s, 889.6 tps, lat 122.957 ms stddev 316.287 2024-05-04T03:22:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:16 smithi066 ceph-mon[33264]: pgmap v730: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 24 MiB/s wr, 1.43k op/s 2024-05-04T03:22:16.895 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-05-04T03:22:16.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-05-04T03:22:17.326 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:22:18.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:18 smithi039 ceph-mon[29715]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 25 MiB/s wr, 1.45k op/s 2024-05-04T03:22:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:18 smithi066 ceph-mon[33264]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 25 MiB/s wr, 1.45k op/s 2024-05-04T03:22:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:18 smithi167 ceph-mon[33614]: pgmap v731: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 25 MiB/s wr, 1.45k op/s 2024-05-04T03:22:19.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.481+0000 7f9d8b199700 1 -- 172.21.15.39:0/1799001829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d84107df0 msgr2=0x7f9d84108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.481+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1799001829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d84107df0 0x7f9d84108200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009920 tx=0x7f9d70009c30 comp rx=0 tx=0).stop 2024-05-04T03:22:19.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.481+0000 7f9d8b199700 1 -- 172.21.15.39:0/1799001829 shutdown_connections 2024-05-04T03:22:19.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.481+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1799001829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d841034c0 0x7f9d841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.481+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1799001829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d84108740 0x7f9d841100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.481+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1799001829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d84107df0 0x7f9d84108200 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.481+0000 7f9d8b199700 1 -- 172.21.15.39:0/1799001829 >> 172.21.15.39:0/1799001829 conn(0x7f9d840fedc0 msgr2=0x7f9d841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.482+0000 7f9d8b199700 1 -- 172.21.15.39:0/1799001829 shutdown_connections 2024-05-04T03:22:19.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.482+0000 7f9d8b199700 1 -- 172.21.15.39:0/1799001829 wait complete. 2024-05-04T03:22:19.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.483+0000 7f9d8b199700 1 Processor -- start 2024-05-04T03:22:19.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.483+0000 7f9d8b199700 1 -- start start 2024-05-04T03:22:19.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d8b199700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d841034c0 0x7f9d84199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d8b199700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84107df0 0x7f9d8419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d88f35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d841034c0 0x7f9d84199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84107df0 0x7f9d8419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84107df0 0x7f9d8419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52162/0 (socket says 172.21.15.39:52162) 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d88f35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d841034c0 0x7f9d84199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43182/0 (socket says 172.21.15.39:43182) 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d8b199700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d84108740 0x7f9d8419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d83fff700 1 -- 172.21.15.39:0/1831744419 learned_addr learned my addr 172.21.15.39:0/1831744419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.484+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84113250 con 0x7f9d84107df0 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841130d0 con 0x7f9d841034c0 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841133d0 con 0x7f9d84108740 2024-05-04T03:22:19.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d89736700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d84108740 0x7f9d8419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d83fff700 1 -- 172.21.15.39:0/1831744419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d84108740 msgr2=0x7f9d8419e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d83fff700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d84108740 0x7f9d8419e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d83fff700 1 -- 172.21.15.39:0/1831744419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d841034c0 msgr2=0x7f9d84199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d83fff700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d841034c0 0x7f9d84199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d83fff700 1 -- 172.21.15.39:0/1831744419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8419ec10 con 0x7f9d84107df0 2024-05-04T03:22:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.485+0000 7f9d83fff700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84107df0 0x7f9d8419a2e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9d70000c00 tx=0x7f9d70009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.486+0000 7f9d81ffb700 1 -- 172.21.15.39:0/1831744419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7000b970 con 0x7f9d84107df0 2024-05-04T03:22:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.486+0000 7f9d81ffb700 1 -- 172.21.15.39:0/1831744419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d70022070 con 0x7f9d84107df0 2024-05-04T03:22:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.486+0000 7f9d81ffb700 1 -- 172.21.15.39:0/1831744419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70003b20 con 0x7f9d84107df0 2024-05-04T03:22:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.486+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8419eea0 con 0x7f9d84107df0 2024-05-04T03:22:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.487+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8419f330 con 0x7f9d84107df0 2024-05-04T03:22:19.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.491+0000 7f9d81ffb700 1 -- 172.21.15.39:0/1831744419 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9d70003cc0 con 0x7f9d84107df0 2024-05-04T03:22:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.493+0000 7f9d99badb80 1 Processor -- start 2024-05-04T03:22:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.493+0000 7f9d81ffb700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d740b29d0 0x7f9d740b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.493+0000 7f9d81ffb700 1 -- 172.21.15.39:0/1831744419 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9d700e6e70 con 0x7f9d84107df0 2024-05-04T03:22:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.493+0000 7f9d89736700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d740b29d0 0x7f9d740b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.493+0000 7f9d99badb80 1 -- start start 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011beb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d99badb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5651010e19f0 con 0x5651011d2180 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d99badb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x565101080950 con 0x5651011be760 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d99badb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5651010dd9b0 con 0x5651011bf0b0 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d88f35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d88f35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011beb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43184/0 (socket says 172.21.15.39:43184) 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d88f35700 1 -- 172.21.15.39:0/606456529 learned_addr learned my addr 172.21.15.39:0/606456529 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d89736700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d740b29d0 0x7f9d740b4ee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c010740 tx=0x7f9d7c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.494+0000 7f9d83fff700 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.495+0000 7f9d89736700 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.495+0000 7f9d88f35700 1 -- 172.21.15.39:0/606456529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 msgr2=0x5651011d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.495+0000 7f9d88f35700 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.495+0000 7f9d88f35700 1 -- 172.21.15.39:0/606456529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 msgr2=0x5651011d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.495+0000 7f9d88f35700 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.495+0000 7f9d88f35700 1 -- 172.21.15.39:0/606456529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56510102ea60 con 0x5651011be760 2024-05-04T03:22:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.495+0000 7f9d89736700 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.495+0000 7f9d88f35700 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011beb70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009e70 tx=0x7f9d78011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3598734a813b7d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.496+0000 7f9d6effd700 1 -- 172.21.15.39:0/606456529 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7801e070 con 0x5651011be760 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.496+0000 7f9d6effd700 1 -- 172.21.15.39:0/606456529 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7800eb00 con 0x5651011be760 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.496+0000 7f9d6effd700 1 -- 172.21.15.39:0/606456529 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78019b10 con 0x5651011be760 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 -- 172.21.15.39:0/606456529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 msgr2=0x5651011beb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011beb70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009e70 tx=0x7f9d78011720 comp rx=0 tx=0).stop 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 -- 172.21.15.39:0/606456529 shutdown_connections 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011beb70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 --2- 172.21.15.39:0/606456529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 -- 172.21.15.39:0/606456529 >> 172.21.15.39:0/606456529 conn(0x565101041fd0 msgr2=0x5651011ba690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 -- 172.21.15.39:0/606456529 shutdown_connections 2024-05-04T03:22:19.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.497+0000 7f9d99badb80 1 -- 172.21.15.39:0/606456529 wait complete. 2024-05-04T03:22:19.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.498+0000 7f9d99badb80 1 -- start start 2024-05-04T03:22:19.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.498+0000 7f9d99badb80 1 Processor -- start 2024-05-04T03:22:19.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d99badb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5651011d4f60 con 0x5651011d2180 2024-05-04T03:22:19.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d99badb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x565101047dc0 con 0x5651011be760 2024-05-04T03:22:19.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d99badb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5651011d50e0 con 0x5651011bf0b0 2024-05-04T03:22:19.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d99badb80 10 client.0 _ll_register_callbacks cb 0x565101079ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d89736700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:37614/0 (socket says 172.21.15.39:37614) 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d83fff700 1 -- 172.21.15.39:0/3222017069 learned_addr learned my addr 172.21.15.39:0/3222017069 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.499+0000 7f9d88f35700 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.500+0000 7f9d83fff700 1 -- 172.21.15.39:0/3222017069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 msgr2=0x5651011cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.500+0000 7f9d83fff700 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.500+0000 7f9d83fff700 1 -- 172.21.15.39:0/3222017069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 msgr2=0x5651011cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.500+0000 7f9d83fff700 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.500+0000 7f9d83fff700 1 -- 172.21.15.39:0/3222017069 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56510107d820 con 0x5651011bf0b0 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.500+0000 7f9d83fff700 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011cf740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9d700aa5b0 tx=0x7f9d70009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c5cc6124c61fba server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.500+0000 7f9d63fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.501+0000 7f9d63fff700 1 -- 172.21.15.39:0/3222017069 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7002e040 con 0x5651011bf0b0 2024-05-04T03:22:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.501+0000 7f9d63fff700 1 -- 172.21.15.39:0/3222017069 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9d700ae9a0 con 0x5651011bf0b0 2024-05-04T03:22:19.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.501+0000 7f9d99badb80 1 -- 172.21.15.39:0/3222017069 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7800eb00 con 0x5651011bf0b0 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.501+0000 7f9d63fff700 1 -- 172.21.15.39:0/3222017069 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d700aec30 con 0x5651011bf0b0 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.501+0000 7f9d99badb80 10 client.26194 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.501+0000 7f9d99badb80 1 -- 172.21.15.39:0/3222017069 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5651011d4fc0 con 0x5651011bf0b0 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.502+0000 7f9d63fff700 1 -- 172.21.15.39:0/3222017069 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7800eb00 con 0x5651011bf0b0 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.502+0000 7f9d63fff700 1 -- 172.21.15.39:0/3222017069 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9d700b6e70 con 0x5651011bf0b0 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.502+0000 7f9d99badb80 10 client.26194 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.502+0000 7f9d99badb80 10 client.26194 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.502+0000 7f9d99badb80 20 client.26194 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.502+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5651011e20d0 0x5651011e4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.502+0000 7f9d99badb80 4 client.26194 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.502+0000 7f9d99badb80 1 -- 172.21.15.39:0/3222017069 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651010ef3d0 con 0x5651011e20d0 2024-05-04T03:22:19.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.503+0000 7f9d88f35700 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5651011e20d0 0x5651011e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.503+0000 7f9d88f35700 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5651011e20d0 0x5651011e4510 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5baff175096c0f77 server_cookie=337405f2be95c952 in_seq=0 out_seq=0 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.503+0000 7f9d63fff700 10 client.26194 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d63fff700 1 -- 172.21.15.39:0/3222017069 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651010ef3d0 con 0x5651011e20d0 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d63fff700 10 client.26194 handle_command_reply: tid=0 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d99badb80 1 client.26194 shutdown 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d99badb80 1 -- 172.21.15.39:0/3222017069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 msgr2=0x5651011cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011cf740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9d700aa5b0 tx=0x7f9d70009c30 comp rx=0 tx=0).stop 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d99badb80 1 -- 172.21.15.39:0/3222017069 shutdown_connections 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5651011e20d0 0x5651011e4510 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d63fff700 0 client.26194 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:19.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011be760 0x5651011cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.504+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3222017069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011cf740 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.505+0000 7f9d99badb80 1 -- 172.21.15.39:0/3222017069 >> 172.21.15.39:0/3222017069 conn(0x565101041fd0 msgr2=0x5651011c01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.505+0000 7f9d99badb80 1 -- 172.21.15.39:0/3222017069 shutdown_connections 2024-05-04T03:22:19.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.505+0000 7f9d99badb80 1 -- 172.21.15.39:0/3222017069 wait complete. 2024-05-04T03:22:19.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.505+0000 7f9d99badb80 20 client.26194 trim_cache size 0 max 16384 2024-05-04T03:22:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d99badb80 1 Processor -- start 2024-05-04T03:22:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d99badb80 1 -- start start 2024-05-04T03:22:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011be760 0x5651011cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011d2180 0x5651011ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d99badb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841ac8b0 con 0x5651011be760 2024-05-04T03:22:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d99badb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841ae3f0 con 0x5651011d2180 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d99badb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841acf80 con 0x5651011bf0b0 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.540+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011e9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:37626/0 (socket says 172.21.15.39:37626) 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d83fff700 1 -- 172.21.15.39:0/3210056481 learned_addr learned my addr 172.21.15.39:0/3210056481 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d89736700 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011d2180 0x5651011ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d88f35700 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011be760 0x5651011cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d88f35700 1 -- 172.21.15.39:0/3210056481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 msgr2=0x5651011e9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d88f35700 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011e9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d88f35700 1 -- 172.21.15.39:0/3210056481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011d2180 msgr2=0x5651011ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d88f35700 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011d2180 0x5651011ea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.541+0000 7f9d88f35700 1 -- 172.21.15.39:0/3210056481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651011d4df0 con 0x5651011be760 2024-05-04T03:22:19.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.542+0000 7f9d88f35700 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011be760 0x5651011cfac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9d7801c040 tx=0x7f9d78005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f160bd1c486edb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.542+0000 7f9d6e7fc700 1 -- 172.21.15.39:0/3210056481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7802f020 con 0x5651011be760 2024-05-04T03:22:19.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.542+0000 7f9d6e7fc700 1 -- 172.21.15.39:0/3210056481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7801f040 con 0x5651011be760 2024-05-04T03:22:19.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.542+0000 7f9d6e7fc700 1 -- 172.21.15.39:0/3210056481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7801e040 con 0x5651011be760 2024-05-04T03:22:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.543+0000 7f9d99badb80 1 -- 172.21.15.39:0/3210056481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011be760 msgr2=0x5651011cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.543+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011be760 0x5651011cfac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9d7801c040 tx=0x7f9d78005210 comp rx=0 tx=0).stop 2024-05-04T03:22:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.544+0000 7f9d99badb80 1 -- 172.21.15.39:0/3210056481 shutdown_connections 2024-05-04T03:22:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.544+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011be760 0x5651011cfac0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.544+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011d2180 0x5651011ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.544+0000 7f9d99badb80 1 --2- 172.21.15.39:0/3210056481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011bf0b0 0x5651011e9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.544+0000 7f9d99badb80 1 -- 172.21.15.39:0/3210056481 >> 172.21.15.39:0/3210056481 conn(0x565101041fd0 msgr2=0x5651011b99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.545+0000 7f9d99badb80 1 -- 172.21.15.39:0/3210056481 shutdown_connections 2024-05-04T03:22:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.545+0000 7f9d99badb80 1 -- 172.21.15.39:0/3210056481 wait complete. 2024-05-04T03:22:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.545+0000 7f9d99badb80 1 -- start start 2024-05-04T03:22:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.546+0000 7f9d99badb80 1 Processor -- start 2024-05-04T03:22:19.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011be760 0x5651011da770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011bf0b0 0x5651011d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d99badb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d99badb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841add20 con 0x5651011d2180 2024-05-04T03:22:19.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d99badb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841ad650 con 0x5651011bf0b0 2024-05-04T03:22:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d99badb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841aa850 con 0x5651011be760 2024-05-04T03:22:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d99badb80 10 client.0 _ll_register_callbacks cb 0x565101079ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d88f35700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011be760 0x5651011da770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d88f35700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011be760 0x5651011da770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:37632/0 (socket says 172.21.15.39:37632) 2024-05-04T03:22:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d88f35700 1 -- 172.21.15.39:0/2172791856 learned_addr learned my addr 172.21.15.39:0/2172791856 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d89736700 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.547+0000 7f9d83fff700 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011bf0b0 0x5651011d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.548+0000 7f9d89736700 1 -- 172.21.15.39:0/2172791856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011be760 msgr2=0x5651011da770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.548+0000 7f9d89736700 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011be760 0x5651011da770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.548+0000 7f9d89736700 1 -- 172.21.15.39:0/2172791856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011bf0b0 msgr2=0x5651011d7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.548+0000 7f9d89736700 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011bf0b0 0x5651011d7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.548+0000 7f9d89736700 1 -- 172.21.15.39:0/2172791856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651011d7e40 con 0x5651011d2180 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.549+0000 7f9d89736700 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d7880 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009fd0 tx=0x7f9d7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3bda4b0003e574 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.549+0000 7f9d63fff700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.549+0000 7f9d63fff700 1 -- 172.21.15.39:0/2172791856 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c018070 con 0x5651011d2180 2024-05-04T03:22:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.549+0000 7f9d63fff700 1 -- 172.21.15.39:0/2172791856 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9d7c004bb0 con 0x5651011d2180 2024-05-04T03:22:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.549+0000 7f9d99badb80 1 -- 172.21.15.39:0/2172791856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7801f040 con 0x5651011d2180 2024-05-04T03:22:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.549+0000 7f9d63fff700 1 -- 172.21.15.39:0/2172791856 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c004db0 con 0x5651011d2180 2024-05-04T03:22:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.549+0000 7f9d99badb80 10 client.16581 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.549+0000 7f9d99badb80 1 -- 172.21.15.39:0/2172791856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56510124d5e0 con 0x5651011d2180 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.550+0000 7f9d63fff700 1 -- 172.21.15.39:0/2172791856 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d7801f040 con 0x5651011d2180 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.550+0000 7f9d63fff700 1 -- 172.21.15.39:0/2172791856 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9d7c035ad0 con 0x5651011d2180 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.550+0000 7f9d99badb80 10 client.16581 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.550+0000 7f9d99badb80 10 client.16581 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.550+0000 7f9d99badb80 20 client.16581 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.550+0000 7f9d99badb80 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5651011e20d0 0x56510124fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.550+0000 7f9d99badb80 4 client.16581 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.550+0000 7f9d99badb80 1 -- 172.21.15.39:0/2172791856 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x5651010e9470 con 0x5651011e20d0 2024-05-04T03:22:19.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.551+0000 7f9d88f35700 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5651011e20d0 0x56510124fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.551+0000 7f9d63fff700 10 client.16581 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.551+0000 7f9d88f35700 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5651011e20d0 0x56510124fe00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4146c6ab962079a1 server_cookie=9d620a162982dafc in_seq=0 out_seq=0 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.952+0000 7f9d63fff700 1 -- 172.21.15.39:0/2172791856 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5651010e9470 con 0x5651011e20d0 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.952+0000 7f9d63fff700 10 client.16581 handle_command_reply: tid=0 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.952+0000 7f9d99badb80 1 client.16581 shutdown 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.952+0000 7f9d99badb80 1 -- 172.21.15.39:0/2172791856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 msgr2=0x5651011d7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.952+0000 7f9d99badb80 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d7880 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009fd0 tx=0x7f9d7c00b040 comp rx=0 tx=0).stop 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 1 -- 172.21.15.39:0/2172791856 shutdown_connections 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5651011e20d0 0x56510124fe00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5651011d2180 0x5651011d7880 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651011be760 0x5651011da770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d63fff700 0 client.16581 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 1 --2- 172.21.15.39:0/2172791856 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5651011bf0b0 0x5651011d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 1 -- 172.21.15.39:0/2172791856 >> 172.21.15.39:0/2172791856 conn(0x565101041fd0 msgr2=0x5651011ba3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 1 -- 172.21.15.39:0/2172791856 shutdown_connections 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 1 -- 172.21.15.39:0/2172791856 wait complete. 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.953+0000 7f9d99badb80 20 client.16581 trim_cache size 0 max 16384 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 30, 2024-05-04T03:22:19.956 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "c858a07d": { 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 30, 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 90.0, 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:22:19.957 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:22:19.958 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:19.958 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:19.958 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:19.958 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:19.958 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:19.958 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d740b29d0 msgr2=0x7f9d740b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d740b29d0 0x7f9d740b4ee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c010740 tx=0x7f9d7c00d040 comp rx=0 tx=0).stop 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84107df0 msgr2=0x7f9d8419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84107df0 0x7f9d8419a2e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9d70000c00 tx=0x7f9d70009400 comp rx=0 tx=0).stop 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 shutdown_connections 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d740b29d0 0x7f9d740b4ee0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d84108740 0x7f9d8419e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d841034c0 0x7f9d84199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 --2- 172.21.15.39:0/1831744419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84107df0 0x7f9d8419a2e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 >> 172.21.15.39:0/1831744419 conn(0x7f9d840fedc0 msgr2=0x7f9d841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 shutdown_connections 2024-05-04T03:22:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:19.954+0000 7f9d8b199700 1 -- 172.21.15.39:0/1831744419 wait complete. 2024-05-04T03:22:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 -- 172.21.15.39:0/991703587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac108a90 msgr2=0x7f86ac10aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 --2- 172.21.15.39:0/991703587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac108a90 0x7f86ac10aeb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009ca0 tx=0x7f86a402a8c0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 -- 172.21.15.39:0/991703587 shutdown_connections 2024-05-04T03:22:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 --2- 172.21.15.39:0/991703587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac108a90 0x7f86ac10aeb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 --2- 172.21.15.39:0/991703587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86ac103d90 0x7f86ac1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 --2- 172.21.15.39:0/991703587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86ac102b50 0x7f86ac103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 -- 172.21.15.39:0/991703587 >> 172.21.15.39:0/991703587 conn(0x7f86ac0fdd40 msgr2=0x7f86ac1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 -- 172.21.15.39:0/991703587 shutdown_connections 2024-05-04T03:22:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.972+0000 7f86b18f9640 1 -- 172.21.15.39:0/991703587 wait complete. 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86b18f9640 1 Processor -- start 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86b18f9640 1 -- start start 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86b18f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac102b50 0x7f86ac06af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86b18f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86ac103d90 0x7f86ac19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86b18f9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86ac108a90 0x7f86ac1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86b18f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac10e560 con 0x7f86ac102b50 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86b18f9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac10e3e0 con 0x7f86ac108a90 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86b18f9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac10e6e0 con 0x7f86ac103d90 2024-05-04T03:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac102b50 0x7f86ac06af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac102b50 0x7f86ac06af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52214/0 (socket says 172.21.15.39:52214) 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86aaffd640 1 -- 172.21.15.39:0/1392338534 learned_addr learned my addr 172.21.15.39:0/1392338534 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.973+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86ac103d90 0x7f86ac19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86ab7fe640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86ac108a90 0x7f86ac1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86aaffd640 1 -- 172.21.15.39:0/1392338534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86ac103d90 msgr2=0x7f86ac19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86aaffd640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86ac103d90 0x7f86ac19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86aaffd640 1 -- 172.21.15.39:0/1392338534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86ac108a90 msgr2=0x7f86ac1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86aaffd640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86ac108a90 0x7f86ac1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86aaffd640 1 -- 172.21.15.39:0/1392338534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ac1a1c90 con 0x7f86ac102b50 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86aaffd640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac102b50 0x7f86ac06af60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f869c00e9f0 tx=0x7f869c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86b08f7640 1 -- 172.21.15.39:0/1392338534 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c00ccc0 con 0x7f86ac102b50 2024-05-04T03:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86b08f7640 1 -- 172.21.15.39:0/1392338534 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f869c0044b0 con 0x7f86ac102b50 2024-05-04T03:22:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86b18f9640 1 -- 172.21.15.39:0/1392338534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ac1a1f80 con 0x7f86ac102b50 2024-05-04T03:22:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86b08f7640 1 -- 172.21.15.39:0/1392338534 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c010430 con 0x7f86ac102b50 2024-05-04T03:22:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.974+0000 7f86b18f9640 1 -- 172.21.15.39:0/1392338534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ac1a23e0 con 0x7f86ac102b50 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.977+0000 7f86b87ab740 1 Processor -- start 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.977+0000 7f86b87ab740 1 -- start start 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.977+0000 7f86b08f7640 1 -- 172.21.15.39:0/1392338534 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f869c0105d0 con 0x7f86ac102b50 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.977+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 0x55b11d938e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.977+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d56a2f0 0x55b11d9393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.977+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d9402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86b87ab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b11d8995d0 con 0x55b11d5724d0 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86b87ab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b11d852890 con 0x55b11d5931b0 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86b87ab740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b11d895590 con 0x55b11d56a2f0 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d9402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d56a2f0 0x55b11d9393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d9402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52230/0 (socket says 172.21.15.39:52230) 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 -- 172.21.15.39:0/3350512434 learned_addr learned my addr 172.21.15.39:0/3350512434 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86ab7fe640 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 0x55b11d938e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 -- 172.21.15.39:0/3350512434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d56a2f0 msgr2=0x55b11d9393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d56a2f0 0x55b11d9393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 -- 172.21.15.39:0/3350512434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 msgr2=0x55b11d938e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 0x55b11d938e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 -- 172.21.15.39:0/3350512434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b11d5ab210 con 0x55b11d5724d0 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aaffd640 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d56a2f0 0x55b11d9393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86b08f7640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f86940b27e0 0x7f86940b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d9402f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f86a000ac80 tx=0x7f86a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17aaf9fbd55890c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.978+0000 7f86ab7fe640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f86940b27e0 0x7f86940b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.979+0000 7f8691ffb640 1 -- 172.21.15.39:0/3350512434 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a001d070 con 0x55b11d5724d0 2024-05-04T03:22:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.979+0000 7f86b08f7640 1 -- 172.21.15.39:0/1392338534 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f869c0e25f0 con 0x7f86ac102b50 2024-05-04T03:22:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.979+0000 7f8691ffb640 1 -- 172.21.15.39:0/3350512434 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86a0018ad0 con 0x55b11d5724d0 2024-05-04T03:22:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.979+0000 7f8691ffb640 1 -- 172.21.15.39:0/3350512434 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0010a00 con 0x55b11d5724d0 2024-05-04T03:22:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 -- 172.21.15.39:0/3350512434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 msgr2=0x55b11d9402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86ab7fe640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f86940b27e0 0x7f86940b4cf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55b11d93a040 tx=0x7f86a4031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d9402f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f86a000ac80 tx=0x7f86a000f790 comp rx=0 tx=0).stop 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 -- 172.21.15.39:0/3350512434 shutdown_connections 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d9402f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d56a2f0 0x55b11d9393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3350512434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 0x55b11d938e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 -- 172.21.15.39:0/3350512434 >> 172.21.15.39:0/3350512434 conn(0x55b11d587300 msgr2=0x55b11d9303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 -- 172.21.15.39:0/3350512434 shutdown_connections 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 -- 172.21.15.39:0/3350512434 wait complete. 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.980+0000 7f86b87ab740 1 -- start start 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86b87ab740 1 Processor -- start 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d942ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d942ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 0x55b11d949f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86b87ab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b11d940d40 con 0x55b11d5724d0 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86b87ab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b11d856370 con 0x55b11d56a2f0 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86b87ab740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b11d940ec0 con 0x55b11d5931b0 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86b87ab740 10 client.0 _ll_register_callbacks cb 0x55b11d5aaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d942ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d942ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d942ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43218/0 (socket says 172.21.15.39:43218) 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d942ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52238/0 (socket says 172.21.15.39:52238) 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aaffd640 1 -- 172.21.15.39:0/3820376084 learned_addr learned my addr 172.21.15.39:0/3820376084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86ab7fe640 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 0x55b11d949f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aaffd640 1 -- 172.21.15.39:0/3820376084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 msgr2=0x55b11d949f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aaffd640 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 0x55b11d949f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aaffd640 1 -- 172.21.15.39:0/3820376084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 msgr2=0x55b11d942ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aaffd640 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d942ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.981+0000 7f86aaffd640 1 -- 172.21.15.39:0/3820376084 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b11d677620 con 0x55b11d56a2f0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f86aaffd640 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d942ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f869c0a9bb0 tx=0x7f869c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680195b4e0129840 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f868affd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f868affd640 1 -- 172.21.15.39:0/3820376084 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c0a9570 con 0x55b11d56a2f0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f868affd640 1 -- 172.21.15.39:0/3820376084 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f869c0a9090 con 0x55b11d56a2f0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f86b87ab740 1 -- 172.21.15.39:0/3820376084 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a0018ad0 con 0x55b11d56a2f0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f868affd640 1 -- 172.21.15.39:0/3820376084 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f869c0a9390 con 0x55b11d56a2f0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f86b87ab740 10 client.26174 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f86b87ab740 1 -- 172.21.15.39:0/3820376084 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b11d94cf50 con 0x55b11d56a2f0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f868affd640 1 -- 172.21.15.39:0/3820376084 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86a0018ad0 con 0x55b11d56a2f0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f868affd640 1 -- 172.21.15.39:0/3820376084 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f869c0e2100 con 0x55b11d56a2f0 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f86b87ab740 10 client.26174 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f86b87ab740 10 client.26174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f86b87ab740 20 client.26174 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.982+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b11d551d80 0x55b11d9506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.983+0000 7f86b87ab740 4 client.26174 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.983+0000 7f86b87ab740 1 -- 172.21.15.39:0/3820376084 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b11d8a71e0 con 0x55b11d551d80 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.983+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b11d551d80 0x55b11d9506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.983+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b11d551d80 0x55b11d9506c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805da2b48555b4e4 server_cookie=3665664e1caa37a4 in_seq=0 out_seq=0 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.983+0000 7f868affd640 10 client.26174 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f868affd640 1 -- 172.21.15.39:0/3820376084 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b11d8a71e0 con 0x55b11d551d80 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f868affd640 10 client.26174 handle_command_reply: tid=0 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 client.26174 shutdown 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 -- 172.21.15.39:0/3820376084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 msgr2=0x55b11d942ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d942ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f869c0a9bb0 tx=0x7f869c009310 comp rx=0 tx=0).stop 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 -- 172.21.15.39:0/3820376084 shutdown_connections 2024-05-04T03:22:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b11d551d80 0x55b11d9506c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 0x55b11d949f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f868affd640 0 client.26174 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d942ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 --2- 172.21.15.39:0/3820376084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d942ab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 -- 172.21.15.39:0/3820376084 >> 172.21.15.39:0/3820376084 conn(0x55b11d587300 msgr2=0x55b11d92f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 -- 172.21.15.39:0/3820376084 shutdown_connections 2024-05-04T03:22:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 1 -- 172.21.15.39:0/3820376084 wait complete. 2024-05-04T03:22:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:19.984+0000 7f86b87ab740 20 client.26174 trim_cache size 0 max 16384 2024-05-04T03:22:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.000+0000 7f86b87ab740 1 Processor -- start 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.000+0000 7f86b87ab740 1 -- start start 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.000+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d56a2f0 0x55b11d942d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5724d0 0x55b11d949ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 0x55b11d92e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86b87ab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1a9050 con 0x55b11d56a2f0 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86b87ab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1aab90 con 0x55b11d5931b0 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86b87ab740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1a9720 con 0x55b11d5724d0 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d56a2f0 0x55b11d942d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d56a2f0 0x55b11d942d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52248/0 (socket says 172.21.15.39:52248) 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 -- 172.21.15.39:0/1792285644 learned_addr learned my addr 172.21.15.39:0/1792285644 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86ab7fe640 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 0x55b11d92e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5724d0 0x55b11d949ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 -- 172.21.15.39:0/1792285644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5724d0 msgr2=0x55b11d949ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5724d0 0x55b11d949ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 -- 172.21.15.39:0/1792285644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 msgr2=0x55b11d92e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 0x55b11d92e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 -- 172.21.15.39:0/1792285644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b11d930d00 con 0x55b11d56a2f0 2024-05-04T03:22:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86aaffd640 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d56a2f0 0x55b11d942d90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f869c0a9920 tx=0x7f869c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ce24f22a7fd227 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86917fa640 1 -- 172.21.15.39:0/1792285644 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c0a5ae0 con 0x55b11d56a2f0 2024-05-04T03:22:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86917fa640 1 -- 172.21.15.39:0/1792285644 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f869c0b23c0 con 0x55b11d56a2f0 2024-05-04T03:22:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.001+0000 7f86917fa640 1 -- 172.21.15.39:0/1792285644 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c0adfd0 con 0x55b11d56a2f0 2024-05-04T03:22:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 -- 172.21.15.39:0/1792285644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d56a2f0 msgr2=0x55b11d942d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d56a2f0 0x55b11d942d90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f869c0a9920 tx=0x7f869c0046c0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 -- 172.21.15.39:0/1792285644 shutdown_connections 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d5931b0 0x55b11d92e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5724d0 0x55b11d949ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 --2- 172.21.15.39:0/1792285644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d56a2f0 0x55b11d942d90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 -- 172.21.15.39:0/1792285644 >> 172.21.15.39:0/1792285644 conn(0x55b11d587300 msgr2=0x55b11d942680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 -- 172.21.15.39:0/1792285644 shutdown_connections 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.002+0000 7f86b87ab740 1 -- 172.21.15.39:0/1792285644 wait complete. 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 1 -- start start 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 1 Processor -- start 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d947430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d947970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 0x55b11d948850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1aa4c0 con 0x55b11d5724d0 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1a9df0 con 0x55b11d56a2f0 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1a7030 con 0x55b11d5931b0 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86b87ab740 10 client.0 _ll_register_callbacks cb 0x7f869c0e5dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d947970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d947970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52250/0 (socket says 172.21.15.39:52250) 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86aa7fc640 1 -- 172.21.15.39:0/2027438036 learned_addr learned my addr 172.21.15.39:0/2027438036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86aaffd640 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d947430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.003+0000 7f86ab7fe640 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 0x55b11d948850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86aa7fc640 1 -- 172.21.15.39:0/2027438036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 msgr2=0x55b11d948850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 0x55b11d948850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86aa7fc640 1 -- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 msgr2=0x55b11d947430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d947430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86aa7fc640 1 -- 172.21.15.39:0/2027438036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b11d948e10 con 0x55b11d5724d0 2024-05-04T03:22:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86aaffd640 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d947430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86aa7fc640 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d947970 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x55b11d931180 tx=0x7f86a000fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2232b1d3d3df711 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f868affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:22:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f868affd640 1 -- 172.21.15.39:0/2027438036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a001d040 con 0x55b11d5724d0 2024-05-04T03:22:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f868affd640 1 -- 172.21.15.39:0/2027438036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f86a00050c0 con 0x55b11d5724d0 2024-05-04T03:22:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86b87ab740 1 -- 172.21.15.39:0/2027438036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b11d8a1050 con 0x55b11d5724d0 2024-05-04T03:22:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f868affd640 1 -- 172.21.15.39:0/2027438036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f86a00052e0 con 0x55b11d5724d0 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86b87ab740 10 client.16599 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f86b87ab740 1 -- 172.21.15.39:0/2027438036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b11d94a0d0 con 0x55b11d5724d0 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f868affd640 1 -- 172.21.15.39:0/2027438036 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b11d8a1050 con 0x55b11d5724d0 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.004+0000 7f868affd640 1 -- 172.21.15.39:0/2027438036 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f86a002bce0 con 0x55b11d5724d0 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f86b87ab740 10 client.16599 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f86b87ab740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f86b87ab740 20 client.16599 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f86b87ab740 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b11d5271c0 0x55b11d9bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f86b87ab740 4 client.16599 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f86b87ab740 1 -- 172.21.15.39:0/2027438036 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b11d81c5f0 con 0x55b11d5271c0 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f86aaffd640 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b11d5271c0 0x55b11d9bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f868affd640 10 client.16599 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.005+0000 7f86aaffd640 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b11d5271c0 0x55b11d9bb010 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71857bac1ee08068 server_cookie=e843e817e3691f52 in_seq=0 out_seq=0 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f868affd640 1 -- 172.21.15.39:0/2027438036 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b11d81c5f0 con 0x55b11d5271c0 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f868affd640 10 client.16599 handle_command_reply: tid=0 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 client.16599 shutdown 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 -- 172.21.15.39:0/2027438036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 msgr2=0x55b11d947970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d947970 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x55b11d931180 tx=0x7f86a000fcf0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 -- 172.21.15.39:0/2027438036 shutdown_connections 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b11d5271c0 0x55b11d9bb010 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b11d5931b0 0x55b11d948850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b11d5724d0 0x55b11d947970 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f868affd640 0 client.16599 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 --2- 172.21.15.39:0/2027438036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b11d56a2f0 0x55b11d947430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 -- 172.21.15.39:0/2027438036 >> 172.21.15.39:0/2027438036 conn(0x55b11d587300 msgr2=0x55b11d933270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 -- 172.21.15.39:0/2027438036 shutdown_connections 2024-05-04T03:22:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 1 -- 172.21.15.39:0/2027438036 wait complete. 2024-05-04T03:22:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.012+0000 7f86b87ab740 20 client.16599 trim_cache size 0 max 16384 2024-05-04T03:22:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:22:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:22:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:22:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 -- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f86940b27e0 msgr2=0x7f86940b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f86940b27e0 0x7f86940b4cf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55b11d93a040 tx=0x7f86a4031000 comp rx=0 tx=0).stop 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 -- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac102b50 msgr2=0x7f86ac06af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac102b50 0x7f86ac06af60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f869c00e9f0 tx=0x7f869c00edf0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 -- 172.21.15.39:0/1392338534 shutdown_connections 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f86940b27e0 0x7f86940b4cf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86ac108a90 0x7f86ac1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86ac103d90 0x7f86ac19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 --2- 172.21.15.39:0/1392338534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac102b50 0x7f86ac06af60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 -- 172.21.15.39:0/1392338534 >> 172.21.15.39:0/1392338534 conn(0x7f86ac0fdd40 msgr2=0x7f86ac104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 -- 172.21.15.39:0/1392338534 shutdown_connections 2024-05-04T03:22:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:20.013+0000 7f86b18f9640 1 -- 172.21.15.39:0/1392338534 wait complete. 2024-05-04T03:22:20.566 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:20 smithi039 ceph-mon[29715]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-05-04T03:22:20.594 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c858a07d', quiesce took 8.5 seconds. Will release after: 30.0 2024-05-04T03:22:20.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:20 smithi066 ceph-mon[33264]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-05-04T03:22:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:20 smithi167 ceph-mon[33614]: pgmap v732: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 24 MiB/s wr, 1.38k op/s 2024-05-04T03:22:21.779 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 510.0 s, 476.0 tps, lat 117.804 ms stddev 164.216 2024-05-04T03:22:22.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:22 smithi039 ceph-mon[29715]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 1.26k op/s 2024-05-04T03:22:22.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:22 smithi066 ceph-mon[33264]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 1.26k op/s 2024-05-04T03:22:22.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:22 smithi167 ceph-mon[33614]: pgmap v733: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 24 MiB/s wr, 1.26k op/s 2024-05-04T03:22:24.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:24 smithi039 ceph-mon[29715]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 22 MiB/s wr, 1.21k op/s 2024-05-04T03:22:24.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:24 smithi167 ceph-mon[33614]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 22 MiB/s wr, 1.21k op/s 2024-05-04T03:22:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:24 smithi066 ceph-mon[33264]: pgmap v734: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 22 MiB/s wr, 1.21k op/s 2024-05-04T03:22:26.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 515.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:22:26.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:26 smithi066 ceph-mon[33264]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-05-04T03:22:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:26 smithi167 ceph-mon[33614]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-05-04T03:22:26.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:26 smithi039 ceph-mon[29715]: pgmap v735: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-05-04T03:22:28.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:28 smithi066 ceph-mon[33264]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 13 MiB/s wr, 748 op/s 2024-05-04T03:22:28.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:28 smithi167 ceph-mon[33614]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 13 MiB/s wr, 748 op/s 2024-05-04T03:22:28.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:28 smithi039 ceph-mon[29715]: pgmap v736: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 13 MiB/s wr, 748 op/s 2024-05-04T03:22:30.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:30 smithi066 ceph-mon[33264]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.3 MiB/s wr, 545 op/s 2024-05-04T03:22:31.562 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:30 smithi039 ceph-mon[29715]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.3 MiB/s wr, 545 op/s 2024-05-04T03:22:31.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:30 smithi167 ceph-mon[33614]: pgmap v737: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.3 MiB/s wr, 545 op/s 2024-05-04T03:22:31.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 520.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:22:32.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:32 smithi066 ceph-mon[33264]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 303 op/s 2024-05-04T03:22:32.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:32 smithi167 ceph-mon[33614]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 303 op/s 2024-05-04T03:22:32.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:32 smithi039 ceph-mon[29715]: pgmap v738: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 303 op/s 2024-05-04T03:22:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 -- 172.21.15.39:0/2819472745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc104810 msgr2=0x7f7dcc10af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 --2- 172.21.15.39:0/2819472745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc104810 0x7f7dcc10af60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009ca0 tx=0x7f7dc402a8c0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 -- 172.21.15.39:0/2819472745 shutdown_connections 2024-05-04T03:22:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 --2- 172.21.15.39:0/2819472745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc104810 0x7f7dcc10af60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 --2- 172.21.15.39:0/2819472745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc103cb0 0x7f7dcc104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 --2- 172.21.15.39:0/2819472745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7dcc103150 0x7f7dcc103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 -- 172.21.15.39:0/2819472745 >> 172.21.15.39:0/2819472745 conn(0x7f7dcc0fd4f0 msgr2=0x7f7dcc0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 -- 172.21.15.39:0/2819472745 shutdown_connections 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.128+0000 7f7dd1596640 1 -- 172.21.15.39:0/2819472745 wait complete. 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dd1596640 1 Processor -- start 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dd1596640 1 -- start start 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dd1596640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc103150 0x7f7dcc10d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dd1596640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7dcc103cb0 0x7f7dcc10bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dd1596640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc10c260 0x7f7dcc10c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dd1596640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc10f8f0 con 0x7f7dcc10c260 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dd1596640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc10f770 con 0x7f7dcc103cb0 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dd1596640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc10fa70 con 0x7f7dcc103150 2024-05-04T03:22:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dcb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc10c260 0x7f7dcc10c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dc3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7dcc103cb0 0x7f7dcc10bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dcb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc10c260 0x7f7dcc10c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50518/0 (socket says 172.21.15.39:50518) 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/1324143802 learned_addr learned my addr 172.21.15.39:0/1324143802 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc103150 0x7f7dcc10d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dc3fff640 1 -- 172.21.15.39:0/1324143802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc103150 msgr2=0x7f7dcc10d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dc3fff640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc103150 0x7f7dcc10d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dc3fff640 1 -- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc10c260 msgr2=0x7f7dcc10c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dc3fff640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc10c260 0x7f7dcc10c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dc3fff640 1 -- 172.21.15.39:0/1324143802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc1a41e0 con 0x7f7dcc103cb0 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.129+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc10c260 0x7f7dcc10c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.130+0000 7f7dc3fff640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7dcc103cb0 0x7f7dcc10bd20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7db400e9f0 tx=0x7f7db400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.130+0000 7f7dc8ff9640 1 -- 172.21.15.39:0/1324143802 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db400ccc0 con 0x7f7dcc103cb0 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.130+0000 7f7dd1596640 1 -- 172.21.15.39:0/1324143802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc1a44d0 con 0x7f7dcc103cb0 2024-05-04T03:22:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.130+0000 7f7dc8ff9640 1 -- 172.21.15.39:0/1324143802 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7db4005020 con 0x7f7dcc103cb0 2024-05-04T03:22:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.130+0000 7f7dc8ff9640 1 -- 172.21.15.39:0/1324143802 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4004a00 con 0x7f7dcc103cb0 2024-05-04T03:22:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.130+0000 7f7dd1596640 1 -- 172.21.15.39:0/1324143802 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc1a4a20 con 0x7f7dcc103cb0 2024-05-04T03:22:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.132+0000 7f7dd844c740 1 Processor -- start 2024-05-04T03:22:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dd844c740 1 -- start start 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dd844c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556c358265d0 con 0x556c354ff4d0 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dd844c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c357df890 con 0x556c355201b0 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dd844c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c35822590 con 0x556c354f72f0 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dcb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dcb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:58888/0 (socket says 172.21.15.39:58888) 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/337949905 learned_addr learned my addr 172.21.15.39:0/337949905 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.133+0000 7f7dc8ff9640 1 -- 172.21.15.39:0/1324143802 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7db4004ba0 con 0x7f7dcc103cb0 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/337949905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 msgr2=0x556c358c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/337949905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 msgr2=0x556c358c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/337949905 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c35538210 con 0x556c355201b0 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358c1a60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7dc402aeb0 tx=0x7f7dc4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba481236ba54c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dc17fa640 1 -- 172.21.15.39:0/337949905 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4031e40 con 0x556c355201b0 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dc17fa640 1 -- 172.21.15.39:0/337949905 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dc4036030 con 0x556c355201b0 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dc17fa640 1 -- 172.21.15.39:0/337949905 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4004860 con 0x556c355201b0 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dc8ff9640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7db80b27e0 0x7f7db80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.134+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7db80b27e0 0x7f7db80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dc8ff9640 1 -- 172.21.15.39:0/1324143802 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7db40e2110 con 0x7f7dcc103cb0 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 -- 172.21.15.39:0/337949905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 msgr2=0x556c358c1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358c1a60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7dc402aeb0 tx=0x7f7dc4009a20 comp rx=0 tx=0).stop 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7db80b27e0 0x7f7db80b4cf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x556c358c2e40 tx=0x7f7dbc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 -- 172.21.15.39:0/337949905 shutdown_connections 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 --2- 172.21.15.39:0/337949905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358c1a60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 -- 172.21.15.39:0/337949905 >> 172.21.15.39:0/337949905 conn(0x556c35514300 msgr2=0x556c358bd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 -- 172.21.15.39:0/337949905 shutdown_connections 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.135+0000 7f7dd844c740 1 -- 172.21.15.39:0/337949905 wait complete. 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 1 -- start start 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 1 Processor -- start 2024-05-04T03:22:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 0x556c358cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 0x556c358cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c358d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556c358c9750 con 0x556c355201b0 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556c357e3370 con 0x556c354f72f0 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556c358c98d0 con 0x556c354ff4d0 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dd844c740 10 client.0 _ll_register_callbacks cb 0x556c35537a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dcb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c358d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dcb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c358d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50534/0 (socket says 172.21.15.39:50534) 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/161660988 learned_addr learned my addr 172.21.15.39:0/161660988 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 0x556c358cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.136+0000 7f7dc3fff640 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 0x556c358cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/161660988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 msgr2=0x556c358cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 0x556c358cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/161660988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 msgr2=0x556c358cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 0x556c358cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/161660988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c35604620 con 0x556c355201b0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c358d2a80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4031040 tx=0x7f7dc4003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427a526b4f89deb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7db27fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7db27fc640 1 -- 172.21.15.39:0/161660988 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc404a040 con 0x556c355201b0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7db27fc640 1 -- 172.21.15.39:0/161660988 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7dc4004150 con 0x556c355201b0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dd844c740 1 -- 172.21.15.39:0/161660988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc4036030 con 0x556c355201b0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7db27fc640 1 -- 172.21.15.39:0/161660988 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7dc40305c0 con 0x556c355201b0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dd844c740 10 client.16608 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7dd844c740 1 -- 172.21.15.39:0/161660988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c358c9750 con 0x556c355201b0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.137+0000 7f7db27fc640 1 -- 172.21.15.39:0/161660988 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dc4036030 con 0x556c355201b0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.138+0000 7f7db27fc640 1 -- 172.21.15.39:0/161660988 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7dc404d030 con 0x556c355201b0 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.138+0000 7f7dd844c740 10 client.16608 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.138+0000 7f7dd844c740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.138+0000 7f7dd844c740 20 client.16608 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.138+0000 7f7dd844c740 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c354ded80 0x556c358d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.138+0000 7f7dd844c740 4 client.16608 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.138+0000 7f7dd844c740 1 -- 172.21.15.39:0/161660988 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c358341e0 con 0x556c354ded80 2024-05-04T03:22:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.138+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c354ded80 0x556c358d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.139+0000 7f7db27fc640 10 client.16608 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.139+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c354ded80 0x556c358d8e30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5a19680d0f650c server_cookie=2e1a4bd2486754f0 in_seq=0 out_seq=0 2024-05-04T03:22:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.139+0000 7f7db27fc640 1 -- 172.21.15.39:0/161660988 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c358341e0 con 0x556c354ded80 2024-05-04T03:22:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.139+0000 7f7db27fc640 10 client.16608 handle_command_reply: tid=0 2024-05-04T03:22:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.139+0000 7f7dd844c740 1 client.16608 shutdown 2024-05-04T03:22:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 -- 172.21.15.39:0/161660988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 msgr2=0x556c358d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c358d2a80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4031040 tx=0x7f7dc4003ca0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 -- 172.21.15.39:0/161660988 shutdown_connections 2024-05-04T03:22:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c354ded80 0x556c358d8e30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c358d2a80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 0x556c358cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7db27fc640 0 client.16608 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 --2- 172.21.15.39:0/161660988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 0x556c358cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 -- 172.21.15.39:0/161660988 >> 172.21.15.39:0/161660988 conn(0x556c35514300 msgr2=0x556c358bc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 -- 172.21.15.39:0/161660988 shutdown_connections 2024-05-04T03:22:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 1 -- 172.21.15.39:0/161660988 wait complete. 2024-05-04T03:22:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.140+0000 7f7dd844c740 20 client.16608 trim_cache size 0 max 16384 2024-05-04T03:22:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.155+0000 7f7dd844c740 1 Processor -- start 2024-05-04T03:22:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.155+0000 7f7dd844c740 1 -- start start 2024-05-04T03:22:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358bd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dd844c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1ad3a0 con 0x556c354ff4d0 2024-05-04T03:22:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dd844c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1aeee0 con 0x556c355201b0 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dd844c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1ada70 con 0x556c354f72f0 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dc3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358bd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dc3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358bd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50536/0 (socket says 172.21.15.39:50536) 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dc3fff640 1 -- 172.21.15.39:0/265556760 learned_addr learned my addr 172.21.15.39:0/265556760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dc3fff640 1 -- 172.21.15.39:0/265556760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 msgr2=0x556c358c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dc3fff640 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dc3fff640 1 -- 172.21.15.39:0/265556760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 msgr2=0x556c358d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dc3fff640 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dc3fff640 1 -- 172.21.15.39:0/265556760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c358bd940 con 0x556c354ff4d0 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.156+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.157+0000 7f7dc3fff640 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358bd400 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7db40a96d0 tx=0x7f7db40a9090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91687b747f62addf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.157+0000 7f7dc0ff9640 1 -- 172.21.15.39:0/265556760 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db401c070 con 0x556c354ff4d0 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.157+0000 7f7dc0ff9640 1 -- 172.21.15.39:0/265556760 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7db40a9e40 con 0x556c354ff4d0 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.157+0000 7f7dc0ff9640 1 -- 172.21.15.39:0/265556760 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db40a3e60 con 0x556c354ff4d0 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- 172.21.15.39:0/265556760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 msgr2=0x556c358bd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358bd400 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7db40a96d0 tx=0x7f7db40a9090 comp rx=0 tx=0).stop 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- 172.21.15.39:0/265556760 shutdown_connections 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c355201b0 0x556c358d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c354ff4d0 0x556c358bd400 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 --2- 172.21.15.39:0/265556760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354f72f0 0x556c358c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- 172.21.15.39:0/265556760 >> 172.21.15.39:0/265556760 conn(0x556c35514300 msgr2=0x556c358bccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- 172.21.15.39:0/265556760 shutdown_connections 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- 172.21.15.39:0/265556760 wait complete. 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- start start 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 Processor -- start 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 0x556c358ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 0x556c358ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c3593f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1ae810 con 0x556c355201b0 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1ae140 con 0x556c354f72f0 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.158+0000 7f7dd844c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc1ab380 con 0x556c354ff4d0 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dd844c740 10 client.0 _ll_register_callbacks cb 0x7f7dbc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c3593f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c3593f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50544/0 (socket says 172.21.15.39:50544) 2024-05-04T03:22:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/908047363 learned_addr learned my addr 172.21.15.39:0/908047363 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dc3fff640 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 0x556c358ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 0x556c358ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/908047363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 msgr2=0x556c358ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 0x556c358ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/908047363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 msgr2=0x556c358ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 0x556c358ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 -- 172.21.15.39:0/908047363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c358cec60 con 0x556c355201b0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dcb7fe640 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c3593f220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4050720 tx=0x7f7dc4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dd8889b0e91f6e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7db27fc640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7db27fc640 1 -- 172.21.15.39:0/908047363 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4004150 con 0x556c355201b0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7db27fc640 1 -- 172.21.15.39:0/908047363 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7dc40042f0 con 0x556c355201b0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dd844c740 1 -- 172.21.15.39:0/908047363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7db40a9e40 con 0x556c355201b0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7db27fc640 1 -- 172.21.15.39:0/908047363 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7dc4030760 con 0x556c355201b0 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dd844c740 10 client.16614 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.159+0000 7f7dd844c740 1 -- 172.21.15.39:0/908047363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c359407f0 con 0x556c355201b0 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7db27fc640 1 -- 172.21.15.39:0/908047363 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7db40a9e40 con 0x556c355201b0 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7db27fc640 1 -- 172.21.15.39:0/908047363 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7dc404d070 con 0x556c355201b0 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7dd844c740 10 client.16614 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7dd844c740 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7dd844c740 20 client.16614 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7dd844c740 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c354b41c0 0x556c35943200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7dd844c740 4 client.16614 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7dd844c740 1 -- 172.21.15.39:0/908047363 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c3582e050 con 0x556c354b41c0 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.160+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c354b41c0 0x556c35943200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.161+0000 7f7db27fc640 10 client.16614 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.161+0000 7f7dcaffd640 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c354b41c0 0x556c35943200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885591e0b26463c8 server_cookie=ff9a125139ecd861 in_seq=0 out_seq=0 2024-05-04T03:22:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7db27fc640 1 -- 172.21.15.39:0/908047363 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c3582e050 con 0x556c354b41c0 2024-05-04T03:22:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7db27fc640 10 client.16614 handle_command_reply: tid=0 2024-05-04T03:22:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 client.16614 shutdown 2024-05-04T03:22:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 -- 172.21.15.39:0/908047363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 msgr2=0x556c3593f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c3593f220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4050720 tx=0x7f7dc4009540 comp rx=0 tx=0).stop 2024-05-04T03:22:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 -- 172.21.15.39:0/908047363 shutdown_connections 2024-05-04T03:22:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556c354b41c0 0x556c35943200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556c355201b0 0x556c3593f220 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556c354ff4d0 0x556c358ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 --2- 172.21.15.39:0/908047363 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556c354f72f0 0x556c358ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7db27fc640 0 client.16614 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 -- 172.21.15.39:0/908047363 >> 172.21.15.39:0/908047363 conn(0x556c35514300 msgr2=0x556c358c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 -- 172.21.15.39:0/908047363 shutdown_connections 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 1 -- 172.21.15.39:0/908047363 wait complete. 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.170+0000 7f7dd844c740 20 client.16614 trim_cache size 0 max 16384 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:22:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:22:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:22:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:22:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 -- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7db80b27e0 msgr2=0x7f7db80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7db80b27e0 0x7f7db80b4cf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x556c358c2e40 tx=0x7f7dbc00b390 comp rx=0 tx=0).stop 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 -- 172.21.15.39:0/1324143802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7dcc103cb0 msgr2=0x7f7dcc10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7dcc103cb0 0x7f7dcc10bd20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7db400e9f0 tx=0x7f7db400eeb0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 -- 172.21.15.39:0/1324143802 shutdown_connections 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7db80b27e0 0x7f7db80b4cf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dcc10c260 0x7f7dcc10c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7dcc103cb0 0x7f7dcc10bd20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 --2- 172.21.15.39:0/1324143802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dcc103150 0x7f7dcc10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 -- 172.21.15.39:0/1324143802 >> 172.21.15.39:0/1324143802 conn(0x7f7dcc0fd4f0 msgr2=0x7f7dcc106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 -- 172.21.15.39:0/1324143802 shutdown_connections 2024-05-04T03:22:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:33.171+0000 7f7dd1596640 1 -- 172.21.15.39:0/1324143802 wait complete. 2024-05-04T03:22:34.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:34 smithi066 ceph-mon[33264]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-05-04T03:22:34.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:34 smithi167 ceph-mon[33614]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-05-04T03:22:34.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:34 smithi039 ceph-mon[29715]: pgmap v739: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-05-04T03:22:36.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 525.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:22:36.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:36 smithi066 ceph-mon[33264]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 388 KiB/s wr, 47 op/s 2024-05-04T03:22:36.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:36 smithi167 ceph-mon[33614]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 388 KiB/s wr, 47 op/s 2024-05-04T03:22:36.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:36 smithi039 ceph-mon[29715]: pgmap v740: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 388 KiB/s wr, 47 op/s 2024-05-04T03:22:38.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:38 smithi066 ceph-mon[33264]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 165 KiB/s wr, 32 op/s 2024-05-04T03:22:38.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:38 smithi167 ceph-mon[33614]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 165 KiB/s wr, 32 op/s 2024-05-04T03:22:38.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:38 smithi039 ceph-mon[29715]: pgmap v741: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 165 KiB/s wr, 32 op/s 2024-05-04T03:22:40.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:40 smithi066 ceph-mon[33264]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 KiB/s wr, 36 op/s 2024-05-04T03:22:40.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:40 smithi167 ceph-mon[33614]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 KiB/s wr, 36 op/s 2024-05-04T03:22:40.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:40 smithi039 ceph-mon[29715]: pgmap v742: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 KiB/s wr, 36 op/s 2024-05-04T03:22:41.780 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 530.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:22:42.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:42 smithi066 ceph-mon[33264]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 223 KiB/s wr, 36 op/s 2024-05-04T03:22:42.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:42 smithi167 ceph-mon[33614]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 223 KiB/s wr, 36 op/s 2024-05-04T03:22:42.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:42 smithi039 ceph-mon[29715]: pgmap v743: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 223 KiB/s wr, 36 op/s 2024-05-04T03:22:44.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:44 smithi066 ceph-mon[33264]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 250 KiB/s wr, 37 op/s 2024-05-04T03:22:44.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:44 smithi167 ceph-mon[33614]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 250 KiB/s wr, 37 op/s 2024-05-04T03:22:44.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:44 smithi039 ceph-mon[29715]: pgmap v744: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 250 KiB/s wr, 37 op/s 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.282+0000 7fa4487ea640 1 -- 172.21.15.39:0/2086977992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 msgr2=0x7fa440104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.282+0000 7fa4487ea640 1 --2- 172.21.15.39:0/2086977992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 0x7fa440104970 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009900 tx=0x7fa43c02a410 comp rx=0 tx=0).stop 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 -- 172.21.15.39:0/2086977992 shutdown_connections 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 --2- 172.21.15.39:0/2086977992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa440109bb0 0x7fa44010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 --2- 172.21.15.39:0/2086977992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa440104eb0 0x7fa4401052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 --2- 172.21.15.39:0/2086977992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 0x7fa440104970 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 -- 172.21.15.39:0/2086977992 >> 172.21.15.39:0/2086977992 conn(0x7fa4400fee60 msgr2=0x7fa440101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 -- 172.21.15.39:0/2086977992 shutdown_connections 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 -- 172.21.15.39:0/2086977992 wait complete. 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 Processor -- start 2024-05-04T03:22:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 -- start start 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 0x7fa44019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa440104eb0 0x7fa44019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa440109bb0 0x7fa4401a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa44010f650 con 0x7fa440103c70 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa44010f4d0 con 0x7fa440104eb0 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.283+0000 7fa4487ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa44010f7d0 con 0x7fa440109bb0 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 0x7fa44019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 0x7fa44019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55458/0 (socket says 172.21.15.39:55458) 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 -- 172.21.15.39:0/1128441553 learned_addr learned my addr 172.21.15.39:0/1128441553 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa445d85640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa440104eb0 0x7fa44019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446d87640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa440109bb0 0x7fa4401a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 -- 172.21.15.39:0/1128441553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa440109bb0 msgr2=0x7fa4401a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa440109bb0 0x7fa4401a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 -- 172.21.15.39:0/1128441553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa440104eb0 msgr2=0x7fa44019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa440104eb0 0x7fa44019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 -- 172.21.15.39:0/1128441553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4401a1dc0 con 0x7fa440103c70 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446d87640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa440109bb0 0x7fa4401a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa446586640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 0x7fa44019a280 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa43c005760 tx=0x7fa43c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa4377fe640 1 -- 172.21.15.39:0/1128441553 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c03f070 con 0x7fa440103c70 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa4377fe640 1 -- 172.21.15.39:0/1128441553 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa43c031da0 con 0x7fa440103c70 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa4487ea640 1 -- 172.21.15.39:0/1128441553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4401a2050 con 0x7fa440103c70 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa4377fe640 1 -- 172.21.15.39:0/1128441553 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c035030 con 0x7fa440103c70 2024-05-04T03:22:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.284+0000 7fa4487ea640 1 -- 172.21.15.39:0/1128441553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4401a2590 con 0x7fa440103c70 2024-05-04T03:22:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.287+0000 7fa4377fe640 1 -- 172.21.15.39:0/1128441553 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa43c042070 con 0x7fa440103c70 2024-05-04T03:22:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.287+0000 7fa44f773740 1 Processor -- start 2024-05-04T03:22:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.287+0000 7fa44f773740 1 -- start start 2024-05-04T03:22:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa44f773740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f01be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa44f773740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f01c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa44f773740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f0232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa44f773740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f59ef7c5d0 con 0x55f59ec761b0 2024-05-04T03:22:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa44f773740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f59ef35890 con 0x55f59ec4d2f0 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa44f773740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f59ef78590 con 0x55f59ec554d0 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f01be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f01be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55474/0 (socket says 172.21.15.39:55474) 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 -- 172.21.15.39:0/702164537 learned_addr learned my addr 172.21.15.39:0/702164537 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446586640 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f01c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa445d85640 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f0232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa4377fe640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa42c0b2860 0x7fa42c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 -- 172.21.15.39:0/702164537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 msgr2=0x55f59f0232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f0232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 -- 172.21.15.39:0/702164537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 msgr2=0x55f59f01c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f01c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 -- 172.21.15.39:0/702164537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59ec8e210 con 0x55f59ec761b0 2024-05-04T03:22:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446586640 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f01c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa445d85640 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f0232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa4377fe640 1 -- 172.21.15.39:0/1128441553 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa43c102a50 con 0x7fa440103c70 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.288+0000 7fa446d87640 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f01be70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa438009fb0 tx=0x7fa438010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfebd9806ab0a23 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.289+0000 7fa446d87640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa42c0b2860 0x7fa42c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.289+0000 7fa434ff9640 1 -- 172.21.15.39:0/702164537 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438016070 con 0x55f59ec761b0 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.289+0000 7fa434ff9640 1 -- 172.21.15.39:0/702164537 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa43801b460 con 0x55f59ec761b0 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.289+0000 7fa434ff9640 1 -- 172.21.15.39:0/702164537 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438011600 con 0x55f59ec761b0 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 -- 172.21.15.39:0/702164537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 msgr2=0x55f59f01be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa446d87640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa42c0b2860 0x7fa42c0b4d70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa438011010 tx=0x7fa438011d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f01be70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa438009fb0 tx=0x7fa438010800 comp rx=0 tx=0).stop 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 -- 172.21.15.39:0/702164537 shutdown_connections 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f0232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f01c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 --2- 172.21.15.39:0/702164537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f01be70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 -- 172.21.15.39:0/702164537 >> 172.21.15.39:0/702164537 conn(0x55f59ec6a300 msgr2=0x55f59f0177b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 -- 172.21.15.39:0/702164537 shutdown_connections 2024-05-04T03:22:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 -- 172.21.15.39:0/702164537 wait complete. 2024-05-04T03:22:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 -- start start 2024-05-04T03:22:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.290+0000 7fa44f773740 1 Processor -- start 2024-05-04T03:22:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa44f773740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec4d2f0 0x55f59f025a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa44f773740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec554d0 0x55f59f025fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa44f773740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec761b0 0x55f59f02cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa44f773740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f59f023d40 con 0x55f59ec554d0 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa44f773740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55f59ef39370 con 0x55f59ec761b0 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa44f773740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f59f023ec0 con 0x55f59ec4d2f0 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa44f773740 10 client.0 _ll_register_callbacks cb 0x55f59ec8da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec554d0 0x55f59f025fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec554d0 0x55f59f025fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55482/0 (socket says 172.21.15.39:55482) 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 -- 172.21.15.39:0/1364375443 learned_addr learned my addr 172.21.15.39:0/1364375443 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa446d87640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec761b0 0x55f59f02cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa446586640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec4d2f0 0x55f59f025a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 -- 172.21.15.39:0/1364375443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec4d2f0 msgr2=0x55f59f025a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec4d2f0 0x55f59f025a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 -- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec761b0 msgr2=0x55f59f02cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec761b0 0x55f59f02cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 -- 172.21.15.39:0/1364375443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59ed5a620 con 0x55f59ec554d0 2024-05-04T03:22:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa446586640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec4d2f0 0x55f59f025a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa446d87640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec761b0 0x55f59f02cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa445d85640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec554d0 0x55f59f025fc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa43000ba70 tx=0x7fa43000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e02b2e86e8015b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa429ffb640 1 -- 172.21.15.39:0/1364375443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43001b640 con 0x55f59ec554d0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa429ffb640 1 -- 172.21.15.39:0/1364375443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa43001b7e0 con 0x55f59ec554d0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.291+0000 7fa44f773740 1 -- 172.21.15.39:0/1364375443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43801b460 con 0x55f59ec554d0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa429ffb640 1 -- 172.21.15.39:0/1364375443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa43001b9e0 con 0x55f59ec554d0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa44f773740 10 client.16626 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa44f773740 1 -- 172.21.15.39:0/1364375443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f59f02ff20 con 0x55f59ec554d0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa429ffb640 1 -- 172.21.15.39:0/1364375443 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa43801b460 con 0x55f59ec554d0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa429ffb640 1 -- 172.21.15.39:0/1364375443 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa430019020 con 0x55f59ec554d0 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa44f773740 10 client.16626 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa44f773740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa44f773740 20 client.16626 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa44f773740 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f59ec34d80 0x55f59f033640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa44f773740 4 client.16626 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa44f773740 1 -- 172.21.15.39:0/1364375443 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f59ef8a1e0 con 0x55f59ec34d80 2024-05-04T03:22:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.292+0000 7fa446586640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f59ec34d80 0x55f59f033640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.293+0000 7fa446586640 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f59ec34d80 0x55f59f033640 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45b1c6a21e43103 server_cookie=eb3f54e8e8c589cb in_seq=0 out_seq=0 2024-05-04T03:22:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.293+0000 7fa429ffb640 10 client.16626 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa429ffb640 1 -- 172.21.15.39:0/1364375443 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f59ef8a1e0 con 0x55f59ec34d80 2024-05-04T03:22:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa429ffb640 10 client.16626 handle_command_reply: tid=0 2024-05-04T03:22:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 client.16626 shutdown 2024-05-04T03:22:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 -- 172.21.15.39:0/1364375443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec554d0 msgr2=0x55f59f025fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec554d0 0x55f59f025fc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa43000ba70 tx=0x7fa43000bf30 comp rx=0 tx=0).stop 2024-05-04T03:22:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 -- 172.21.15.39:0/1364375443 shutdown_connections 2024-05-04T03:22:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f59ec34d80 0x55f59f033640 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec761b0 0x55f59f02cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec554d0 0x55f59f025fc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa429ffb640 0 client.16626 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 --2- 172.21.15.39:0/1364375443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec4d2f0 0x55f59f025a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 -- 172.21.15.39:0/1364375443 >> 172.21.15.39:0/1364375443 conn(0x55f59ec6a300 msgr2=0x55f59f01a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 -- 172.21.15.39:0/1364375443 shutdown_connections 2024-05-04T03:22:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 1 -- 172.21.15.39:0/1364375443 wait complete. 2024-05-04T03:22:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.294+0000 7fa44f773740 20 client.16626 trim_cache size 0 max 16384 2024-05-04T03:22:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.309+0000 7fa44f773740 1 Processor -- start 2024-05-04T03:22:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.309+0000 7fa44f773740 1 -- start start 2024-05-04T03:22:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa44f773740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f017770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa44f773740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f02ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa44f773740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f02d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa44f773740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401a9060 con 0x55f59ec761b0 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa44f773740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401aaba0 con 0x55f59ec4d2f0 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa44f773740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401a9730 con 0x55f59ec554d0 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f02d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f02d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55496/0 (socket says 172.21.15.39:55496) 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 -- 172.21.15.39:0/3488655229 learned_addr learned my addr 172.21.15.39:0/3488655229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa445d85640 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f02ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446586640 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f017770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 -- 172.21.15.39:0/3488655229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 msgr2=0x55f59f02ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f02ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 -- 172.21.15.39:0/3488655229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 msgr2=0x55f59f017770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f017770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 -- 172.21.15.39:0/3488655229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f59f018110 con 0x55f59ec761b0 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa445d85640 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f02ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa446d87640 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f02d380 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa438009ed0 tx=0x7fa438027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac57182e629880a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.310+0000 7fa42bfff640 1 -- 172.21.15.39:0/3488655229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438036070 con 0x55f59ec761b0 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa42bfff640 1 -- 172.21.15.39:0/3488655229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa438012970 con 0x55f59ec761b0 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa42bfff640 1 -- 172.21.15.39:0/3488655229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43802d040 con 0x55f59ec761b0 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 -- 172.21.15.39:0/3488655229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 msgr2=0x55f59f02d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f02d380 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa438009ed0 tx=0x7fa438027000 comp rx=0 tx=0).stop 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 -- 172.21.15.39:0/3488655229 shutdown_connections 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec761b0 0x55f59f02d380 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec554d0 0x55f59f02ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 --2- 172.21.15.39:0/3488655229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec4d2f0 0x55f59f017770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 -- 172.21.15.39:0/3488655229 >> 172.21.15.39:0/3488655229 conn(0x55f59ec6a300 msgr2=0x55f59f017060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 -- 172.21.15.39:0/3488655229 shutdown_connections 2024-05-04T03:22:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.311+0000 7fa44f773740 1 -- 172.21.15.39:0/3488655229 wait complete. 2024-05-04T03:22:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 1 -- start start 2024-05-04T03:22:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 1 Processor -- start 2024-05-04T03:22:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec4d2f0 0x55f59f02a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec554d0 0x55f59f02ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec761b0 0x55f59f02bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401aa4d0 con 0x55f59ec4d2f0 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401a9e00 con 0x55f59ec554d0 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4401a7040 con 0x55f59ec761b0 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa44f773740 10 client.0 _ll_register_callbacks cb 0x7fa430004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446586640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec4d2f0 0x55f59f02a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446586640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec4d2f0 0x55f59f02a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55506/0 (socket says 172.21.15.39:55506) 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446586640 1 -- 172.21.15.39:0/1674777843 learned_addr learned my addr 172.21.15.39:0/1674777843 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446d87640 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec761b0 0x55f59f02bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa445d85640 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec554d0 0x55f59f02ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446586640 1 -- 172.21.15.39:0/1674777843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec761b0 msgr2=0x55f59f02bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446586640 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec761b0 0x55f59f02bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446586640 1 -- 172.21.15.39:0/1674777843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec554d0 msgr2=0x55f59f02ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446586640 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec554d0 0x55f59f02ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.312+0000 7fa446586640 1 -- 172.21.15.39:0/1674777843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59f02c320 con 0x55f59ec4d2f0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa446586640 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec4d2f0 0x55f59f02a8f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa43c02a9d0 tx=0x7fa43c0c98b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c629760aff671bef server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa429ffb640 1 -- 172.21.15.39:0/1674777843 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c049040 con 0x55f59ec4d2f0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa429ffb640 1 -- 172.21.15.39:0/1674777843 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa43c031910 con 0x55f59ec4d2f0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 1 -- 172.21.15.39:0/1674777843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa438012970 con 0x55f59ec4d2f0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa429ffb640 1 -- 172.21.15.39:0/1674777843 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa43c0ce430 con 0x55f59ec4d2f0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 10 client.16632 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 1 -- 172.21.15.39:0/1674777843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f59f02d5f0 con 0x55f59ec4d2f0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa429ffb640 1 -- 172.21.15.39:0/1674777843 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa438012970 con 0x55f59ec4d2f0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa429ffb640 1 -- 172.21.15.39:0/1674777843 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa43c03b460 con 0x55f59ec4d2f0 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 10 client.16632 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 20 client.16632 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f59ec0a1c0 0x55f59f09dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 4 client.16632 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.313+0000 7fa44f773740 1 -- 172.21.15.39:0/1674777843 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f59ef84050 con 0x55f59ec0a1c0 2024-05-04T03:22:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.314+0000 7fa445d85640 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f59ec0a1c0 0x55f59f09dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.314+0000 7fa429ffb640 10 client.16632 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.314+0000 7fa445d85640 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f59ec0a1c0 0x55f59f09dd70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93be80ba30f2c6db server_cookie=4dfdce136bd2e2bc in_seq=0 out_seq=0 2024-05-04T03:22:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa429ffb640 1 -- 172.21.15.39:0/1674777843 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f59ef84050 con 0x55f59ec0a1c0 2024-05-04T03:22:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa429ffb640 10 client.16632 handle_command_reply: tid=0 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 client.16632 shutdown 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 -- 172.21.15.39:0/1674777843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec4d2f0 msgr2=0x55f59f02a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec4d2f0 0x55f59f02a8f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa43c02a9d0 tx=0x7fa43c0c98b0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 -- 172.21.15.39:0/1674777843 shutdown_connections 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55f59ec0a1c0 0x55f59f09dd70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f59ec761b0 0x55f59f02bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f59ec554d0 0x55f59f02ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa429ffb640 0 client.16632 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 --2- 172.21.15.39:0/1674777843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f59ec4d2f0 0x55f59f02a8f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 -- 172.21.15.39:0/1674777843 >> 172.21.15.39:0/1674777843 conn(0x55f59ec6a300 msgr2=0x55f59f011970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 -- 172.21.15.39:0/1674777843 shutdown_connections 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 1 -- 172.21.15.39:0/1674777843 wait complete. 2024-05-04T03:22:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.323+0000 7fa44f773740 20 client.16632 trim_cache size 0 max 16384 2024-05-04T03:22:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:22:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:22:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:22:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 -- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa42c0b2860 msgr2=0x7fa42c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa42c0b2860 0x7fa42c0b4d70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa438011010 tx=0x7fa438011d20 comp rx=0 tx=0).stop 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 -- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 msgr2=0x7fa44019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 0x7fa44019a280 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa43c005760 tx=0x7fa43c003d50 comp rx=0 tx=0).stop 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 -- 172.21.15.39:0/1128441553 shutdown_connections 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa42c0b2860 0x7fa42c0b4d70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa440109bb0 0x7fa4401a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa440104eb0 0x7fa44019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 --2- 172.21.15.39:0/1128441553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa440103c70 0x7fa44019a280 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 -- 172.21.15.39:0/1128441553 >> 172.21.15.39:0/1128441553 conn(0x7fa4400fee60 msgr2=0x7fa440105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 -- 172.21.15.39:0/1128441553 shutdown_connections 2024-05-04T03:22:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:46.324+0000 7fa4487ea640 1 -- 172.21.15.39:0/1128441553 wait complete. 2024-05-04T03:22:46.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:46 smithi039 ceph-mon[29715]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-05-04T03:22:46.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 535.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:22:46.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:46 smithi066 ceph-mon[33264]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-05-04T03:22:46.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:46 smithi167 ceph-mon[33614]: pgmap v745: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-05-04T03:22:48.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:48 smithi066 ceph-mon[33264]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 232 KiB/s wr, 35 op/s 2024-05-04T03:22:48.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:48 smithi167 ceph-mon[33614]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 232 KiB/s wr, 35 op/s 2024-05-04T03:22:48.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:48 smithi039 ceph-mon[29715]: pgmap v746: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 232 KiB/s wr, 35 op/s 2024-05-04T03:22:50.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:22:50.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:50 smithi066 ceph-mon[33264]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 256 KiB/s wr, 36 op/s 2024-05-04T03:22:50.867 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:22:50.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:50 smithi167 ceph-mon[33614]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 256 KiB/s wr, 36 op/s 2024-05-04T03:22:50.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:50 smithi039 ceph-mon[29715]: pgmap v747: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 256 KiB/s wr, 36 op/s 2024-05-04T03:22:51.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 540.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:22:52.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/2431906776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c5410e910 msgr2=0x7f8c54110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:52.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/2431906776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c5410e910 0x7f8c54110df0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b0a0 tx=0x7f8c4c00b3b0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/2431906776 shutdown_connections 2024-05-04T03:22:52.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:52 smithi039 ceph-mon[29715]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 272 KiB/s wr, 37 op/s 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/2431906776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c5410e910 0x7f8c54110df0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/2431906776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c541054e0 0x7f8c541058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/2431906776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c54105e30 0x7f8c5410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/2431906776 >> 172.21.15.39:0/2431906776 conn(0x7f8c540fef30 msgr2=0x7f8c54101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/2431906776 shutdown_connections 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.619+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/2431906776 wait complete. 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5c3a8700 1 Processor -- start 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5c3a8700 1 -- start start 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5c3a8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c541054e0 0x7f8c541a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5c3a8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c54105e30 0x7f8c541a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5c3a8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c5410e910 0x7f8c5419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5c3a8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54114320 con 0x7f8c54105e30 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5c3a8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541141a0 con 0x7f8c541054e0 2024-05-04T03:22:52.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5c3a8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541144a0 con 0x7f8c5410e910 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c59943700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c54105e30 0x7f8c541a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c59943700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c54105e30 0x7f8c541a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55524/0 (socket says 172.21.15.39:55524) 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c59943700 1 -- 172.21.15.39:0/1066980714 learned_addr learned my addr 172.21.15.39:0/1066980714 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5a144700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c541054e0 0x7f8c541a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.621+0000 7f8c5a945700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c5410e910 0x7f8c5419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c59943700 1 -- 172.21.15.39:0/1066980714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c5410e910 msgr2=0x7f8c5419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c59943700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c5410e910 0x7f8c5419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c59943700 1 -- 172.21.15.39:0/1066980714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c541054e0 msgr2=0x7f8c541a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c59943700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c541054e0 0x7f8c541a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c59943700 1 -- 172.21.15.39:0/1066980714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5419b170 con 0x7f8c54105e30 2024-05-04T03:22:52.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c59943700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c54105e30 0x7f8c541a0de0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400d950 tx=0x7f8c4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:52.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c4b7fe700 1 -- 172.21.15.39:0/1066980714 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44016070 con 0x7f8c54105e30 2024-05-04T03:22:52.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c4b7fe700 1 -- 172.21.15.39:0/1066980714 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c4400fd20 con 0x7f8c54105e30 2024-05-04T03:22:52.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5419b400 con 0x7f8c54105e30 2024-05-04T03:22:52.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c4b7fe700 1 -- 172.21.15.39:0/1066980714 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400e3b0 con 0x7f8c54105e30 2024-05-04T03:22:52.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.622+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541a7730 con 0x7f8c54105e30 2024-05-04T03:22:52.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.624+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c540512c0 con 0x7f8c54105e30 2024-05-04T03:22:52.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.625+0000 7f8c4b7fe700 1 -- 172.21.15.39:0/1066980714 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8c44019070 con 0x7f8c54105e30 2024-05-04T03:22:52.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.626+0000 7f8c4b7fe700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c400b2900 0x7f8c400b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:52.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.626+0000 7f8c5a144700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c400b2900 0x7f8c400b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:52.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.626+0000 7f8c4b7fe700 1 -- 172.21.15.39:0/1066980714 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8c440e1c10 con 0x7f8c54105e30 2024-05-04T03:22:52.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.626+0000 7f8c5a144700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c400b2900 0x7f8c400b4e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8c5419bc00 tx=0x7f8c5000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:52.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.631+0000 7f8c4b7fe700 1 -- 172.21.15.39:0/1066980714 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c440a9940 con 0x7f8c54105e30 2024-05-04T03:22:52.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:52 smithi066 ceph-mon[33264]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 272 KiB/s wr, 37 op/s 2024-05-04T03:22:52.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.903+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c5402e5f0 con 0x7f8c54105e30 2024-05-04T03:22:52.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.904+0000 7f8c4b7fe700 1 -- 172.21.15.39:0/1066980714 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f8c440a9240 con 0x7f8c54105e30 2024-05-04T03:22:52.906 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:22:52.906 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:22:52.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c400b2900 msgr2=0x7f8c400b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:52.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c400b2900 0x7f8c400b4e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8c5419bc00 tx=0x7f8c5000b410 comp rx=0 tx=0).stop 2024-05-04T03:22:52.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c54105e30 msgr2=0x7f8c541a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:52.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c54105e30 0x7f8c541a0de0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400d950 tx=0x7f8c4400dc60 comp rx=0 tx=0).stop 2024-05-04T03:22:52.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 shutdown_connections 2024-05-04T03:22:52.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c400b2900 0x7f8c400b4e10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c5410e910 0x7f8c5419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c541054e0 0x7f8c541a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 --2- 172.21.15.39:0/1066980714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c54105e30 0x7f8c541a0de0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 >> 172.21.15.39:0/1066980714 conn(0x7f8c540fef30 msgr2=0x7f8c540ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 shutdown_connections 2024-05-04T03:22:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:52.906+0000 7f8c5c3a8700 1 -- 172.21.15.39:0/1066980714 wait complete. 2024-05-04T03:22:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:22:52.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:52 smithi167 ceph-mon[33614]: pgmap v748: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 272 KiB/s wr, 37 op/s 2024-05-04T03:22:53.604 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id c858a07d --release --await' 2024-05-04T03:22:53.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id c858a07d --release --await 2024-05-04T03:22:53.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:53 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1066980714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:53.857 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:22:53.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:53 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1066980714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:53.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:53 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1066980714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:54 smithi066 ceph-mon[33264]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 273 KiB/s wr, 37 op/s 2024-05-04T03:22:54.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:54 smithi167 ceph-mon[33614]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 273 KiB/s wr, 37 op/s 2024-05-04T03:22:54.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:54 smithi039 ceph-mon[29715]: pgmap v749: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 273 KiB/s wr, 37 op/s 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.595+0000 7fad0786e700 1 -- 172.21.15.39:0/254037992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 msgr2=0x7fad00103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.595+0000 7fad0786e700 1 --2- 172.21.15.39:0/254037992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 0x7fad00103930 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7facfc0099c0 tx=0x7facfc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.595+0000 7fad0786e700 1 -- 172.21.15.39:0/254037992 shutdown_connections 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.595+0000 7fad0786e700 1 --2- 172.21.15.39:0/254037992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 0x7fad00103930 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.595+0000 7fad0786e700 1 --2- 172.21.15.39:0/254037992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad00108750 0x7fad001100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.595+0000 7fad0786e700 1 --2- 172.21.15.39:0/254037992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad00107e00 0x7fad00108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.595+0000 7fad0786e700 1 -- 172.21.15.39:0/254037992 >> 172.21.15.39:0/254037992 conn(0x7fad000fef90 msgr2=0x7fad001013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.596+0000 7fad0786e700 1 -- 172.21.15.39:0/254037992 shutdown_connections 2024-05-04T03:22:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.596+0000 7fad0786e700 1 -- 172.21.15.39:0/254037992 wait complete. 2024-05-04T03:22:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0786e700 1 Processor -- start 2024-05-04T03:22:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0786e700 1 -- start start 2024-05-04T03:22:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0786e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 0x7fad0019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0786e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad00107e00 0x7fad0019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0786e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad00108750 0x7fad0019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0786e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad00113250 con 0x7fad00103520 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0786e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad001130d0 con 0x7fad00107e00 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0786e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fad001133d0 con 0x7fad00108750 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0560a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 0x7fad0019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0560a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 0x7fad0019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55544/0 (socket says 172.21.15.39:55544) 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0560a700 1 -- 172.21.15.39:0/2231952675 learned_addr learned my addr 172.21.15.39:0/2231952675 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad04e09700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad00107e00 0x7fad0019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad05e0b700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad00108750 0x7fad0019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0560a700 1 -- 172.21.15.39:0/2231952675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad00108750 msgr2=0x7fad0019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0560a700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad00108750 0x7fad0019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0560a700 1 -- 172.21.15.39:0/2231952675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad00107e00 msgr2=0x7fad0019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0560a700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad00107e00 0x7fad0019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad0560a700 1 -- 172.21.15.39:0/2231952675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0019fc60 con 0x7fad00103520 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.597+0000 7fad04e09700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad00107e00 0x7fad0019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.598+0000 7fad0560a700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 0x7fad0019ad90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7facfc006010 tx=0x7facfc004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.598+0000 7facf67fc700 1 -- 172.21.15.39:0/2231952675 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc026070 con 0x7fad00103520 2024-05-04T03:22:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.598+0000 7facf67fc700 1 -- 172.21.15.39:0/2231952675 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facfc00f460 con 0x7fad00103520 2024-05-04T03:22:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.598+0000 7fad0786e700 1 -- 172.21.15.39:0/2231952675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0019fef0 con 0x7fad00103520 2024-05-04T03:22:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.598+0000 7facf67fc700 1 -- 172.21.15.39:0/2231952675 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc009e60 con 0x7fad00103520 2024-05-04T03:22:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.598+0000 7fad0786e700 1 -- 172.21.15.39:0/2231952675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad001a03d0 con 0x7fad00103520 2024-05-04T03:22:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.600+0000 7facf67fc700 1 -- 172.21.15.39:0/2231952675 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7facfc029070 con 0x7fad00103520 2024-05-04T03:22:55.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.601+0000 7facf67fc700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7facec0b2880 0x7facec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.601+0000 7fad04e09700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7facec0b2880 0x7facec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.601+0000 7facf67fc700 1 -- 172.21.15.39:0/2231952675 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7facfc0e2b90 con 0x7fad00103520 2024-05-04T03:22:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad16282b80 1 Processor -- start 2024-05-04T03:22:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad04e09700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7facec0b2880 0x7facec0b4d90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fad001098a0 tx=0x7facf000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad16282b80 1 -- start start 2024-05-04T03:22:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad16282b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bbc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad16282b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbcf20 0x557e27bc73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad16282b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bc78e0 0x557e27bc9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad16282b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557e27adf960 con 0x557e27bc78e0 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad16282b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557e27a7e8c0 con 0x557e27bbc5d0 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad16282b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557e27adb920 con 0x557e27bbcf20 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad05e0b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bc78e0 0x557e27bc9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad0560a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bbc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad0560a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bbc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:47154/0 (socket says 172.21.15.39:47154) 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad0560a700 1 -- 172.21.15.39:0/3918525706 learned_addr learned my addr 172.21.15.39:0/3918525706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.602+0000 7fad04e09700 1 --2- 172.21.15.39:0/3918525706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbcf20 0x557e27bc73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7fad05e0b700 1 -- 172.21.15.39:0/3918525706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbcf20 msgr2=0x557e27bc73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7fad05e0b700 1 --2- 172.21.15.39:0/3918525706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbcf20 0x557e27bc73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7fad05e0b700 1 -- 172.21.15.39:0/3918525706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 msgr2=0x557e27bbc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7fad05e0b700 1 --2- 172.21.15.39:0/3918525706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bbc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7fad05e0b700 1 -- 172.21.15.39:0/3918525706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e27a2c9d0 con 0x557e27bc78e0 2024-05-04T03:22:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7fad05e0b700 1 --2- 172.21.15.39:0/3918525706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bc78e0 0x557e27bc9d00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7facf8009310 tx=0x7facf8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9e43532036a681 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7faceb7fe700 1 -- 172.21.15.39:0/3918525706 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8019020 con 0x557e27bc78e0 2024-05-04T03:22:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7faceb7fe700 1 -- 172.21.15.39:0/3918525706 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facf8015ad0 con 0x557e27bc78e0 2024-05-04T03:22:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.603+0000 7faceb7fe700 1 -- 172.21.15.39:0/3918525706 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf80148b0 con 0x557e27bc78e0 2024-05-04T03:22:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 -- 172.21.15.39:0/3918525706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bc78e0 msgr2=0x557e27bc9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 --2- 172.21.15.39:0/3918525706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bc78e0 0x557e27bc9d00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7facf8009310 tx=0x7facf8009920 comp rx=0 tx=0).stop 2024-05-04T03:22:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 -- 172.21.15.39:0/3918525706 shutdown_connections 2024-05-04T03:22:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 --2- 172.21.15.39:0/3918525706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bc78e0 0x557e27bc9d00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 --2- 172.21.15.39:0/3918525706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bbc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 --2- 172.21.15.39:0/3918525706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbcf20 0x557e27bc73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 -- 172.21.15.39:0/3918525706 >> 172.21.15.39:0/3918525706 conn(0x557e27a3ff40 msgr2=0x557e27bb8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 -- 172.21.15.39:0/3918525706 shutdown_connections 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.604+0000 7fad16282b80 1 -- 172.21.15.39:0/3918525706 wait complete. 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 1 -- start start 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 1 Processor -- start 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bc4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bc4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bc78e0 0x557e27bc52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557e27bcea80 con 0x557e27bbcf20 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557e27a45d30 con 0x557e27bbc5d0 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557e27bcec00 con 0x557e27bc78e0 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.605+0000 7fad16282b80 10 client.0 _ll_register_callbacks cb 0x557e27a77a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bc4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bc4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55562/0 (socket says 172.21.15.39:55562) 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 -- 172.21.15.39:0/2839859858 learned_addr learned my addr 172.21.15.39:0/2839859858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad05e0b700 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bc78e0 0x557e27bc52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad0560a700 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bc4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 -- 172.21.15.39:0/2839859858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bc78e0 msgr2=0x557e27bc52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bc78e0 0x557e27bc52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 -- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 msgr2=0x557e27bc4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bc4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 -- 172.21.15.39:0/2839859858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e27a7b790 con 0x557e27bbcf20 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad0560a700 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bc4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad04e09700 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bc4d90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7facf000fe30 tx=0x7facf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fd14af8ac18226 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7face8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7face8ff9700 1 -- 172.21.15.39:0/2839859858 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0004220 con 0x557e27bbcf20 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7face8ff9700 1 -- 172.21.15.39:0/2839859858 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7facf00043c0 con 0x557e27bbcf20 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.606+0000 7fad16282b80 1 -- 172.21.15.39:0/2839859858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf8015ad0 con 0x557e27bbcf20 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7face8ff9700 1 -- 172.21.15.39:0/2839859858 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7facf000fe60 con 0x557e27bbcf20 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad16282b80 10 client.16650 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad16282b80 1 -- 172.21.15.39:0/2839859858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557e27bd96b0 con 0x557e27bbcf20 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7face8ff9700 1 -- 172.21.15.39:0/2839859858 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facf8015ad0 con 0x557e27bbcf20 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7face8ff9700 1 -- 172.21.15.39:0/2839859858 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7facf001f500 con 0x557e27bbcf20 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad16282b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad16282b80 10 client.16650 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad16282b80 20 client.16650 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad16282b80 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557e27bdbd40 0x557e27bde1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad16282b80 4 client.16650 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad16282b80 1 -- 172.21.15.39:0/2839859858 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e27aed340 con 0x557e27bdbd40 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.607+0000 7fad0560a700 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557e27bdbd40 0x557e27bde1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.608+0000 7fad0560a700 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557e27bdbd40 0x557e27bde1c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb23260854dc7c2 server_cookie=7ee6e6f456efaa97 in_seq=0 out_seq=0 2024-05-04T03:22:55.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.608+0000 7face8ff9700 10 client.16650 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:55.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7face8ff9700 1 -- 172.21.15.39:0/2839859858 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e27aed340 con 0x557e27bdbd40 2024-05-04T03:22:55.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7face8ff9700 10 client.16650 handle_command_reply: tid=0 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 client.16650 shutdown 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 -- 172.21.15.39:0/2839859858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 msgr2=0x557e27bc4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bc4d90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7facf000fe30 tx=0x7facf0004750 comp rx=0 tx=0).stop 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 -- 172.21.15.39:0/2839859858 shutdown_connections 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557e27bdbd40 0x557e27bde1c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bc78e0 0x557e27bc52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bc4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 --2- 172.21.15.39:0/2839859858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bc4d90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 -- 172.21.15.39:0/2839859858 >> 172.21.15.39:0/2839859858 conn(0x557e27a3ff40 msgr2=0x557e27bbe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7face8ff9700 0 client.16650 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 -- 172.21.15.39:0/2839859858 shutdown_connections 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 1 -- 172.21.15.39:0/2839859858 wait complete. 2024-05-04T03:22:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.609+0000 7fad16282b80 20 client.16650 trim_cache size 0 max 16384 2024-05-04T03:22:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.629+0000 7fad16282b80 1 Processor -- start 2024-05-04T03:22:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.629+0000 7fad16282b80 1 -- start start 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad16282b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27a3ff40 0x557e27bc5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad16282b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bb8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad16282b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bb8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad16282b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad001ac7e0 con 0x557e27bbcf20 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad16282b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad001ae320 con 0x557e27bbc5d0 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad16282b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fad001aceb0 con 0x557e27a3ff40 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bb8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bb8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55570/0 (socket says 172.21.15.39:55570) 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 -- 172.21.15.39:0/3562141685 learned_addr learned my addr 172.21.15.39:0/3562141685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad0560a700 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27a3ff40 0x557e27bc5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad04e09700 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bb8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 -- 172.21.15.39:0/3562141685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27a3ff40 msgr2=0x557e27bc5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27a3ff40 0x557e27bc5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 -- 172.21.15.39:0/3562141685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 msgr2=0x557e27bb8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bb8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 -- 172.21.15.39:0/3562141685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e27bc2dc0 con 0x557e27bbcf20 2024-05-04T03:22:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.630+0000 7fad05e0b700 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bb8570 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7facf8004cd0 tx=0x7facf8009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b18dee56f46952 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.631+0000 7faceaffd700 1 -- 172.21.15.39:0/3562141685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8007e60 con 0x557e27bbcf20 2024-05-04T03:22:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.631+0000 7faceaffd700 1 -- 172.21.15.39:0/3562141685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facf8004e30 con 0x557e27bbcf20 2024-05-04T03:22:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.631+0000 7faceaffd700 1 -- 172.21.15.39:0/3562141685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8019040 con 0x557e27bbcf20 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.631+0000 7fad16282b80 1 -- 172.21.15.39:0/3562141685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 msgr2=0x557e27bb8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.631+0000 7fad16282b80 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bb8570 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7facf8004cd0 tx=0x7facf8009d80 comp rx=0 tx=0).stop 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 -- 172.21.15.39:0/3562141685 shutdown_connections 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27bbcf20 0x557e27bb8570 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27a3ff40 0x557e27bc5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 --2- 172.21.15.39:0/3562141685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbc5d0 0x557e27bb8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 -- 172.21.15.39:0/3562141685 >> 172.21.15.39:0/3562141685 conn(0x557e27bc0480 msgr2=0x557e27bb7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 -- 172.21.15.39:0/3562141685 shutdown_connections 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 -- 172.21.15.39:0/3562141685 wait complete. 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 -- start start 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.632+0000 7fad16282b80 1 Processor -- start 2024-05-04T03:22:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad16282b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27a3ff40 0x557e27bd4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad16282b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbc5d0 0x557e27bd24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad16282b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbcf20 0x557e27bd29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad16282b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad001adc50 con 0x557e27a3ff40 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad16282b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fad001ad580 con 0x557e27bbcf20 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad16282b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fad001aa780 con 0x557e27bbc5d0 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad16282b80 10 client.0 _ll_register_callbacks cb 0x557e27a77a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27a3ff40 0x557e27bd4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27a3ff40 0x557e27bd4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55576/0 (socket says 172.21.15.39:55576) 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 -- 172.21.15.39:0/566728904 learned_addr learned my addr 172.21.15.39:0/566728904 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad04e09700 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbc5d0 0x557e27bd24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad05e0b700 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbcf20 0x557e27bd29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 -- 172.21.15.39:0/566728904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbc5d0 msgr2=0x557e27bd24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbc5d0 0x557e27bd24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 -- 172.21.15.39:0/566728904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbcf20 msgr2=0x557e27bd29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbcf20 0x557e27bd29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 -- 172.21.15.39:0/566728904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e27bd2f30 con 0x557e27a3ff40 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7fad0560a700 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27a3ff40 0x557e27bd4120 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7facfc0e06f0 tx=0x7facfc00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8fd690086074fe server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.633+0000 7face8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7face8ff9700 1 -- 172.21.15.39:0/566728904 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0c3070 con 0x557e27a3ff40 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7face8ff9700 1 -- 172.21.15.39:0/566728904 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7facfc00f930 con 0x557e27a3ff40 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 1 -- 172.21.15.39:0/566728904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf8004e30 con 0x557e27a3ff40 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7face8ff9700 1 -- 172.21.15.39:0/566728904 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7facfc0ae600 con 0x557e27a3ff40 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 10 client.16656 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 1 -- 172.21.15.39:0/566728904 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557e27c471f0 con 0x557e27a3ff40 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7face8ff9700 1 -- 172.21.15.39:0/566728904 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facf8004e30 con 0x557e27a3ff40 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7face8ff9700 1 -- 172.21.15.39:0/566728904 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7facfc0b2c90 con 0x557e27a3ff40 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 10 client.16656 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 10 client.16656 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 20 client.16656 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557e27bc78e0 0x557e27c49910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 4 client.16656 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c858a07d", "release": true, "await": true}] 2024-05-04T03:22:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad16282b80 1 -- 172.21.15.39:0/566728904 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c858a07d", "release": true, "await": true}) -- 0x557e27ae73e0 con 0x557e27bc78e0 2024-05-04T03:22:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.634+0000 7fad04e09700 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557e27bc78e0 0x557e27c49910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.635+0000 7face8ff9700 10 client.16656 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.635+0000 7fad04e09700 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557e27bc78e0 0x557e27c49910 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ae23ce94aab7c9 server_cookie=8f5c021005a92738 in_seq=0 out_seq=0 2024-05-04T03:22:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7face8ff9700 1 -- 172.21.15.39:0/566728904 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557e27ae73e0 con 0x557e27bc78e0 2024-05-04T03:22:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7face8ff9700 10 client.16656 handle_command_reply: tid=0 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 client.16656 shutdown 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 -- 172.21.15.39:0/566728904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27a3ff40 msgr2=0x557e27bd4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27a3ff40 0x557e27bd4120 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7facfc0e06f0 tx=0x7facfc00b5b0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 -- 172.21.15.39:0/566728904 shutdown_connections 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557e27bc78e0 0x557e27c49910 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557e27a3ff40 0x557e27bd4120 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557e27bbcf20 0x557e27bd29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7face8ff9700 0 client.16656 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 --2- 172.21.15.39:0/566728904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557e27bbc5d0 0x557e27bd24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 -- 172.21.15.39:0/566728904 >> 172.21.15.39:0/566728904 conn(0x557e27bc0480 msgr2=0x557e27bb8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 -- 172.21.15.39:0/566728904 shutdown_connections 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.640+0000 7fad16282b80 1 -- 172.21.15.39:0/566728904 wait complete. 2024-05-04T03:22:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad16282b80 20 client.16656 trim_cache size 0 max 16384 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 32, 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "c858a07d": { 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 32, 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:22:55.645 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 90.0, 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:22:55.646 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:22:55.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 -- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7facec0b2880 msgr2=0x7facec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7facec0b2880 0x7facec0b4d90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fad001098a0 tx=0x7facf000b5f0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 -- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 msgr2=0x7fad0019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 0x7fad0019ad90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7facfc006010 tx=0x7facfc004690 comp rx=0 tx=0).stop 2024-05-04T03:22:55.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 -- 172.21.15.39:0/2231952675 shutdown_connections 2024-05-04T03:22:55.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7facec0b2880 0x7facec0b4d90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad00103520 0x7fad0019ad90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad00108750 0x7fad0019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 --2- 172.21.15.39:0/2231952675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad00107e00 0x7fad0019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 -- 172.21.15.39:0/2231952675 >> 172.21.15.39:0/2231952675 conn(0x7fad000fef90 msgr2=0x7fad0006c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.641+0000 7fad0786e700 1 -- 172.21.15.39:0/2231952675 shutdown_connections 2024-05-04T03:22:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:55.642+0000 7fad0786e700 1 -- 172.21.15.39:0/2231952675 wait complete. 2024-05-04T03:22:56.440 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c858a07d', seconds elapsed: 44.4 2024-05-04T03:22:56.440 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-04T03:22:56.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:22:56.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:56 smithi039 ceph-mon[29715]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 29 op/s 2024-05-04T03:22:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 545.0 s, 280.9 tps, lat 2651.509 ms stddev 9296.780 2024-05-04T03:22:56.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:56 smithi066 ceph-mon[33264]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 29 op/s 2024-05-04T03:22:56.858 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:22:56.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:56 smithi167 ceph-mon[33614]: pgmap v750: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 KiB/s wr, 29 op/s 2024-05-04T03:22:58.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.797+0000 7fd1414ac700 1 -- 172.21.15.39:0/3190983168 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138004230 con 0x7fd1340977d0 2024-05-04T03:22:58.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.798+0000 7fd1434b0700 1 -- 172.21.15.39:0/3190983168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1340977d0 msgr2=0x7fd134097be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:58.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.798+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3190983168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1340977d0 0x7fd134097be0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd138009990 tx=0x7fd138009ca0 comp rx=0 tx=0).stop 2024-05-04T03:22:58.802 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:58 smithi039 ceph-mon[29715]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 474 KiB/s wr, 45 op/s 2024-05-04T03:22:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.799+0000 7fd1434b0700 1 -- 172.21.15.39:0/3190983168 shutdown_connections 2024-05-04T03:22:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.799+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3190983168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1340977d0 0x7fd134097be0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.799+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3190983168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1340a0bb0 0x7fd1340a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.799+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3190983168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd134098120 0x7fd1340a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.799+0000 7fd1434b0700 1 -- 172.21.15.39:0/3190983168 >> 172.21.15.39:0/3190983168 conn(0x7fd13400b060 msgr2=0x7fd13400b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.799+0000 7fd1434b0700 1 -- 172.21.15.39:0/3190983168 shutdown_connections 2024-05-04T03:22:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.799+0000 7fd1434b0700 1 -- 172.21.15.39:0/3190983168 wait complete. 2024-05-04T03:22:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.800+0000 7fd1434b0700 1 Processor -- start 2024-05-04T03:22:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.800+0000 7fd1434b0700 1 -- start start 2024-05-04T03:22:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd1434b0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1340977d0 0x7fd1340a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd1434b0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd134098120 0x7fd13409d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd1434b0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340a0bb0 0x7fd13409dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd1434b0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd134096a80 con 0x7fd134098120 2024-05-04T03:22:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd1434b0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd134096900 con 0x7fd1340a0bb0 2024-05-04T03:22:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd1434b0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd134096c00 con 0x7fd1340977d0 2024-05-04T03:22:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd142caf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340a0bb0 0x7fd13409dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:58.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd142caf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340a0bb0 0x7fd13409dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46130/0 (socket says 172.21.15.39:46130) 2024-05-04T03:22:58.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.801+0000 7fd142caf700 1 -- 172.21.15.39:0/3095571932 learned_addr learned my addr 172.21.15.39:0/3095571932 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:58.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd142caf700 1 -- 172.21.15.39:0/3095571932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1340977d0 msgr2=0x7fd1340a0580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd142caf700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1340977d0 0x7fd1340a0580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd142caf700 1 -- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd134098120 msgr2=0x7fd13409d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd142caf700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd134098120 0x7fd13409d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd142caf700 1 -- 172.21.15.39:0/3095571932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13409e510 con 0x7fd1340a0bb0 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd142caf700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340a0bb0 0x7fd13409dce0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd13c052d10 tx=0x7fd13c06b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd1337fe700 1 -- 172.21.15.39:0/3095571932 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c084070 con 0x7fd1340a0bb0 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd1337fe700 1 -- 172.21.15.39:0/3095571932 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd13c07fb20 con 0x7fd1340a0bb0 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.802+0000 7fd1337fe700 1 -- 172.21.15.39:0/3095571932 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c080b50 con 0x7fd1340a0bb0 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.803+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13413fa70 con 0x7fd1340a0bb0 2024-05-04T03:22:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.803+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13413ff30 con 0x7fd1340a0bb0 2024-05-04T03:22:58.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.804+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd134005550 con 0x7fd1340a0bb0 2024-05-04T03:22:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.806+0000 7fd1337fe700 1 -- 172.21.15.39:0/3095571932 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd13c087070 con 0x7fd1340a0bb0 2024-05-04T03:22:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.807+0000 7fd1337fe700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1280b29d0 0x7fd1280b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.808+0000 7fd1337fe700 1 -- 172.21.15.39:0/3095571932 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd13c150280 con 0x7fd1340a0bb0 2024-05-04T03:22:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.808+0000 7fd1424ae700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1280b29d0 0x7fd1280b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.808+0000 7fd1424ae700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1280b29d0 0x7fd1280b4ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd1380098e0 tx=0x7fd138005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:58.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:58.813+0000 7fd1337fe700 1 -- 172.21.15.39:0/3095571932 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd13c117fb0 con 0x7fd1340a0bb0 2024-05-04T03:22:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:58 smithi066 ceph-mon[33264]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 474 KiB/s wr, 45 op/s 2024-05-04T03:22:58.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:58 smithi167 ceph-mon[33614]: pgmap v751: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 474 KiB/s wr, 45 op/s 2024-05-04T03:22:59.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.287+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd134140220 con 0x7fd1340a0bb0 2024-05-04T03:22:59.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.289+0000 7fd1337fe700 1 -- 172.21.15.39:0/3095571932 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7fd13c1178b0 con 0x7fd1340a0bb0 2024-05-04T03:22:59.292 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:22:59.293 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:22:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1280b29d0 msgr2=0x7fd1280b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1280b29d0 0x7fd1280b4ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd1380098e0 tx=0x7fd138005900 comp rx=0 tx=0).stop 2024-05-04T03:22:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340a0bb0 msgr2=0x7fd13409dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340a0bb0 0x7fd13409dce0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd13c052d10 tx=0x7fd13c06b940 comp rx=0 tx=0).stop 2024-05-04T03:22:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 shutdown_connections 2024-05-04T03:22:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1280b29d0 0x7fd1280b4ee0 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd1380098e0 tx=0x7fd138005900 comp rx=0 tx=0).stop 2024-05-04T03:22:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340a0bb0 0x7fd13409dce0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1340977d0 0x7fd1340a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 --2- 172.21.15.39:0/3095571932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd134098120 0x7fd13409d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.293+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 >> 172.21.15.39:0/3095571932 conn(0x7fd13400b060 msgr2=0x7fd1340a4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:59.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.294+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 shutdown_connections 2024-05-04T03:22:59.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:22:59.294+0000 7fd1434b0700 1 -- 172.21.15.39:0/3095571932 wait complete. 2024-05-04T03:22:59.298 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:22:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.477+0000 7f2920a54640 1 -- 172.21.15.39:0/1495244703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c1070d0 msgr2=0x7f291c1074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.477+0000 7f2920a54640 1 --2- 172.21.15.39:0/1495244703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c1070d0 0x7f291c1074e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2910009900 tx=0x7f291002a410 comp rx=0 tx=0).stop 2024-05-04T03:22:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.478+0000 7f2920a54640 1 -- 172.21.15.39:0/1495244703 shutdown_connections 2024-05-04T03:22:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.478+0000 7f2920a54640 1 --2- 172.21.15.39:0/1495244703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c10bdd0 0x7f291c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.478+0000 7f2920a54640 1 --2- 172.21.15.39:0/1495244703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c1070d0 0x7f291c1074e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.478+0000 7f2920a54640 1 --2- 172.21.15.39:0/1495244703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f291c105e90 0x7f291c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.478+0000 7f2920a54640 1 -- 172.21.15.39:0/1495244703 >> 172.21.15.39:0/1495244703 conn(0x7f291c101060 msgr2=0x7f291c103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.478+0000 7f2920a54640 1 -- 172.21.15.39:0/1495244703 shutdown_connections 2024-05-04T03:22:59.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.478+0000 7f2920a54640 1 -- 172.21.15.39:0/1495244703 wait complete. 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.479+0000 7f2920a54640 1 Processor -- start 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2920a54640 1 -- start start 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2920a54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c105e90 0x7f291c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2920a54640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f291c1070d0 0x7f291c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2920a54640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c10bdd0 0x7f291c1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2920a54640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f291c111850 con 0x7f291c105e90 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2920a54640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1116d0 con 0x7f291c1070d0 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2920a54640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1119d0 con 0x7f291c10bdd0 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2919d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f291c1070d0 0x7f291c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f291a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c105e90 0x7f291c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2919d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f291c1070d0 0x7f291c19c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46138/0 (socket says 172.21.15.39:46138) 2024-05-04T03:22:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f291a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c105e90 0x7f291c19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52070/0 (socket says 172.21.15.39:52070) 2024-05-04T03:22:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f2919d9b640 1 -- 172.21.15.39:0/2227154941 learned_addr learned my addr 172.21.15.39:0/2227154941 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.480+0000 7f291ad9d640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c10bdd0 0x7f291c1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.481+0000 7f291a59c640 1 -- 172.21.15.39:0/2227154941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c10bdd0 msgr2=0x7f291c1a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.481+0000 7f291a59c640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c10bdd0 0x7f291c1a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.481+0000 7f291a59c640 1 -- 172.21.15.39:0/2227154941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f291c1070d0 msgr2=0x7f291c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.481+0000 7f291a59c640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f291c1070d0 0x7f291c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.481+0000 7f291a59c640 1 -- 172.21.15.39:0/2227154941 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c06c820 con 0x7f291c105e90 2024-05-04T03:22:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.481+0000 7f291a59c640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c105e90 0x7f291c19c420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f291c105760 tx=0x7f290c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.482+0000 7f290b7fe640 1 -- 172.21.15.39:0/2227154941 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c00cc40 con 0x7f291c105e90 2024-05-04T03:22:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.482+0000 7f290b7fe640 1 -- 172.21.15.39:0/2227154941 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f290c0044b0 con 0x7f291c105e90 2024-05-04T03:22:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.482+0000 7f2920a54640 1 -- 172.21.15.39:0/2227154941 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c06cb10 con 0x7f291c105e90 2024-05-04T03:22:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.482+0000 7f290b7fe640 1 -- 172.21.15.39:0/2227154941 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c003e10 con 0x7f291c105e90 2024-05-04T03:22:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.482+0000 7f2920a54640 1 -- 172.21.15.39:0/2227154941 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c06d060 con 0x7f291c105e90 2024-05-04T03:22:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f29279e0740 1 Processor -- start 2024-05-04T03:22:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f29279e0740 1 -- start start 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f29279e0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e138a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f29279e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f29279e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e13fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f290b7fe640 1 -- 172.21.15.39:0/2227154941 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f290c003fb0 con 0x7f291c105e90 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f29279e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c6e09d5d0 con 0x557c6dd764d0 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f29279e0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c6e056890 con 0x557c6dd971b0 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.486+0000 7f29279e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557c6e099590 con 0x557c6dd6e2f0 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e138fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34718/0 (socket says 172.21.15.39:34718) 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e138a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46154/0 (socket says 172.21.15.39:46154) 2024-05-04T03:22:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291a59c640 1 -- 172.21.15.39:0/1373788000 learned_addr learned my addr 172.21.15.39:0/1373788000 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f2919d9b640 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e13fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291a59c640 1 -- 172.21.15.39:0/1373788000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 msgr2=0x557c6e138a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291a59c640 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e138a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291a59c640 1 -- 172.21.15.39:0/1373788000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 msgr2=0x557c6e13fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291a59c640 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e13fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.487+0000 7f291a59c640 1 -- 172.21.15.39:0/1373788000 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6ddaf210 con 0x557c6dd6e2f0 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.488+0000 7f2919d9b640 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e13fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.488+0000 7f291a59c640 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e138fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f290c0e84d0 tx=0x7f290c0e8990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a2c0844b90df93 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.488+0000 7f290b7fe640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f29040b2930 0x7f29040b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.489+0000 7f2908ff9640 1 -- 172.21.15.39:0/1373788000 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0f5070 con 0x557c6dd6e2f0 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.489+0000 7f291ad9d640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f29040b2930 0x7f29040b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.489+0000 7f2908ff9640 1 -- 172.21.15.39:0/1373788000 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f290c09a2c0 con 0x557c6dd6e2f0 2024-05-04T03:22:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.489+0000 7f2908ff9640 1 -- 172.21.15.39:0/1373788000 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0203e0 con 0x557c6dd6e2f0 2024-05-04T03:22:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.489+0000 7f290b7fe640 1 -- 172.21.15.39:0/2227154941 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f290c0e2560 con 0x7f291c105e90 2024-05-04T03:22:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.490+0000 7f291ad9d640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f29040b2930 0x7f29040b4e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557c6e139c30 tx=0x7f2914009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.490+0000 7f29279e0740 1 -- 172.21.15.39:0/1373788000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 msgr2=0x557c6e138fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.491+0000 7f29279e0740 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e138fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f290c0e84d0 tx=0x7f290c0e8990 comp rx=0 tx=0).stop 2024-05-04T03:22:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.491+0000 7f29279e0740 1 -- 172.21.15.39:0/1373788000 shutdown_connections 2024-05-04T03:22:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.491+0000 7f29279e0740 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e13fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.491+0000 7f29279e0740 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e138fe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.491+0000 7f29279e0740 1 --2- 172.21.15.39:0/1373788000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e138a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.491+0000 7f29279e0740 1 -- 172.21.15.39:0/1373788000 >> 172.21.15.39:0/1373788000 conn(0x557c6dd8b300 msgr2=0x557c6e1343a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.491+0000 7f29279e0740 1 -- 172.21.15.39:0/1373788000 shutdown_connections 2024-05-04T03:22:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.491+0000 7f29279e0740 1 -- 172.21.15.39:0/1373788000 wait complete. 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.492+0000 7f29279e0740 1 -- start start 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.492+0000 7f29279e0740 1 Processor -- start 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.492+0000 7f29279e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e14ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.492+0000 7f29279e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e14b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.492+0000 7f29279e0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e152330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.492+0000 7f29279e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c6e140930 con 0x557c6dd764d0 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.492+0000 7f29279e0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557c6e05a370 con 0x557c6dd971b0 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.492+0000 7f29279e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557c6e140ab0 con 0x557c6dd6e2f0 2024-05-04T03:22:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e14ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46164/0 (socket says 172.21.15.39:46164) 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f29279e0740 10 client.0 _ll_register_callbacks cb 0x557c6ddaea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291ad9d640 1 -- 172.21.15.39:0/3938567427 learned_addr learned my addr 172.21.15.39:0/3938567427 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f2919d9b640 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e14b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291ad9d640 1 -- 172.21.15.39:0/3938567427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 msgr2=0x557c6e14ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291ad9d640 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e14ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291ad9d640 1 -- 172.21.15.39:0/3938567427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 msgr2=0x557c6e14b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291ad9d640 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e14b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.493+0000 7f291ad9d640 1 -- 172.21.15.39:0/3938567427 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6de7b620 con 0x557c6dd971b0 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f291ad9d640 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e152330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2914009cd0 tx=0x7f2914004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7ea39828552533 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f2901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f2901ffb640 1 -- 172.21.15.39:0/3938567427 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291400f830 con 0x557c6dd971b0 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f2901ffb640 1 -- 172.21.15.39:0/3938567427 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f291400f9d0 con 0x557c6dd971b0 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f29279e0740 1 -- 172.21.15.39:0/3938567427 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c09a2c0 con 0x557c6dd971b0 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f2901ffb640 1 -- 172.21.15.39:0/3938567427 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f291400fbd0 con 0x557c6dd971b0 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f29279e0740 10 client.26249 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f29279e0740 1 -- 172.21.15.39:0/3938567427 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c6e1552a0 con 0x557c6dd971b0 2024-05-04T03:22:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.494+0000 7f2901ffb640 1 -- 172.21.15.39:0/3938567427 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f290c09a2c0 con 0x557c6dd971b0 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.495+0000 7f2901ffb640 1 -- 172.21.15.39:0/3938567427 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2914026070 con 0x557c6dd971b0 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.495+0000 7f29279e0740 10 client.26249 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.495+0000 7f29279e0740 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.495+0000 7f29279e0740 20 client.26249 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.495+0000 7f29279e0740 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557c6dd55d80 0x557c6e158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.495+0000 7f29279e0740 4 client.26249 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.495+0000 7f29279e0740 1 -- 172.21.15.39:0/3938567427 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c6e0ab1e0 con 0x557c6dd55d80 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.495+0000 7f291a59c640 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557c6dd55d80 0x557c6e158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.496+0000 7f291a59c640 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557c6dd55d80 0x557c6e158a00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08618274522463e server_cookie=4ea6153638de1ae1 in_seq=0 out_seq=0 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.496+0000 7f2901ffb640 10 client.26249 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.496+0000 7f2901ffb640 1 -- 172.21.15.39:0/3938567427 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c6e0ab1e0 con 0x557c6dd55d80 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.496+0000 7f2901ffb640 10 client.26249 handle_command_reply: tid=0 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.496+0000 7f29279e0740 1 client.26249 shutdown 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 -- 172.21.15.39:0/3938567427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 msgr2=0x557c6e152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e152330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2914009cd0 tx=0x7f2914004ab0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 -- 172.21.15.39:0/3938567427 shutdown_connections 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557c6dd55d80 0x557c6e158a00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f2901ffb640 0 client.26249 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd971b0 0x557c6e152330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e14b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 --2- 172.21.15.39:0/3938567427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e14ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 -- 172.21.15.39:0/3938567427 >> 172.21.15.39:0/3938567427 conn(0x557c6dd8b300 msgr2=0x557c6e137300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 -- 172.21.15.39:0/3938567427 shutdown_connections 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 1 -- 172.21.15.39:0/3938567427 wait complete. 2024-05-04T03:22:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.497+0000 7f29279e0740 20 client.26249 trim_cache size 0 max 16384 2024-05-04T03:22:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.524+0000 7f29279e0740 1 Processor -- start 2024-05-04T03:22:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.524+0000 7f29279e0740 1 -- start start 2024-05-04T03:22:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f29279e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd55d80 0x557c6e133c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f29279e0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd6e2f0 0x557c6e152250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f29279e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e134140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f291a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd55d80 0x557c6e133c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f291a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd55d80 0x557c6e133c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34736/0 (socket says 172.21.15.39:34736) 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f2919d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd6e2f0 0x557c6e152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f2919d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd6e2f0 0x557c6e152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46172/0 (socket says 172.21.15.39:46172) 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f291ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.525+0000 7f291ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52098/0 (socket says 172.21.15.39:52098) 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.526+0000 7f291a59c640 1 -- 172.21.15.39:0/759220226 learned_addr learned my addr 172.21.15.39:0/759220226 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.526+0000 7f29279e0740 1 -- 172.21.15.39:0/759220226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f291c0719a0 con 0x557c6dd764d0 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.526+0000 7f29279e0740 1 -- 172.21.15.39:0/759220226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f291c0734e0 con 0x557c6dd6e2f0 2024-05-04T03:22:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.526+0000 7f29279e0740 1 -- 172.21.15.39:0/759220226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f291c072070 con 0x557c6dd55d80 2024-05-04T03:22:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.526+0000 7f291a59c640 1 -- 172.21.15.39:0/759220226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd6e2f0 msgr2=0x557c6e152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.526+0000 7f291a59c640 1 --2- 172.21.15.39:0/759220226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd6e2f0 0x557c6e152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.527+0000 7f291a59c640 1 -- 172.21.15.39:0/759220226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 msgr2=0x557c6e134140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.527+0000 7f291a59c640 1 --2- 172.21.15.39:0/759220226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e134140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.527+0000 7f291a59c640 1 -- 172.21.15.39:0/759220226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6e134d00 con 0x557c6dd55d80 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.527+0000 7f291ad9d640 1 --2- 172.21.15.39:0/759220226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.527+0000 7f291a59c640 1 --2- 172.21.15.39:0/759220226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd55d80 0x557c6e133c00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f290c0e8e90 tx=0x7f290c0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6487ff0126cee8a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.527+0000 7f2903fff640 1 -- 172.21.15.39:0/759220226 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c00fe40 con 0x557c6dd55d80 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.527+0000 7f2903fff640 1 -- 172.21.15.39:0/759220226 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f290c018e50 con 0x557c6dd55d80 2024-05-04T03:22:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.527+0000 7f2903fff640 1 -- 172.21.15.39:0/759220226 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c01f340 con 0x557c6dd55d80 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 -- 172.21.15.39:0/759220226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd55d80 msgr2=0x557c6e133c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 --2- 172.21.15.39:0/759220226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd55d80 0x557c6e133c00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f290c0e8e90 tx=0x7f290c0cc040 comp rx=0 tx=0).stop 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 -- 172.21.15.39:0/759220226 shutdown_connections 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 --2- 172.21.15.39:0/759220226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 --2- 172.21.15.39:0/759220226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd6e2f0 0x557c6e152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 --2- 172.21.15.39:0/759220226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd55d80 0x557c6e133c00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 -- 172.21.15.39:0/759220226 >> 172.21.15.39:0/759220226 conn(0x557c6dd8b300 msgr2=0x557c6e13cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 -- 172.21.15.39:0/759220226 shutdown_connections 2024-05-04T03:22:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 -- 172.21.15.39:0/759220226 wait complete. 2024-05-04T03:22:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.529+0000 7f29279e0740 1 -- start start 2024-05-04T03:22:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f29279e0740 1 Processor -- start 2024-05-04T03:22:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f29279e0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd55d80 0x557c6e14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f29279e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e14ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f29279e0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e150e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f29279e0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f291c072e10 con 0x557c6dd764d0 2024-05-04T03:22:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f291a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd55d80 0x557c6e14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f291a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd55d80 0x557c6e14f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46188/0 (socket says 172.21.15.39:46188) 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f291a59c640 1 -- 172.21.15.39:0/3662939953 learned_addr learned my addr 172.21.15.39:0/3662939953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f2919d9b640 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e14ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f291c072740 con 0x557c6dd55d80 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.530+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f291c06f980 con 0x557c6dd6e2f0 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f29279e0740 10 client.0 _ll_register_callbacks cb 0x7f2914009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f291ad9d640 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f291a59c640 1 -- 172.21.15.39:0/3662939953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 msgr2=0x557c6e14ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f291a59c640 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e14ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f291a59c640 1 -- 172.21.15.39:0/3662939953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 msgr2=0x557c6e150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f291a59c640 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f291a59c640 1 -- 172.21.15.39:0/3662939953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6e151410 con 0x557c6dd55d80 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f291a59c640 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd55d80 0x557c6e14f9e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f290c0a6da0 tx=0x7f290c00c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5646e77438f0e446 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.531+0000 7f2901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:22:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.537+0000 7f2901ffb640 1 -- 172.21.15.39:0/3662939953 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0c2020 con 0x557c6dd55d80 2024-05-04T03:22:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.537+0000 7f2901ffb640 1 -- 172.21.15.39:0/3662939953 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f290c014230 con 0x557c6dd55d80 2024-05-04T03:22:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.537+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c018e50 con 0x557c6dd55d80 2024-05-04T03:22:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.538+0000 7f2901ffb640 1 -- 172.21.15.39:0/3662939953 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f290c0aae60 con 0x557c6dd55d80 2024-05-04T03:22:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.539+0000 7f29279e0740 10 client.26255 fetch_fsmap learned FSMap version 32 2024-05-04T03:22:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.539+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c6e1c0db0 con 0x557c6dd55d80 2024-05-04T03:22:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.539+0000 7f2901ffb640 1 -- 172.21.15.39:0/3662939953 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f290c018e50 con 0x557c6dd55d80 2024-05-04T03:22:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.540+0000 7f2901ffb640 1 -- 172.21.15.39:0/3662939953 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f290c0b2be0 con 0x557c6dd55d80 2024-05-04T03:22:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.540+0000 7f29279e0740 10 client.26255 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:22:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.540+0000 7f29279e0740 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.540+0000 7f29279e0740 20 client.26255 populate_metadata read hostname 'smithi039' 2024-05-04T03:22:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.540+0000 7f29279e0740 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557c6dd971b0 0x557c6e1c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.540+0000 7f29279e0740 4 client.26255 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.540+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c6e0a5050 con 0x557c6dd971b0 2024-05-04T03:22:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.540+0000 7f2919d9b640 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557c6dd971b0 0x557c6e1c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.541+0000 7f2901ffb640 10 client.26255 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.542+0000 7f2919d9b640 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557c6dd971b0 0x557c6e1c3230 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e890772326d88a3 server_cookie=e77f0801ebd5162c in_seq=0 out_seq=0 2024-05-04T03:22:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f2901ffb640 1 -- 172.21.15.39:0/3662939953 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c6e0a5050 con 0x557c6dd971b0 2024-05-04T03:22:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f2901ffb640 10 client.26255 handle_command_reply: tid=0 2024-05-04T03:22:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 client.26255 shutdown 2024-05-04T03:22:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd55d80 msgr2=0x557c6e14f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd55d80 0x557c6e14f9e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f290c0a6da0 tx=0x7f290c00c750 comp rx=0 tx=0).stop 2024-05-04T03:22:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 shutdown_connections 2024-05-04T03:22:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557c6dd971b0 0x557c6e1c3230 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c6dd764d0 0x557c6e150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c6dd6e2f0 0x557c6e14ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 --2- 172.21.15.39:0/3662939953 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557c6dd55d80 0x557c6e14f9e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 >> 172.21.15.39:0/3662939953 conn(0x557c6dd8b300 msgr2=0x557c6e1371f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f2901ffb640 0 client.26255 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:22:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.546+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 shutdown_connections 2024-05-04T03:22:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.547+0000 7f29279e0740 1 -- 172.21.15.39:0/3662939953 wait complete. 2024-05-04T03:22:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.547+0000 7f29279e0740 20 client.26255 trim_cache size 0 max 16384 2024-05-04T03:22:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:22:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:22:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:22:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.548+0000 7f2920a54640 1 -- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f29040b2930 msgr2=0x7f29040b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.548+0000 7f2920a54640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f29040b2930 0x7f29040b4e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557c6e139c30 tx=0x7f2914009500 comp rx=0 tx=0).stop 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.548+0000 7f2920a54640 1 -- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c105e90 msgr2=0x7f291c19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.548+0000 7f2920a54640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c105e90 0x7f291c19c420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f291c105760 tx=0x7f290c00ee50 comp rx=0 tx=0).stop 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.548+0000 7f2920a54640 1 -- 172.21.15.39:0/2227154941 shutdown_connections 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.548+0000 7f2920a54640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f29040b2930 0x7f29040b4e40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.548+0000 7f2920a54640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f291c10bdd0 0x7f291c1a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.549+0000 7f2920a54640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f291c1070d0 0x7f291c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.549+0000 7f2920a54640 1 --2- 172.21.15.39:0/2227154941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f291c105e90 0x7f291c19c420 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.549+0000 7f2920a54640 1 -- 172.21.15.39:0/2227154941 >> 172.21.15.39:0/2227154941 conn(0x7f291c101060 msgr2=0x7f291c107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.549+0000 7f2920a54640 1 -- 172.21.15.39:0/2227154941 shutdown_connections 2024-05-04T03:22:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:22:59.549+0000 7f2920a54640 1 -- 172.21.15.39:0/2227154941 wait complete. 2024-05-04T03:22:59.645 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:22:59 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3095571932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:59.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:22:59 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3095571932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:59.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:22:59 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3095571932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:00.013 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-05-04T03:23:00.014 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-05-04T03:23:00.438 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:23:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:00 smithi066 ceph-mon[33264]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-04T03:23:00.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:00 smithi039 ceph-mon[29715]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-04T03:23:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:00 smithi167 ceph-mon[33614]: pgmap v752: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-05-04T03:23:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 550.0 s, 778.9 tps, lat 114.046 ms stddev 131.598 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.305+0000 7ff029af7700 1 -- 172.21.15.39:0/3436169276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02410e8f0 msgr2=0x7ff024110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.305+0000 7ff029af7700 1 --2- 172.21.15.39:0/3436169276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02410e8f0 0x7ff024110dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00b0a0 tx=0x7ff01c00b3b0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.306+0000 7ff029af7700 1 -- 172.21.15.39:0/3436169276 shutdown_connections 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.306+0000 7ff029af7700 1 --2- 172.21.15.39:0/3436169276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff024105480 0x7ff024105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.306+0000 7ff029af7700 1 --2- 172.21.15.39:0/3436169276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02410e8f0 0x7ff024110dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.306+0000 7ff029af7700 1 --2- 172.21.15.39:0/3436169276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff024105dd0 0x7ff02410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.306+0000 7ff029af7700 1 -- 172.21.15.39:0/3436169276 >> 172.21.15.39:0/3436169276 conn(0x7ff0240fef30 msgr2=0x7ff024101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.307+0000 7ff029af7700 1 -- 172.21.15.39:0/3436169276 shutdown_connections 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.307+0000 7ff029af7700 1 -- 172.21.15.39:0/3436169276 wait complete. 2024-05-04T03:23:02.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.308+0000 7ff029af7700 1 Processor -- start 2024-05-04T03:23:02.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.308+0000 7ff029af7700 1 -- start start 2024-05-04T03:23:02.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.308+0000 7ff029af7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff024105480 0x7ff0241a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.308+0000 7ff029af7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff024105dd0 0x7ff0241a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.308+0000 7ff029af7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff02410e8f0 0x7ff02419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.308+0000 7ff029af7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff024114300 con 0x7ff024105dd0 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.308+0000 7ff029af7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff024114180 con 0x7ff02410e8f0 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.308+0000 7ff029af7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff024114480 con 0x7ff024105480 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff024105480 0x7ff0241a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff024105480 0x7ff0241a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34756/0 (socket says 172.21.15.39:34756) 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff0237fe700 1 -- 172.21.15.39:0/757204524 learned_addr learned my addr 172.21.15.39:0/757204524 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff022ffd700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff024105dd0 0x7ff0241a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff023fff700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff02410e8f0 0x7ff02419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff0237fe700 1 -- 172.21.15.39:0/757204524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff02410e8f0 msgr2=0x7ff02419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff0237fe700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff02410e8f0 0x7ff02419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff0237fe700 1 -- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff024105dd0 msgr2=0x7ff0241a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff0237fe700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff024105dd0 0x7ff0241a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff0237fe700 1 -- 172.21.15.39:0/757204524 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02419b100 con 0x7ff024105480 2024-05-04T03:23:02.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.309+0000 7ff022ffd700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff024105dd0 0x7ff0241a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:02.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.310+0000 7ff0237fe700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff024105480 0x7ff0241a08a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff01400d950 tx=0x7ff01400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:02.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.310+0000 7ff020ff9700 1 -- 172.21.15.39:0/757204524 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014016070 con 0x7ff024105480 2024-05-04T03:23:02.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.310+0000 7ff020ff9700 1 -- 172.21.15.39:0/757204524 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff014009dd0 con 0x7ff024105480 2024-05-04T03:23:02.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.310+0000 7ff029af7700 1 -- 172.21.15.39:0/757204524 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02419b390 con 0x7ff024105480 2024-05-04T03:23:02.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.310+0000 7ff029af7700 1 -- 172.21.15.39:0/757204524 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02419b8d0 con 0x7ff024105480 2024-05-04T03:23:02.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.310+0000 7ff020ff9700 1 -- 172.21.15.39:0/757204524 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01400f3c0 con 0x7ff024105480 2024-05-04T03:23:02.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.314+0000 7ff020ff9700 1 -- 172.21.15.39:0/757204524 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff014019070 con 0x7ff024105480 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.315+0000 7ff03850bb80 1 Processor -- start 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.315+0000 7ff020ff9700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff00c0b29d0 0x7ff00c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.315+0000 7ff03850bb80 1 -- start start 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6660 0x555bcb8a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8b1970 0x555bcb8b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff03850bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555bcb7c99f0 con 0x555bcb8b1970 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff03850bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555bcb768950 con 0x555bcb8a6fb0 2024-05-04T03:23:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff03850bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555bcb7c59b0 con 0x555bcb8a6660 2024-05-04T03:23:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff020ff9700 1 -- 172.21.15.39:0/757204524 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff0140e1cc0 con 0x7ff024105480 2024-05-04T03:23:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff022ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6660 0x555bcb8a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6660 0x555bcb8a6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34770/0 (socket says 172.21.15.39:34770) 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.316+0000 7ff0237fe700 1 -- 172.21.15.39:0/4024503020 learned_addr learned my addr 172.21.15.39:0/4024503020 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.317+0000 7ff023fff700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff00c0b29d0 0x7ff00c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.317+0000 7ff0237fe700 1 -- 172.21.15.39:0/4024503020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8a6fb0 msgr2=0x555bcb8b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.317+0000 7ff0237fe700 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.317+0000 7ff023fff700 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8b1970 0x555bcb8b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.317+0000 7ff0237fe700 1 -- 172.21.15.39:0/4024503020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8b1970 msgr2=0x555bcb8b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.317+0000 7ff0237fe700 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8b1970 0x555bcb8b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.317+0000 7ff0237fe700 1 -- 172.21.15.39:0/4024503020 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcb716a60 con 0x555bcb8a6660 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.317+0000 7ff023fff700 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8b1970 0x555bcb8b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:23:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.318+0000 7ff0237fe700 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6660 0x555bcb8a6a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0140a5310 tx=0x7ff0140a8e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89e117720a3ab8c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:02.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.318+0000 7ff009ffb700 1 -- 172.21.15.39:0/4024503020 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014020070 con 0x555bcb8a6660 2024-05-04T03:23:02.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.318+0000 7ff023fff700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff00c0b29d0 0x7ff00c0b4ee0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009900 tx=0x7ff01c020040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:02.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.318+0000 7ff009ffb700 1 -- 172.21.15.39:0/4024503020 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0140a9290 con 0x555bcb8a6660 2024-05-04T03:23:02.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.318+0000 7ff009ffb700 1 -- 172.21.15.39:0/4024503020 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014011290 con 0x555bcb8a6660 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.319+0000 7ff03850bb80 1 -- 172.21.15.39:0/4024503020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6660 msgr2=0x555bcb8a6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.319+0000 7ff03850bb80 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6660 0x555bcb8a6a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0140a5310 tx=0x7ff0140a8e20 comp rx=0 tx=0).stop 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.319+0000 7ff03850bb80 1 -- 172.21.15.39:0/4024503020 shutdown_connections 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.319+0000 7ff03850bb80 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8b1970 0x555bcb8b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.319+0000 7ff03850bb80 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6660 0x555bcb8a6a70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.320+0000 7ff03850bb80 1 --2- 172.21.15.39:0/4024503020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.320+0000 7ff03850bb80 1 -- 172.21.15.39:0/4024503020 >> 172.21.15.39:0/4024503020 conn(0x555bcb729fd0 msgr2=0x555bcb8a2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.320+0000 7ff03850bb80 1 -- 172.21.15.39:0/4024503020 shutdown_connections 2024-05-04T03:23:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.320+0000 7ff03850bb80 1 -- 172.21.15.39:0/4024503020 wait complete. 2024-05-04T03:23:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.320+0000 7ff03850bb80 1 -- start start 2024-05-04T03:23:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.321+0000 7ff03850bb80 1 Processor -- start 2024-05-04T03:23:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.321+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.321+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff03850bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555bcb8b46a0 con 0x555bcb8a6660 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff03850bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x555bcb72fdc0 con 0x555bcb8b1970 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff03850bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555bcb8b4820 con 0x555bcb8a6fb0 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff03850bb80 10 client.0 _ll_register_callbacks cb 0x555bcb761ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff022ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff022ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34774/0 (socket says 172.21.15.39:34774) 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff022ffd700 1 -- 172.21.15.39:0/3892844543 learned_addr learned my addr 172.21.15.39:0/3892844543 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff023fff700 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff022ffd700 1 -- 172.21.15.39:0/3892844543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 msgr2=0x555bcb8af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff022ffd700 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff022ffd700 1 -- 172.21.15.39:0/3892844543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 msgr2=0x555bcb8ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff022ffd700 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.322+0000 7ff022ffd700 1 -- 172.21.15.39:0/3892844543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcb765820 con 0x555bcb8a6fb0 2024-05-04T03:23:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.323+0000 7ff022ffd700 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff018011820 tx=0x7ff018011b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e74f151458665d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.323+0000 7ff002ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.323+0000 7ff002ffd700 1 -- 172.21.15.39:0/3892844543 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018004520 con 0x555bcb8a6fb0 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.323+0000 7ff002ffd700 1 -- 172.21.15.39:0/3892844543 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff018004830 con 0x555bcb8a6fb0 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.323+0000 7ff03850bb80 1 -- 172.21.15.39:0/3892844543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0140a9290 con 0x555bcb8a6fb0 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.323+0000 7ff002ffd700 1 -- 172.21.15.39:0/3892844543 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff018004a30 con 0x555bcb8a6fb0 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff03850bb80 10 client.26302 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff03850bb80 1 -- 172.21.15.39:0/3892844543 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555bcb8bf280 con 0x555bcb8a6fb0 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff002ffd700 1 -- 172.21.15.39:0/3892844543 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0140a9290 con 0x555bcb8a6fb0 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff002ffd700 1 -- 172.21.15.39:0/3892844543 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff018018070 con 0x555bcb8a6fb0 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff03850bb80 10 client.26302 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff03850bb80 10 client.26302 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff03850bb80 20 client.26302 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555bcb8c18b0 0x555bcb8c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff03850bb80 4 client.26302 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:02.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.324+0000 7ff03850bb80 1 -- 172.21.15.39:0/3892844543 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bcb7d73d0 con 0x555bcb8c18b0 2024-05-04T03:23:02.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.325+0000 7ff0237fe700 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555bcb8c18b0 0x555bcb8c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.326+0000 7ff0237fe700 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555bcb8c18b0 0x555bcb8c3d80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c010f80933d7a31a server_cookie=105ace9ab1f91a96 in_seq=0 out_seq=0 2024-05-04T03:23:02.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.326+0000 7ff002ffd700 10 client.26302 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.326+0000 7ff002ffd700 1 -- 172.21.15.39:0/3892844543 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555bcb7d73d0 con 0x555bcb8c18b0 2024-05-04T03:23:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.326+0000 7ff002ffd700 10 client.26302 handle_command_reply: tid=0 2024-05-04T03:23:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.326+0000 7ff03850bb80 1 client.26302 shutdown 2024-05-04T03:23:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 -- 172.21.15.39:0/3892844543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 msgr2=0x555bcb8aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff018011820 tx=0x7ff018011b30 comp rx=0 tx=0).stop 2024-05-04T03:23:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 -- 172.21.15.39:0/3892844543 shutdown_connections 2024-05-04T03:23:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555bcb8c18b0 0x555bcb8c3d80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3892844543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 -- 172.21.15.39:0/3892844543 >> 172.21.15.39:0/3892844543 conn(0x555bcb729fd0 msgr2=0x555bcb8a8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff002ffd700 0 client.26302 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 -- 172.21.15.39:0/3892844543 shutdown_connections 2024-05-04T03:23:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 1 -- 172.21.15.39:0/3892844543 wait complete. 2024-05-04T03:23:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.327+0000 7ff03850bb80 20 client.26302 trim_cache size 0 max 16384 2024-05-04T03:23:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff03850bb80 1 Processor -- start 2024-05-04T03:23:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff03850bb80 1 -- start start 2024-05-04T03:23:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff03850bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff01000d360 con 0x555bcb8a6660 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff03850bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff01000eea0 con 0x555bcb8b1970 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff03850bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff01000da30 con 0x555bcb8a6fb0 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52140/0 (socket says 172.21.15.39:52140) 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff0237fe700 1 -- 172.21.15.39:0/3126610658 learned_addr learned my addr 172.21.15.39:0/3126610658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff022ffd700 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.361+0000 7ff023fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff0237fe700 1 -- 172.21.15.39:0/3126610658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 msgr2=0x555bcb8aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff0237fe700 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff0237fe700 1 -- 172.21.15.39:0/3126610658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 msgr2=0x555bcb8c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff0237fe700 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff0237fe700 1 -- 172.21.15.39:0/3126610658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcb8ace30 con 0x555bcb8a6660 2024-05-04T03:23:02.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.362+0000 7ff023fff700 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:02.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.363+0000 7ff0237fe700 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8aa380 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff014000c00 tx=0x7ff0140a9190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6778b7032e8d223f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:02.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.363+0000 7ff0097fa700 1 -- 172.21.15.39:0/3126610658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014020040 con 0x555bcb8a6660 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.363+0000 7ff0097fa700 1 -- 172.21.15.39:0/3126610658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0140b20f0 con 0x555bcb8a6660 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.363+0000 7ff0097fa700 1 -- 172.21.15.39:0/3126610658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0140b1c60 con 0x555bcb8a6660 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.364+0000 7ff03850bb80 1 -- 172.21.15.39:0/3126610658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 msgr2=0x555bcb8aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.364+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8aa380 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff014000c00 tx=0x7ff0140a9190 comp rx=0 tx=0).stop 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.364+0000 7ff03850bb80 1 -- 172.21.15.39:0/3126610658 shutdown_connections 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.364+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8aa380 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.364+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.364+0000 7ff03850bb80 1 --2- 172.21.15.39:0/3126610658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.364+0000 7ff03850bb80 1 -- 172.21.15.39:0/3126610658 >> 172.21.15.39:0/3126610658 conn(0x555bcb729fd0 msgr2=0x555bcb8a19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.365+0000 7ff03850bb80 1 -- 172.21.15.39:0/3126610658 shutdown_connections 2024-05-04T03:23:02.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.365+0000 7ff03850bb80 1 -- 172.21.15.39:0/3126610658 wait complete. 2024-05-04T03:23:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.365+0000 7ff03850bb80 1 -- start start 2024-05-04T03:23:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.366+0000 7ff03850bb80 1 Processor -- start 2024-05-04T03:23:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.366+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.366+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.366+0000 7ff03850bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.366+0000 7ff03850bb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff01000e7d0 con 0x555bcb8a6660 2024-05-04T03:23:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.366+0000 7ff03850bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff01000e100 con 0x555bcb8b1970 2024-05-04T03:23:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.366+0000 7ff03850bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff01000b300 con 0x555bcb8a6fb0 2024-05-04T03:23:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.366+0000 7ff03850bb80 10 client.0 _ll_register_callbacks cb 0x555bcb761ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff022ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff022ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:34786/0 (socket says 172.21.15.39:34786) 2024-05-04T03:23:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff023fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff022ffd700 1 -- 172.21.15.39:0/2827372875 learned_addr learned my addr 172.21.15.39:0/2827372875 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff022ffd700 1 -- 172.21.15.39:0/2827372875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 msgr2=0x555bcb8b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff022ffd700 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff022ffd700 1 -- 172.21.15.39:0/2827372875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 msgr2=0x555bcb8b9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff022ffd700 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8b9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.367+0000 7ff022ffd700 1 -- 172.21.15.39:0/2827372875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcb8b6d40 con 0x555bcb8a6fb0 2024-05-04T03:23:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.368+0000 7ff022ffd700 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b6240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x555bcb8a64b0 tx=0x7ff018003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce538104df2c3e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.368+0000 7ff002ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:23:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.368+0000 7ff002ffd700 1 -- 172.21.15.39:0/2827372875 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018029070 con 0x555bcb8a6fb0 2024-05-04T03:23:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.369+0000 7ff002ffd700 1 -- 172.21.15.39:0/2827372875 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff018004520 con 0x555bcb8a6fb0 2024-05-04T03:23:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.369+0000 7ff03850bb80 1 -- 172.21.15.39:0/2827372875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bcb7d1470 con 0x555bcb8a6fb0 2024-05-04T03:23:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.369+0000 7ff002ffd700 1 -- 172.21.15.39:0/2827372875 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff018004740 con 0x555bcb8a6fb0 2024-05-04T03:23:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.369+0000 7ff03850bb80 10 client.26308 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.369+0000 7ff03850bb80 1 -- 172.21.15.39:0/2827372875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555bcb935640 con 0x555bcb8a6fb0 2024-05-04T03:23:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.370+0000 7ff002ffd700 1 -- 172.21.15.39:0/2827372875 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555bcb7d1470 con 0x555bcb8a6fb0 2024-05-04T03:23:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.370+0000 7ff002ffd700 1 -- 172.21.15.39:0/2827372875 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff018018040 con 0x555bcb8a6fb0 2024-05-04T03:23:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.370+0000 7ff03850bb80 10 client.26308 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.370+0000 7ff03850bb80 10 client.26308 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:23:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.370+0000 7ff03850bb80 20 client.26308 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.370+0000 7ff03850bb80 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555bcb935da0 0x555bcb9381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.370+0000 7ff03850bb80 4 client.26308 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-05-04T03:23:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.370+0000 7ff03850bb80 1 -- 172.21.15.39:0/2827372875 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x555bcb6493e0 con 0x555bcb935da0 2024-05-04T03:23:02.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.371+0000 7ff0237fe700 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555bcb935da0 0x555bcb9381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:02.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.371+0000 7ff0237fe700 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555bcb935da0 0x555bcb9381c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf29dd884e7c96d server_cookie=bcf1d83adf91810e in_seq=0 out_seq=0 2024-05-04T03:23:02.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.371+0000 7ff002ffd700 10 client.26308 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:02.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:02 smithi039 ceph-mon[29715]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 292 op/s 2024-05-04T03:23:02.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.790+0000 7ff002ffd700 1 -- 172.21.15.39:0/2827372875 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555bcb6493e0 con 0x555bcb935da0 2024-05-04T03:23:02.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.790+0000 7ff002ffd700 10 client.26308 handle_command_reply: tid=0 2024-05-04T03:23:02.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.790+0000 7ff03850bb80 1 client.26308 shutdown 2024-05-04T03:23:02.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.790+0000 7ff03850bb80 1 -- 172.21.15.39:0/2827372875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 msgr2=0x555bcb8b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.790+0000 7ff03850bb80 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b6240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x555bcb8a64b0 tx=0x7ff018003680 comp rx=0 tx=0).stop 2024-05-04T03:23:02.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 1 -- 172.21.15.39:0/2827372875 shutdown_connections 2024-05-04T03:23:02.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555bcb935da0 0x555bcb9381c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555bcb8a6660 0x555bcb8b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff002ffd700 0 client.26308 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555bcb8b1970 0x555bcb8b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 1 --2- 172.21.15.39:0/2827372875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555bcb8a6fb0 0x555bcb8b6240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 1 -- 172.21.15.39:0/2827372875 >> 172.21.15.39:0/2827372875 conn(0x555bcb729fd0 msgr2=0x555bcb8a23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 1 -- 172.21.15.39:0/2827372875 shutdown_connections 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 1 -- 172.21.15.39:0/2827372875 wait complete. 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff03850bb80 20 client.26308 trim_cache size 0 max 16384 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:23:02.795 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 34, 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "fcc00b97": { 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 34, 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 90.0, 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:23:02.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:23:02.797 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:23:02.797 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:02.797 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:02.797 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:02.797 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:02.797 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:02.797 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 -- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff00c0b29d0 msgr2=0x7ff00c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff00c0b29d0 0x7ff00c0b4ee0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009900 tx=0x7ff01c020040 comp rx=0 tx=0).stop 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 -- 172.21.15.39:0/757204524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff024105480 msgr2=0x7ff0241a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff024105480 0x7ff0241a08a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff01400d950 tx=0x7ff01400dc60 comp rx=0 tx=0).stop 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 -- 172.21.15.39:0/757204524 shutdown_connections 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff00c0b29d0 0x7ff00c0b4ee0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff02410e8f0 0x7ff02419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff024105480 0x7ff0241a08a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 --2- 172.21.15.39:0/757204524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff024105dd0 0x7ff0241a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.791+0000 7ff029af7700 1 -- 172.21.15.39:0/757204524 >> 172.21.15.39:0/757204524 conn(0x7ff0240fef30 msgr2=0x7ff0240ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.792+0000 7ff029af7700 1 -- 172.21.15.39:0/757204524 shutdown_connections 2024-05-04T03:23:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:02.792+0000 7ff029af7700 1 -- 172.21.15.39:0/757204524 wait complete. 2024-05-04T03:23:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:02 smithi066 ceph-mon[33264]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 292 op/s 2024-05-04T03:23:02.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:02 smithi167 ceph-mon[33614]: pgmap v753: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.2 MiB/s wr, 292 op/s 2024-05-04T03:23:03.676 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fcc00b97', quiesce took 7.2 seconds. Will release after: 30.0 2024-05-04T03:23:03.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:03 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:23:03.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:03 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:23:03.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:03 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:23:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:04 smithi066 ceph-mon[33264]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.5 MiB/s wr, 525 op/s 2024-05-04T03:23:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:23:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:23:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:23:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:04 smithi167 ceph-mon[33614]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.5 MiB/s wr, 525 op/s 2024-05-04T03:23:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:23:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:23:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:23:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:04 smithi039 ceph-mon[29715]: pgmap v754: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.5 MiB/s wr, 525 op/s 2024-05-04T03:23:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:23:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:23:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:23:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 555.0 s, 128.7 tps, lat 180.372 ms stddev 246.991 2024-05-04T03:23:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:06 smithi066 ceph-mon[33264]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 627 op/s 2024-05-04T03:23:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:06 smithi167 ceph-mon[33614]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 627 op/s 2024-05-04T03:23:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:06 smithi039 ceph-mon[29715]: pgmap v755: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 627 op/s 2024-05-04T03:23:08.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:08 smithi066 ceph-mon[33264]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-05-04T03:23:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:08 smithi167 ceph-mon[33614]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-05-04T03:23:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:08 smithi039 ceph-mon[29715]: pgmap v756: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-05-04T03:23:10.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:10 smithi066 ceph-mon[33264]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 696 op/s 2024-05-04T03:23:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:10 smithi167 ceph-mon[33614]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 696 op/s 2024-05-04T03:23:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:10 smithi039 ceph-mon[29715]: pgmap v757: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 696 op/s 2024-05-04T03:23:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 560.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 -- 172.21.15.39:0/3888337427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 msgr2=0x7f6a1c101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 --2- 172.21.15.39:0/3888337427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 0x7f6a1c101d40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009900 tx=0x7f6a0c02a410 comp rx=0 tx=0).stop 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 -- 172.21.15.39:0/3888337427 shutdown_connections 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 --2- 172.21.15.39:0/3888337427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a1c1023d0 0x7f6a1c10be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 --2- 172.21.15.39:0/3888337427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 0x7f6a1c101d40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 --2- 172.21.15.39:0/3888337427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a1c105220 0x7f6a1c105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 -- 172.21.15.39:0/3888337427 >> 172.21.15.39:0/3888337427 conn(0x7f6a1c0fd4f0 msgr2=0x7f6a1c0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 -- 172.21.15.39:0/3888337427 shutdown_connections 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.668+0000 7f6a245f3640 1 -- 172.21.15.39:0/3888337427 wait complete. 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a245f3640 1 Processor -- start 2024-05-04T03:23:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a245f3640 1 -- start start 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a245f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 0x7f6a1c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a245f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a1c1023d0 0x7f6a1c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a245f3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a1c105220 0x7f6a1c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a245f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c10f640 con 0x7f6a1c1018d0 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a245f3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c10f4c0 con 0x7f6a1c105220 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a245f3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c10f7c0 con 0x7f6a1c1023d0 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a2238f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 0x7f6a1c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a2238f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 0x7f6a1c19a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52352/0 (socket says 172.21.15.39:52352) 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a2238f640 1 -- 172.21.15.39:0/2666436682 learned_addr learned my addr 172.21.15.39:0/2666436682 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a21b8e640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a1c1023d0 0x7f6a1c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.669+0000 7f6a22b90640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a1c105220 0x7f6a1c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a2238f640 1 -- 172.21.15.39:0/2666436682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a1c1023d0 msgr2=0x7f6a1c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a1c1023d0 0x7f6a1c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a2238f640 1 -- 172.21.15.39:0/2666436682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a1c105220 msgr2=0x7f6a1c1a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a1c105220 0x7f6a1c1a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a2238f640 1 -- 172.21.15.39:0/2666436682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1c1a1de0 con 0x7f6a1c1018d0 2024-05-04T03:23:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 0x7f6a1c19a2a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800d740 tx=0x7f6a1800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a137fe640 1 -- 172.21.15.39:0/2666436682 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18017070 con 0x7f6a1c1018d0 2024-05-04T03:23:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a137fe640 1 -- 172.21.15.39:0/2666436682 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a1800de50 con 0x7f6a1c1018d0 2024-05-04T03:23:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a137fe640 1 -- 172.21.15.39:0/2666436682 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18003bb0 con 0x7f6a1c1018d0 2024-05-04T03:23:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a245f3640 1 -- 172.21.15.39:0/2666436682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1c1a20d0 con 0x7f6a1c1018d0 2024-05-04T03:23:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.670+0000 7f6a245f3640 1 -- 172.21.15.39:0/2666436682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1c1a2620 con 0x7f6a1c1018d0 2024-05-04T03:23:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.673+0000 7f6a2b57e740 1 Processor -- start 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.673+0000 7f6a2b57e740 1 -- start start 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.673+0000 7f6a137fe640 1 -- 172.21.15.39:0/2666436682 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6a18003d50 con 0x7f6a1c1018d0 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.673+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.673+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.673+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56331c12f5d0 con 0x56331be002f0 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56331c0e8890 con 0x56331be084d0 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56331c12b590 con 0x56331be291b0 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52356/0 (socket says 172.21.15.39:52356) 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 -- 172.21.15.39:0/3772392828 learned_addr learned my addr 172.21.15.39:0/3772392828 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a22b90640 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a21b8e640 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 -- 172.21.15.39:0/3772392828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 msgr2=0x56331c1caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 -- 172.21.15.39:0/3772392828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 msgr2=0x56331c1d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 -- 172.21.15.39:0/3772392828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331be41210 con 0x56331be002f0 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a137fe640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a080b27e0 0x7f6a080b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a2238f640 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1cafe0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6a180e5cd0 tx=0x7f6a180ea600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492aa09af6830f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.674+0000 7f6a22b90640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a080b27e0 0x7f6a080b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.675+0000 7f6a137fe640 1 -- 172.21.15.39:0/2666436682 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a180e1b30 con 0x7f6a1c1018d0 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.675+0000 7f6a10bfe640 1 -- 172.21.15.39:0/3772392828 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180a50d0 con 0x56331be002f0 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.675+0000 7f6a10bfe640 1 -- 172.21.15.39:0/3772392828 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a180a8d20 con 0x56331be002f0 2024-05-04T03:23:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.675+0000 7f6a10bfe640 1 -- 172.21.15.39:0/3772392828 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180a8980 con 0x56331be002f0 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 -- 172.21.15.39:0/3772392828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 msgr2=0x56331c1cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1cafe0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6a180e5cd0 tx=0x7f6a180ea600 comp rx=0 tx=0).stop 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a22b90640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a080b27e0 0x7f6a080b4cf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x56331c1cbc30 tx=0x7f6a14009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 -- 172.21.15.39:0/3772392828 shutdown_connections 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1cafe0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/3772392828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 -- 172.21.15.39:0/3772392828 >> 172.21.15.39:0/3772392828 conn(0x56331be1d300 msgr2=0x56331c1c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 -- 172.21.15.39:0/3772392828 shutdown_connections 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 -- 172.21.15.39:0/3772392828 wait complete. 2024-05-04T03:23:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 -- start start 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 Processor -- start 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be002f0 0x56331c1d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.676+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be291b0 0x56331c1dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56331c1d2930 con 0x56331be291b0 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x56331c0ec370 con 0x56331be084d0 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56331c1d2ab0 con 0x56331be002f0 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a2b57e740 10 client.0 _ll_register_callbacks cb 0x56331be40a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be291b0 0x56331c1dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be291b0 0x56331c1dbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52358/0 (socket says 172.21.15.39:52358) 2024-05-04T03:23:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 -- 172.21.15.39:0/1436018147 learned_addr learned my addr 172.21.15.39:0/1436018147 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a2238f640 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be002f0 0x56331c1d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a21b8e640 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 -- 172.21.15.39:0/1436018147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be002f0 msgr2=0x56331c1d4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be002f0 0x56331c1d4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 -- 172.21.15.39:0/1436018147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 msgr2=0x56331c1d4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1d4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 -- 172.21.15.39:0/1436018147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331bf0d620 con 0x56331be291b0 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a2238f640 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be002f0 0x56331c1d4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a22b90640 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be291b0 0x56331c1dbae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6a140042e0 tx=0x7f6a14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb28f6770ff8f091 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a05ffb640 1 -- 172.21.15.39:0/1436018147 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14025070 con 0x56331be291b0 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a05ffb640 1 -- 172.21.15.39:0/1436018147 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a14011d60 con 0x56331be291b0 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.677+0000 7f6a2b57e740 1 -- 172.21.15.39:0/1436018147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a180a8d20 con 0x56331be291b0 2024-05-04T03:23:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a05ffb640 1 -- 172.21.15.39:0/1436018147 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a14031050 con 0x56331be291b0 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2b57e740 10 client.16698 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2b57e740 1 -- 172.21.15.39:0/1436018147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56331c1deae0 con 0x56331be291b0 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a05ffb640 1 -- 172.21.15.39:0/1436018147 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a180a8d20 con 0x56331be291b0 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a05ffb640 1 -- 172.21.15.39:0/1436018147 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6a1400f040 con 0x56331be291b0 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2b57e740 10 client.16698 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2b57e740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2b57e740 20 client.16698 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56331bde7d80 0x56331c1e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2b57e740 4 client.16698 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2b57e740 1 -- 172.21.15.39:0/1436018147 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56331c13d1e0 con 0x56331bde7d80 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.678+0000 7f6a2238f640 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56331bde7d80 0x56331c1e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.679+0000 7f6a05ffb640 10 client.16698 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.679+0000 7f6a2238f640 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56331bde7d80 0x56331c1e21f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908f974543ecaa6b server_cookie=591db753f7685c4a in_seq=0 out_seq=0 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a05ffb640 1 -- 172.21.15.39:0/1436018147 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56331c13d1e0 con 0x56331bde7d80 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a05ffb640 10 client.16698 handle_command_reply: tid=0 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 client.16698 shutdown 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 -- 172.21.15.39:0/1436018147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be291b0 msgr2=0x56331c1dbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be291b0 0x56331c1dbae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6a140042e0 tx=0x7f6a14004a40 comp rx=0 tx=0).stop 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 -- 172.21.15.39:0/1436018147 shutdown_connections 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56331bde7d80 0x56331c1e21f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be291b0 0x56331c1dbae0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a05ffb640 0 client.16698 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/1436018147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be002f0 0x56331c1d4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 -- 172.21.15.39:0/1436018147 >> 172.21.15.39:0/1436018147 conn(0x56331be1d300 msgr2=0x56331c1c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 -- 172.21.15.39:0/1436018147 shutdown_connections 2024-05-04T03:23:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 1 -- 172.21.15.39:0/1436018147 wait complete. 2024-05-04T03:23:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.680+0000 7f6a2b57e740 20 client.16698 trim_cache size 0 max 16384 2024-05-04T03:23:12.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.695+0000 7f6a2b57e740 1 Processor -- start 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.695+0000 7f6a2b57e740 1 -- start start 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1dba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1ab340 con 0x56331be002f0 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1ace80 con 0x56331be084d0 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1aba10 con 0x56331be291b0 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52362/0 (socket says 172.21.15.39:52362) 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 -- 172.21.15.39:0/2283580590 learned_addr learned my addr 172.21.15.39:0/2283580590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a22b90640 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a21b8e640 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 -- 172.21.15.39:0/2283580590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 msgr2=0x56331c1dbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1dbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 -- 172.21.15.39:0/2283580590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 msgr2=0x56331c1dba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1dba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 -- 172.21.15.39:0/2283580590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331c1c6d00 con 0x56331be002f0 2024-05-04T03:23:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1c6470 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6a180df9b0 tx=0x7f6a180e5220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94b07c6f4b0705 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a07fff640 1 -- 172.21.15.39:0/2283580590 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18020040 con 0x56331be002f0 2024-05-04T03:23:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a07fff640 1 -- 172.21.15.39:0/2283580590 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a180a8d20 con 0x56331be002f0 2024-05-04T03:23:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.696+0000 7f6a07fff640 1 -- 172.21.15.39:0/2283580590 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180a8b60 con 0x56331be002f0 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2283580590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 msgr2=0x56331c1c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1c6470 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6a180df9b0 tx=0x7f6a180e5220 comp rx=0 tx=0).stop 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2283580590 shutdown_connections 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1dbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2283580590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1c6470 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2283580590 >> 172.21.15.39:0/2283580590 conn(0x56331be1d300 msgr2=0x56331c1c5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2283580590 shutdown_connections 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.697+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2283580590 wait complete. 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 1 -- start start 2024-05-04T03:23:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 1 Processor -- start 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1d9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1db3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1ac7b0 con 0x56331be002f0 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1ac0e0 con 0x56331be084d0 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1a9320 con 0x56331be291b0 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2b57e740 10 client.0 _ll_register_callbacks cb 0x7f6a14009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2238f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1d9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2238f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1d9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52376/0 (socket says 172.21.15.39:52376) 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2238f640 1 -- 172.21.15.39:0/2048835615 learned_addr learned my addr 172.21.15.39:0/2048835615 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a21b8e640 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a22b90640 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1db3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2238f640 1 -- 172.21.15.39:0/2048835615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 msgr2=0x56331c1db3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1db3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2238f640 1 -- 172.21.15.39:0/2048835615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 msgr2=0x56331c1da4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1da4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.698+0000 7f6a2238f640 1 -- 172.21.15.39:0/2048835615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331c1db970 con 0x56331be002f0 2024-05-04T03:23:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2238f640 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1d9f90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6a180e51f0 tx=0x7f6a180e1e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbeb78ff50ff8ce6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:23:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a05ffb640 1 -- 172.21.15.39:0/2048835615 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18020070 con 0x56331be002f0 2024-05-04T03:23:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a05ffb640 1 -- 172.21.15.39:0/2048835615 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6a180e5480 con 0x56331be002f0 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2048835615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a180a8d20 con 0x56331be002f0 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a05ffb640 1 -- 172.21.15.39:0/2048835615 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a180a8fe0 con 0x56331be002f0 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 10 client.16704 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2048835615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56331c24a9c0 con 0x56331be002f0 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a05ffb640 1 -- 172.21.15.39:0/2048835615 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a180a8d20 con 0x56331be002f0 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a05ffb640 1 -- 172.21.15.39:0/2048835615 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6a180a54a0 con 0x56331be002f0 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 10 client.16704 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 20 client.16704 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56331bdbd1c0 0x56331c24cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 4 client.16704 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.699+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2048835615 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56331c137050 con 0x56331bdbd1c0 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.700+0000 7f6a21b8e640 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56331bdbd1c0 0x56331c24cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.700+0000 7f6a05ffb640 10 client.16704 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.700+0000 7f6a21b8e640 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56331bdbd1c0 0x56331c24cef0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db442feb465f5f6 server_cookie=532e5a374391d677 in_seq=0 out_seq=0 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a05ffb640 1 -- 172.21.15.39:0/2048835615 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56331c137050 con 0x56331bdbd1c0 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a05ffb640 10 client.16704 handle_command_reply: tid=0 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 client.16704 shutdown 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2048835615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 msgr2=0x56331c1d9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1d9f90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6a180e51f0 tx=0x7f6a180e1e30 comp rx=0 tx=0).stop 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2048835615 shutdown_connections 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x56331bdbd1c0 0x56331c24cef0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56331be291b0 0x56331c1db3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56331be084d0 0x56331c1da4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 --2- 172.21.15.39:0/2048835615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56331be002f0 0x56331c1d9f90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a05ffb640 0 client.16704 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2048835615 >> 172.21.15.39:0/2048835615 conn(0x56331be1d300 msgr2=0x56331c1c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2048835615 shutdown_connections 2024-05-04T03:23:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 1 -- 172.21.15.39:0/2048835615 wait complete. 2024-05-04T03:23:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.709+0000 7f6a2b57e740 20 client.16704 trim_cache size 0 max 16384 2024-05-04T03:23:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:23:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:23:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:23:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:23:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 -- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a080b27e0 msgr2=0x7f6a080b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a080b27e0 0x7f6a080b4cf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x56331c1cbc30 tx=0x7f6a14009600 comp rx=0 tx=0).stop 2024-05-04T03:23:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 -- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 msgr2=0x7f6a1c19a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 0x7f6a1c19a2a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800d740 tx=0x7f6a1800dc00 comp rx=0 tx=0).stop 2024-05-04T03:23:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 -- 172.21.15.39:0/2666436682 shutdown_connections 2024-05-04T03:23:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6a080b27e0 0x7f6a080b4cf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a1c105220 0x7f6a1c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a1c1023d0 0x7f6a1c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 --2- 172.21.15.39:0/2666436682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a1c1018d0 0x7f6a1c19a2a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 -- 172.21.15.39:0/2666436682 >> 172.21.15.39:0/2666436682 conn(0x7f6a1c0fd4f0 msgr2=0x7f6a1c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 -- 172.21.15.39:0/2666436682 shutdown_connections 2024-05-04T03:23:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:12.710+0000 7f6a245f3640 1 -- 172.21.15.39:0/2666436682 wait complete. 2024-05-04T03:23:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:12 smithi066 ceph-mon[33264]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 619 op/s 2024-05-04T03:23:12.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:12 smithi167 ceph-mon[33614]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 619 op/s 2024-05-04T03:23:12.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:12 smithi039 ceph-mon[29715]: pgmap v758: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 619 op/s 2024-05-04T03:23:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:14 smithi066 ceph-mon[33264]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 450 op/s 2024-05-04T03:23:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:14 smithi167 ceph-mon[33614]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 450 op/s 2024-05-04T03:23:14.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:14 smithi039 ceph-mon[29715]: pgmap v759: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 450 op/s 2024-05-04T03:23:16.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 565.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:23:16.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:16 smithi167 ceph-mon[33614]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.3 MiB/s wr, 209 op/s 2024-05-04T03:23:16.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:16 smithi039 ceph-mon[29715]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.3 MiB/s wr, 209 op/s 2024-05-04T03:23:17.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:16 smithi066 ceph-mon[33264]: pgmap v760: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.3 MiB/s wr, 209 op/s 2024-05-04T03:23:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:18 smithi167 ceph-mon[33614]: pgmap v761: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 113 op/s 2024-05-04T03:23:18.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:18 smithi039 ceph-mon[29715]: pgmap v761: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 113 op/s 2024-05-04T03:23:19.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:18 smithi066 ceph-mon[33264]: pgmap v761: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.8 MiB/s wr, 113 op/s 2024-05-04T03:23:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:20 smithi167 ceph-mon[33614]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 217 KiB/s wr, 32 op/s 2024-05-04T03:23:20.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:20 smithi039 ceph-mon[29715]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 217 KiB/s wr, 32 op/s 2024-05-04T03:23:21.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:20 smithi066 ceph-mon[33264]: pgmap v762: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 217 KiB/s wr, 32 op/s 2024-05-04T03:23:21.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 570.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:23:22.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:21 smithi066 ceph-mon[33264]: pgmap v763: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 243 KiB/s wr, 34 op/s 2024-05-04T03:23:22.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:21 smithi167 ceph-mon[33614]: pgmap v763: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 243 KiB/s wr, 34 op/s 2024-05-04T03:23:22.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:21 smithi039 ceph-mon[29715]: pgmap v763: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 243 KiB/s wr, 34 op/s 2024-05-04T03:23:24.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:24 smithi167 ceph-mon[33614]: pgmap v764: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 291 KiB/s wr, 39 op/s 2024-05-04T03:23:24.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:24 smithi039 ceph-mon[29715]: pgmap v764: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 291 KiB/s wr, 39 op/s 2024-05-04T03:23:24.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:24 smithi066 ceph-mon[33264]: pgmap v764: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 291 KiB/s wr, 39 op/s 2024-05-04T03:23:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 -- 172.21.15.39:0/2486726682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 msgr2=0x7fd54810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 --2- 172.21.15.39:0/2486726682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 0x7fd54810aea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd540009ca0 tx=0x7fd54002cad0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 -- 172.21.15.39:0/2486726682 shutdown_connections 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 --2- 172.21.15.39:0/2486726682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 0x7fd54810aea0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 --2- 172.21.15.39:0/2486726682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd548103d80 0x7fd548104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 --2- 172.21.15.39:0/2486726682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd548102b40 0x7fd548103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 -- 172.21.15.39:0/2486726682 >> 172.21.15.39:0/2486726682 conn(0x7fd5480fdd30 msgr2=0x7fd5481001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 -- 172.21.15.39:0/2486726682 shutdown_connections 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 -- 172.21.15.39:0/2486726682 wait complete. 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.822+0000 7fd54de49640 1 Processor -- start 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd54de49640 1 -- start start 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd54de49640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd548102b40 0x7fd548198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd54de49640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd548103d80 0x7fd548198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd54de49640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 0x7fd54819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd54de49640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd54810d410 con 0x7fd548108a80 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd54de49640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd54810d290 con 0x7fd548103d80 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd54de49640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd54810d590 con 0x7fd548102b40 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd547fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 0x7fd54819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd547fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 0x7fd54819f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37776/0 (socket says 172.21.15.39:37776) 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd547fff640 1 -- 172.21.15.39:0/1543511084 learned_addr learned my addr 172.21.15.39:0/1543511084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd5477fe640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd548102b40 0x7fd548198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd546ffd640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd548103d80 0x7fd548198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd547fff640 1 -- 172.21.15.39:0/1543511084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd548102b40 msgr2=0x7fd548198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd547fff640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd548102b40 0x7fd548198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd547fff640 1 -- 172.21.15.39:0/1543511084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd548103d80 msgr2=0x7fd548198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd547fff640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd548103d80 0x7fd548198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd547fff640 1 -- 172.21.15.39:0/1543511084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54819fb80 con 0x7fd548108a80 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.823+0000 7fd546ffd640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd548103d80 0x7fd548198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.824+0000 7fd547fff640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 0x7fd54819f510 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd540009dc0 tx=0x7fd540038b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.824+0000 7fd544ff9640 1 -- 172.21.15.39:0/1543511084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540046070 con 0x7fd548108a80 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.824+0000 7fd544ff9640 1 -- 172.21.15.39:0/1543511084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5400046e0 con 0x7fd548108a80 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.824+0000 7fd544ff9640 1 -- 172.21.15.39:0/1543511084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54002ec70 con 0x7fd548108a80 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.824+0000 7fd54de49640 1 -- 172.21.15.39:0/1543511084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54819fe10 con 0x7fd548108a80 2024-05-04T03:23:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.824+0000 7fd54de49640 1 -- 172.21.15.39:0/1543511084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5481a0270 con 0x7fd548108a80 2024-05-04T03:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.826+0000 7fd554dd6740 1 Processor -- start 2024-05-04T03:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd544ff9640 1 -- 172.21.15.39:0/1543511084 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd540004250 con 0x7fd548108a80 2024-05-04T03:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd554dd6740 1 -- start start 2024-05-04T03:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fcba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fcbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd554dd6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6b8f305d0 con 0x55b6b8c012f0 2024-05-04T03:23:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd554dd6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b6b8ee9890 con 0x55b6b8c2a1b0 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd554dd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6b8f2c590 con 0x55b6b8c094d0 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fcbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fcbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37782/0 (socket says 172.21.15.39:37782) 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd5477fe640 1 -- 172.21.15.39:0/3927164131 learned_addr learned my addr 172.21.15.39:0/3927164131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd546ffd640 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fcba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd546ffd640 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd5477fe640 1 -- 172.21.15.39:0/3927164131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 msgr2=0x55b6b8fd2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd5477fe640 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd5477fe640 1 -- 172.21.15.39:0/3927164131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c2a1b0 msgr2=0x55b6b8fcba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd5477fe640 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fcba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd5477fe640 1 -- 172.21.15.39:0/3927164131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b8c42210 con 0x55b6b8c012f0 2024-05-04T03:23:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.827+0000 7fd544ff9640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5300b2860 0x7fd5300b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.828+0000 7fd5477fe640 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fcbfe0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd538009c60 tx=0x7fd538011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a91c9d7c8704a88 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.828+0000 7fd546ffd640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5300b2860 0x7fd5300b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.828+0000 7fd544ff9640 1 -- 172.21.15.39:0/1543511084 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd540104520 con 0x7fd548108a80 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.828+0000 7fd52dffb640 1 -- 172.21.15.39:0/3927164131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538004220 con 0x55b6b8c012f0 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.828+0000 7fd52dffb640 1 -- 172.21.15.39:0/3927164131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5380043c0 con 0x55b6b8c012f0 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.828+0000 7fd52dffb640 1 -- 172.21.15.39:0/3927164131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538004ca0 con 0x55b6b8c012f0 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 -- 172.21.15.39:0/3927164131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 msgr2=0x55b6b8fcbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fcbfe0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd538009c60 tx=0x7fd538011810 comp rx=0 tx=0).stop 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd546ffd640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5300b2860 0x7fd5300b4d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55b6b8fccc30 tx=0x7fd53c004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 -- 172.21.15.39:0/3927164131 shutdown_connections 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fcbfe0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3927164131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fcba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 -- 172.21.15.39:0/3927164131 >> 172.21.15.39:0/3927164131 conn(0x55b6b8c1e300 msgr2=0x55b6b8fc73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 -- 172.21.15.39:0/3927164131 shutdown_connections 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 -- 172.21.15.39:0/3927164131 wait complete. 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 -- start start 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.829+0000 7fd554dd6740 1 Processor -- start 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fe0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd554dd6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6b8fd3930 con 0x55b6b8c094d0 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd554dd6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b6b8eed370 con 0x55b6b8c012f0 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd554dd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6b8fd3ab0 con 0x55b6b8c2a1b0 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd554dd6740 10 client.0 _ll_register_callbacks cb 0x55b6b8c41a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37784/0 (socket says 172.21.15.39:37784) 2024-05-04T03:23:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 -- 172.21.15.39:0/3209653528 learned_addr learned my addr 172.21.15.39:0/3209653528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd5477fe640 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd547fff640 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fe0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 -- 172.21.15.39:0/3209653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 msgr2=0x55b6b8fe0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fe0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 -- 172.21.15.39:0/3209653528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 msgr2=0x55b6b8fd9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 -- 172.21.15.39:0/3209653528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b8d0e620 con 0x55b6b8c094d0 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd547fff640 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fe0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd546ffd640 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd9f60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd53c004650 tx=0x7fd53c00b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5295b66c7bd8bea server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.830+0000 7fd526ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd526ffd640 1 -- 172.21.15.39:0/3209653528 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c02a060 con 0x55b6b8c094d0 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd526ffd640 1 -- 172.21.15.39:0/3209653528 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd53c01d790 con 0x55b6b8c094d0 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 1 -- 172.21.15.39:0/3209653528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5380043c0 con 0x55b6b8c094d0 2024-05-04T03:23:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd526ffd640 1 -- 172.21.15.39:0/3209653528 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd53c01d990 con 0x55b6b8c094d0 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 10 client.16716 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 1 -- 172.21.15.39:0/3209653528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6b8fe3ec0 con 0x55b6b8c094d0 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd526ffd640 1 -- 172.21.15.39:0/3209653528 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5380043c0 con 0x55b6b8c094d0 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd526ffd640 1 -- 172.21.15.39:0/3209653528 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd53c019040 con 0x55b6b8c094d0 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 10 client.16716 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 20 client.16716 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b6b8be8d80 0x55b6b8fe7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 4 client.16716 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.831+0000 7fd554dd6740 1 -- 172.21.15.39:0/3209653528 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6b8f3e1e0 con 0x55b6b8be8d80 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.832+0000 7fd5477fe640 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b6b8be8d80 0x55b6b8fe7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.832+0000 7fd526ffd640 10 client.16716 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.832+0000 7fd5477fe640 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b6b8be8d80 0x55b6b8fe7670 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40707c623065dae server_cookie=b4ddbcfbbfd05b8f in_seq=0 out_seq=0 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd526ffd640 1 -- 172.21.15.39:0/3209653528 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b6b8f3e1e0 con 0x55b6b8be8d80 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd526ffd640 10 client.16716 handle_command_reply: tid=0 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 client.16716 shutdown 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 -- 172.21.15.39:0/3209653528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c094d0 msgr2=0x55b6b8fd9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd9f60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd53c004650 tx=0x7fd53c00b4c0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 -- 172.21.15.39:0/3209653528 shutdown_connections 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b6b8be8d80 0x55b6b8fe7670 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fe0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd9f60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 --2- 172.21.15.39:0/3209653528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd526ffd640 0 client.16716 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 -- 172.21.15.39:0/3209653528 >> 172.21.15.39:0/3209653528 conn(0x55b6b8c1e300 msgr2=0x55b6b8fc5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 -- 172.21.15.39:0/3209653528 shutdown_connections 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 1 -- 172.21.15.39:0/3209653528 wait complete. 2024-05-04T03:23:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.833+0000 7fd554dd6740 20 client.16716 trim_cache size 0 max 16384 2024-05-04T03:23:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd554dd6740 1 Processor -- start 2024-05-04T03:23:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd554dd6740 1 -- start start 2024-05-04T03:23:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd554dd6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481a8ff0 con 0x55b6b8c2a1b0 2024-05-04T03:23:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd554dd6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481aab30 con 0x55b6b8c012f0 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd554dd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481a96c0 con 0x55b6b8c094d0 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd547fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd547fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fece70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37790/0 (socket says 172.21.15.39:37790) 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd547fff640 1 -- 172.21.15.39:0/1771212891 learned_addr learned my addr 172.21.15.39:0/1771212891 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd546ffd640 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd5477fe640 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd547fff640 1 -- 172.21.15.39:0/1771212891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 msgr2=0x55b6b8fd8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.849+0000 7fd547fff640 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.850+0000 7fd547fff640 1 -- 172.21.15.39:0/1771212891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 msgr2=0x55b6b8fd8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.850+0000 7fd547fff640 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.850+0000 7fd547fff640 1 -- 172.21.15.39:0/1771212891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b8fd3290 con 0x55b6b8c2a1b0 2024-05-04T03:23:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.850+0000 7fd5477fe640 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.850+0000 7fd547fff640 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fece70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd5400c8dc0 tx=0x7fd5400cb9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21208e37bc3027d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.850+0000 7fd52d7fa640 1 -- 172.21.15.39:0/1771212891 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400c72e0 con 0x55b6b8c2a1b0 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.850+0000 7fd52d7fa640 1 -- 172.21.15.39:0/1771212891 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd540038400 con 0x55b6b8c2a1b0 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.850+0000 7fd52d7fa640 1 -- 172.21.15.39:0/1771212891 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400cbb50 con 0x55b6b8c2a1b0 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 -- 172.21.15.39:0/1771212891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c2a1b0 msgr2=0x55b6b8fece70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fece70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd5400c8dc0 tx=0x7fd5400cb9e0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 -- 172.21.15.39:0/1771212891 shutdown_connections 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c2a1b0 0x55b6b8fece70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c094d0 0x55b6b8fd8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1771212891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c012f0 0x55b6b8fd8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 -- 172.21.15.39:0/1771212891 >> 172.21.15.39:0/1771212891 conn(0x55b6b8c1e300 msgr2=0x55b6b8fd48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 -- 172.21.15.39:0/1771212891 shutdown_connections 2024-05-04T03:23:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 -- 172.21.15.39:0/1771212891 wait complete. 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 -- start start 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.851+0000 7fd554dd6740 1 Processor -- start 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fdfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c094d0 0x55b6b8fe00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd554dd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b904df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd554dd6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481aa460 con 0x55b6b8c012f0 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd554dd6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481a9d90 con 0x55b6b8c094d0 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd554dd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5481a6fd0 con 0x55b6b8c2a1b0 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd554dd6740 10 client.0 _ll_register_callbacks cb 0x7fd53c004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fdfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fdfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37798/0 (socket says 172.21.15.39:37798) 2024-05-04T03:23:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 -- 172.21.15.39:0/1291876263 learned_addr learned my addr 172.21.15.39:0/1291876263 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd547fff640 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b904df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 -- 172.21.15.39:0/1291876263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 msgr2=0x55b6b904df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd546ffd640 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c094d0 0x55b6b8fe00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b904df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 -- 172.21.15.39:0/1291876263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c094d0 msgr2=0x55b6b8fe00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c094d0 0x55b6b8fe00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 -- 172.21.15.39:0/1291876263 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b904e510 con 0x55b6b8c012f0 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd5477fe640 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fdfb70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd53800a010 tx=0x7fd538005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854fd1de617cedf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd526ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.852+0000 7fd526ffd640 1 -- 172.21.15.39:0/1291876263 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53801d040 con 0x55b6b8c012f0 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 1 -- 172.21.15.39:0/1291876263 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd540038400 con 0x55b6b8c012f0 2024-05-04T03:23:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd526ffd640 1 -- 172.21.15.39:0/1291876263 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd538005360 con 0x55b6b8c012f0 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd526ffd640 1 -- 172.21.15.39:0/1291876263 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd53803b050 con 0x55b6b8c012f0 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 10 client.16722 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 1 -- 172.21.15.39:0/1291876263 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6b904f810 con 0x55b6b8c012f0 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd526ffd640 1 -- 172.21.15.39:0/1291876263 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd540038400 con 0x55b6b8c012f0 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd526ffd640 1 -- 172.21.15.39:0/1291876263 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd53802cce0 con 0x55b6b8c012f0 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 10 client.16722 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 20 client.16722 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b6b8bbe1c0 0x55b6b90520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 4 client.16722 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.853+0000 7fd554dd6740 1 -- 172.21.15.39:0/1291876263 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6b8f38050 con 0x55b6b8bbe1c0 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.854+0000 7fd546ffd640 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b6b8bbe1c0 0x55b6b90520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.854+0000 7fd546ffd640 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b6b8bbe1c0 0x55b6b90520c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bff4147964cb6d server_cookie=6e9b0bfe4ff66cc4 in_seq=0 out_seq=0 2024-05-04T03:23:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.854+0000 7fd526ffd640 10 client.16722 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.862+0000 7fd526ffd640 1 -- 172.21.15.39:0/1291876263 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6b8f38050 con 0x55b6b8bbe1c0 2024-05-04T03:23:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.862+0000 7fd526ffd640 10 client.16722 handle_command_reply: tid=0 2024-05-04T03:23:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 client.16722 shutdown 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 -- 172.21.15.39:0/1291876263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 msgr2=0x55b6b8fdfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fdfb70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd53800a010 tx=0x7fd538005db0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 -- 172.21.15.39:0/1291876263 shutdown_connections 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b6b8bbe1c0 0x55b6b90520c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b6b8c2a1b0 0x55b6b904df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b6b8c094d0 0x55b6b8fe00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd526ffd640 0 client.16722 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 --2- 172.21.15.39:0/1291876263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6b8c012f0 0x55b6b8fdfb70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 -- 172.21.15.39:0/1291876263 >> 172.21.15.39:0/1291876263 conn(0x55b6b8c1e300 msgr2=0x55b6b8fd7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 -- 172.21.15.39:0/1291876263 shutdown_connections 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 1 -- 172.21.15.39:0/1291876263 wait complete. 2024-05-04T03:23:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd554dd6740 20 client.16722 trim_cache size 0 max 16384 2024-05-04T03:23:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:23:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:23:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:23:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd54de49640 1 -- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5300b2860 msgr2=0x7fd5300b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.863+0000 7fd54de49640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5300b2860 0x7fd5300b4d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55b6b8fccc30 tx=0x7fd53c004730 comp rx=0 tx=0).stop 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 -- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 msgr2=0x7fd54819f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 0x7fd54819f510 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd540009dc0 tx=0x7fd540038b50 comp rx=0 tx=0).stop 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 -- 172.21.15.39:0/1543511084 shutdown_connections 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5300b2860 0x7fd5300b4d70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd548108a80 0x7fd54819f510 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd548103d80 0x7fd548198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 --2- 172.21.15.39:0/1543511084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd548102b40 0x7fd548198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 -- 172.21.15.39:0/1543511084 >> 172.21.15.39:0/1543511084 conn(0x7fd5480fdd30 msgr2=0x7fd548104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 -- 172.21.15.39:0/1543511084 shutdown_connections 2024-05-04T03:23:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:25.864+0000 7fd54de49640 1 -- 172.21.15.39:0/1543511084 wait complete. 2024-05-04T03:23:26.620 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:26 smithi066 ceph-mon[33264]: pgmap v765: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 250 KiB/s wr, 35 op/s 2024-05-04T03:23:26.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:26 smithi167 ceph-mon[33614]: pgmap v765: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 250 KiB/s wr, 35 op/s 2024-05-04T03:23:26.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:26 smithi039 ceph-mon[29715]: pgmap v765: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 250 KiB/s wr, 35 op/s 2024-05-04T03:23:26.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 575.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:23:28.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:28 smithi167 ceph-mon[33614]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 287 KiB/s wr, 42 op/s 2024-05-04T03:23:28.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:28 smithi039 ceph-mon[29715]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 287 KiB/s wr, 42 op/s 2024-05-04T03:23:28.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:28 smithi066 ceph-mon[33264]: pgmap v766: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 287 KiB/s wr, 42 op/s 2024-05-04T03:23:30.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:30 smithi167 ceph-mon[33614]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 268 KiB/s wr, 41 op/s 2024-05-04T03:23:30.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:30 smithi039 ceph-mon[29715]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 268 KiB/s wr, 41 op/s 2024-05-04T03:23:30.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:30 smithi066 ceph-mon[33264]: pgmap v767: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 268 KiB/s wr, 41 op/s 2024-05-04T03:23:31.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 580.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:23:32.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:32 smithi167 ceph-mon[33614]: pgmap v768: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 304 KiB/s wr, 44 op/s 2024-05-04T03:23:32.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:32 smithi039 ceph-mon[29715]: pgmap v768: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 304 KiB/s wr, 44 op/s 2024-05-04T03:23:32.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:32 smithi066 ceph-mon[33264]: pgmap v768: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 304 KiB/s wr, 44 op/s 2024-05-04T03:23:33.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:23:33.944 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:23:34.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:34 smithi167 ceph-mon[33614]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-05-04T03:23:34.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:34 smithi039 ceph-mon[29715]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-05-04T03:23:34.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:34 smithi066 ceph-mon[33264]: pgmap v769: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-05-04T03:23:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.868+0000 7fd1af381700 1 -- 172.21.15.39:0/1092929773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a81034c0 msgr2=0x7fd1a81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.868+0000 7fd1af381700 1 --2- 172.21.15.39:0/1092929773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a81034c0 0x7fd1a81038d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000b0d0 tx=0x7fd1a000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.869+0000 7fd1af381700 1 -- 172.21.15.39:0/1092929773 shutdown_connections 2024-05-04T03:23:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.869+0000 7fd1af381700 1 --2- 172.21.15.39:0/1092929773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a81034c0 0x7fd1a81038d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.869+0000 7fd1af381700 1 --2- 172.21.15.39:0/1092929773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a8108740 0x7fd1a81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.869+0000 7fd1af381700 1 --2- 172.21.15.39:0/1092929773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1a8107df0 0x7fd1a8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.869+0000 7fd1af381700 1 -- 172.21.15.39:0/1092929773 >> 172.21.15.39:0/1092929773 conn(0x7fd1a80fedc0 msgr2=0x7fd1a81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:35.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.869+0000 7fd1af381700 1 -- 172.21.15.39:0/1092929773 shutdown_connections 2024-05-04T03:23:35.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.869+0000 7fd1af381700 1 -- 172.21.15.39:0/1092929773 wait complete. 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1af381700 1 Processor -- start 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1af381700 1 -- start start 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1af381700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a81034c0 0x7fd1a819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1af381700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a8107df0 0x7fd1a819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1af381700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1a8108740 0x7fd1a81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1af381700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8113220 con 0x7fd1a8107df0 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1af381700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a81130a0 con 0x7fd1a81034c0 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1af381700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a81133a0 con 0x7fd1a8108740 2024-05-04T03:23:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ac91c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a8107df0 0x7fd1a819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ad11d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a81034c0 0x7fd1a819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ac91c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a8107df0 0x7fd1a819d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47338/0 (socket says 172.21.15.39:47338) 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ac91c700 1 -- 172.21.15.39:0/3716387098 learned_addr learned my addr 172.21.15.39:0/3716387098 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ad91e700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1a8108740 0x7fd1a81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ac91c700 1 -- 172.21.15.39:0/3716387098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1a8108740 msgr2=0x7fd1a81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ac91c700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1a8108740 0x7fd1a81a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ac91c700 1 -- 172.21.15.39:0/3716387098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a81034c0 msgr2=0x7fd1a819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ac91c700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a81034c0 0x7fd1a819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.870+0000 7fd1ac91c700 1 -- 172.21.15.39:0/3716387098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a81a1d30 con 0x7fd1a8107df0 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.871+0000 7fd1ac91c700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a8107df0 0x7fd1a819d4b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00c9e0 tx=0x7fd19c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.871+0000 7fd1a67fc700 1 -- 172.21.15.39:0/3716387098 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c007aa0 con 0x7fd1a8107df0 2024-05-04T03:23:35.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.871+0000 7fd1a67fc700 1 -- 172.21.15.39:0/3716387098 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd19c007c40 con 0x7fd1a8107df0 2024-05-04T03:23:35.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.871+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a81a2020 con 0x7fd1a8107df0 2024-05-04T03:23:35.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.871+0000 7fd1a67fc700 1 -- 172.21.15.39:0/3716387098 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c018400 con 0x7fd1a8107df0 2024-05-04T03:23:35.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.871+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a81a99f0 con 0x7fd1a8107df0 2024-05-04T03:23:35.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.872+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a80512c0 con 0x7fd1a8107df0 2024-05-04T03:23:35.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.874+0000 7fd1a67fc700 1 -- 172.21.15.39:0/3716387098 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd19c00f450 con 0x7fd1a8107df0 2024-05-04T03:23:35.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.874+0000 7fd1a67fc700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1900b2900 0x7fd1900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.874+0000 7fd1ad11d700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1900b2900 0x7fd1900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.875+0000 7fd1a67fc700 1 -- 172.21.15.39:0/3716387098 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd19c0e22b0 con 0x7fd1a8107df0 2024-05-04T03:23:35.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.875+0000 7fd1ad11d700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1900b2900 0x7fd1900b4e10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009fd0 tx=0x7fd1a000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:35.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:35.878+0000 7fd1a67fc700 1 -- 172.21.15.39:0/3716387098 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd19c014030 con 0x7fd1a8107df0 2024-05-04T03:23:36.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.141+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1a802e5f0 con 0x7fd1a8107df0 2024-05-04T03:23:36.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.142+0000 7fd1a67fc700 1 -- 172.21.15.39:0/3716387098 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7fd19c0aa0f0 con 0x7fd1a8107df0 2024-05-04T03:23:36.144 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:23:36.145 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:23:36.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1900b2900 msgr2=0x7fd1900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1900b2900 0x7fd1900b4e10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009fd0 tx=0x7fd1a000b040 comp rx=0 tx=0).stop 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a8107df0 msgr2=0x7fd1a819d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a8107df0 0x7fd1a819d4b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00c9e0 tx=0x7fd19c00ccf0 comp rx=0 tx=0).stop 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 shutdown_connections 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd1900b2900 0x7fd1900b4e10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1a8108740 0x7fd1a81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1a81034c0 0x7fd1a819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 --2- 172.21.15.39:0/3716387098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1a8107df0 0x7fd1a819d4b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 >> 172.21.15.39:0/3716387098 conn(0x7fd1a80fedc0 msgr2=0x7fd1a81064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 shutdown_connections 2024-05-04T03:23:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:36.144+0000 7fd1af381700 1 -- 172.21.15.39:0/3716387098 wait complete. 2024-05-04T03:23:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:23:36.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:36 smithi039 ceph-mon[29715]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 248 KiB/s wr, 36 op/s 2024-05-04T03:23:36.442 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:36 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3716387098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:36.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:36 smithi167 ceph-mon[33614]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 248 KiB/s wr, 36 op/s 2024-05-04T03:23:36.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:36 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3716387098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 585.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:23:36.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:36 smithi066 ceph-mon[33264]: pgmap v770: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 248 KiB/s wr, 36 op/s 2024-05-04T03:23:36.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:36 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3716387098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:36.865 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id fcc00b97 --release --await' 2024-05-04T03:23:36.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id fcc00b97 --release --await 2024-05-04T03:23:37.124 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:23:38.610 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:38 smithi039 ceph-mon[29715]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 282 KiB/s wr, 38 op/s 2024-05-04T03:23:38.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:38 smithi167 ceph-mon[33614]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 282 KiB/s wr, 38 op/s 2024-05-04T03:23:38.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:38 smithi066 ceph-mon[33264]: pgmap v771: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 282 KiB/s wr, 38 op/s 2024-05-04T03:23:38.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 -- 172.21.15.39:0/2291157395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 msgr2=0x7fbfc4108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2291157395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 0x7fbfc4108210 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b0d0 tx=0x7fbfc000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 -- 172.21.15.39:0/2291157395 shutdown_connections 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2291157395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc4108750 0x7fbfc41100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2291157395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfc41034d0 0x7fbfc41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2291157395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 0x7fbfc4108210 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 -- 172.21.15.39:0/2291157395 >> 172.21.15.39:0/2291157395 conn(0x7fbfc40fef20 msgr2=0x7fbfc4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 -- 172.21.15.39:0/2291157395 shutdown_connections 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.871+0000 7fbfcb428700 1 -- 172.21.15.39:0/2291157395 wait complete. 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfcb428700 1 Processor -- start 2024-05-04T03:23:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfcb428700 1 -- start start 2024-05-04T03:23:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfcb428700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc41034d0 0x7fbfc419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfcb428700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 0x7fbfc419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfcb428700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfc4108750 0x7fbfc419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfcb428700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4113280 con 0x7fbfc4107e00 2024-05-04T03:23:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfcb428700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4113100 con 0x7fbfc41034d0 2024-05-04T03:23:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfcb428700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4113400 con 0x7fbfc4108750 2024-05-04T03:23:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfc89c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 0x7fbfc419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfc89c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 0x7fbfc419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47366/0 (socket says 172.21.15.39:47366) 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfc89c3700 1 -- 172.21.15.39:0/2822713032 learned_addr learned my addr 172.21.15.39:0/2822713032 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfc99c5700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfc4108750 0x7fbfc419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc41034d0 0x7fbfc419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfc89c3700 1 -- 172.21.15.39:0/2822713032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfc4108750 msgr2=0x7fbfc419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.872+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfc4108750 0x7fbfc419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfc89c3700 1 -- 172.21.15.39:0/2822713032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc41034d0 msgr2=0x7fbfc419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc41034d0 0x7fbfc419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfc89c3700 1 -- 172.21.15.39:0/2822713032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc419fc30 con 0x7fbfc4107e00 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfc99c5700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfc4108750 0x7fbfc419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 0x7fbfc419b300 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0007d90 tx=0x7fbfc0007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfb67fc700 1 -- 172.21.15.39:0/2822713032 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc001f030 con 0x7fbfc4107e00 2024-05-04T03:23:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfcb428700 1 -- 172.21.15.39:0/2822713032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc419fec0 con 0x7fbfc4107e00 2024-05-04T03:23:38.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfb67fc700 1 -- 172.21.15.39:0/2822713032 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc001bb90 con 0x7fbfc4107e00 2024-05-04T03:23:38.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfb67fc700 1 -- 172.21.15.39:0/2822713032 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0012c20 con 0x7fbfc4107e00 2024-05-04T03:23:38.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.873+0000 7fbfcb428700 1 -- 172.21.15.39:0/2822713032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc41a7840 con 0x7fbfc4107e00 2024-05-04T03:23:38.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.876+0000 7fbfb67fc700 1 -- 172.21.15.39:0/2822713032 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbfc0022030 con 0x7fbfc4107e00 2024-05-04T03:23:38.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfb67fc700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfb00b2880 0x7fbfb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfb67fc700 1 -- 172.21.15.39:0/2822713032 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbfc000f020 con 0x7fbfc4107e00 2024-05-04T03:23:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfb00b2880 0x7fbfb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfd9e3cb80 1 Processor -- start 2024-05-04T03:23:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfd9e3cb80 1 -- start start 2024-05-04T03:23:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfb00b2880 0x7fbfb00b4d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e2bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2c420 0x562845e36700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e36c40 0x562845e39060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562845d4ea40 con 0x562845e2bad0 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562845cb3ea0 con 0x562845e2c420 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.877+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562845d4aa00 con 0x562845e36c40 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e2bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e2bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47374/0 (socket says 172.21.15.39:47374) 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 -- 172.21.15.39:0/726520035 learned_addr learned my addr 172.21.15.39:0/726520035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2c420 0x562845e36700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc99c5700 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e36c40 0x562845e39060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 -- 172.21.15.39:0/726520035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e36c40 msgr2=0x562845e39060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e36c40 0x562845e39060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 -- 172.21.15.39:0/726520035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2c420 msgr2=0x562845e36700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2c420 0x562845e36700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 -- 172.21.15.39:0/726520035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562845c9b9d0 con 0x562845e2bad0 2024-05-04T03:23:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.878+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e2bee0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800ec80 tx=0x7fbfb8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d89d94c140d62c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.879+0000 7fbfaf7fe700 1 -- 172.21.15.39:0/726520035 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8021070 con 0x562845e2bad0 2024-05-04T03:23:38.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.879+0000 7fbfaf7fe700 1 -- 172.21.15.39:0/726520035 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfb8010cd0 con 0x562845e2bad0 2024-05-04T03:23:38.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.879+0000 7fbfaf7fe700 1 -- 172.21.15.39:0/726520035 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800f620 con 0x562845e2bad0 2024-05-04T03:23:38.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.879+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/726520035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 msgr2=0x562845e2bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.879+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e2bee0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800ec80 tx=0x7fbfb8011000 comp rx=0 tx=0).stop 2024-05-04T03:23:38.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/726520035 shutdown_connections 2024-05-04T03:23:38.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e2bee0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e36c40 0x562845e39060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/726520035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2c420 0x562845e36700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/726520035 >> 172.21.15.39:0/726520035 conn(0x562845caef40 msgr2=0x562845e279d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/726520035 shutdown_connections 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/726520035 wait complete. 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 -- start start 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.880+0000 7fbfd9e3cb80 1 Processor -- start 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2bad0 0x562845e33cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2c420 0x562845e34200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e36c40 0x562845e34740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562845e39a10 con 0x562845e2c420 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x562845d07fd0 con 0x562845e36c40 2024-05-04T03:23:38.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562845e39b90 con 0x562845e2bad0 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfd9e3cb80 10 client.0 _ll_register_callbacks cb 0x562845ce6a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2c420 0x562845e34200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2c420 0x562845e34200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47378/0 (socket says 172.21.15.39:47378) 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 -- 172.21.15.39:0/3422422975 learned_addr learned my addr 172.21.15.39:0/3422422975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2bad0 0x562845e33cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc99c5700 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e36c40 0x562845e34740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 -- 172.21.15.39:0/3422422975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2bad0 msgr2=0x562845e33cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2bad0 0x562845e33cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 -- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e36c40 msgr2=0x562845e34740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e36c40 0x562845e34740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 -- 172.21.15.39:0/3422422975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562845d05860 con 0x562845e2c420 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc99c5700 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e36c40 0x562845e34740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2c420 0x562845e34200 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00e1010 tx=0x7fbfc000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c493a8e4e21f19ad server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.881+0000 7fbfacff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:23:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfacff9700 1 -- 172.21.15.39:0/3422422975 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00aef70 con 0x562845e2c420 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfacff9700 1 -- 172.21.15.39:0/3422422975 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbfc00e30e0 con 0x562845e2c420 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3422422975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb8010cd0 con 0x562845e2c420 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfacff9700 1 -- 172.21.15.39:0/3422422975 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbfc00b3990 con 0x562845e2c420 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 10 client.16740 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3422422975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562845e446f0 con 0x562845e2c420 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfacff9700 1 -- 172.21.15.39:0/3422422975 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfc00e32e0 con 0x562845e2c420 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfacff9700 1 -- 172.21.15.39:0/3422422975 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbfc0011070 con 0x562845e2c420 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 10 client.16740 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 10 client.16740 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 20 client.16740 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562845e46bc0 0x562845e48fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 4 client.16740 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.882+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3422422975 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbfb8010cd0 con 0x562845e46bc0 2024-05-04T03:23:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.883+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562845e46bc0 0x562845e48fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.883+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562845e46bc0 0x562845e48fe0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3714679c5cebc489 server_cookie=f5c7d8d9cfc00b43 in_seq=0 out_seq=0 2024-05-04T03:23:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.883+0000 7fbfacff9700 10 client.16740 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfacff9700 1 -- 172.21.15.39:0/3422422975 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbfb8010cd0 con 0x562845e46bc0 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfacff9700 10 client.16740 handle_command_reply: tid=0 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 client.16740 shutdown 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3422422975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2c420 msgr2=0x562845e34200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2c420 0x562845e34200 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00e1010 tx=0x7fbfc000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3422422975 shutdown_connections 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562845e46bc0 0x562845e48fe0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e36c40 0x562845e34740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2bad0 0x562845e33cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3422422975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2c420 0x562845e34200 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfacff9700 0 client.16740 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3422422975 >> 172.21.15.39:0/3422422975 conn(0x562845caef40 msgr2=0x562845e2d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:38.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3422422975 shutdown_connections 2024-05-04T03:23:38.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.884+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3422422975 wait complete. 2024-05-04T03:23:38.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.885+0000 7fbfd9e3cb80 20 client.16740 trim_cache size 0 max 16384 2024-05-04T03:23:38.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfd9e3cb80 1 Processor -- start 2024-05-04T03:23:38.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfd9e3cb80 1 -- start start 2024-05-04T03:23:38.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845caef40 0x562845e34580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2bad0 0x562845e2db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e2e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41aeb20 con 0x562845caef40 2024-05-04T03:23:38.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41b0660 con 0x562845e2bad0 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41af1f0 con 0x562845e2c420 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfc91c4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845caef40 0x562845e34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfc91c4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845caef40 0x562845e34580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47390/0 (socket says 172.21.15.39:47390) 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.905+0000 7fbfc91c4700 1 -- 172.21.15.39:0/3008411144 learned_addr learned my addr 172.21.15.39:0/3008411144 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc99c5700 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e2e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2bad0 0x562845e2db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc91c4700 1 -- 172.21.15.39:0/3008411144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 msgr2=0x562845e2e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e2e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc91c4700 1 -- 172.21.15.39:0/3008411144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2bad0 msgr2=0x562845e2db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2bad0 0x562845e2db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc91c4700 1 -- 172.21.15.39:0/3008411144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562845e34d80 con 0x562845caef40 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2bad0 0x562845e2db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:38.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845caef40 0x562845e34580 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8010710 tx=0x7fbfb800a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2b1c3033e35611 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfaeffd700 1 -- 172.21.15.39:0/3008411144 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8039070 con 0x562845caef40 2024-05-04T03:23:38.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfaeffd700 1 -- 172.21.15.39:0/3008411144 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfb80046f0 con 0x562845caef40 2024-05-04T03:23:38.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.906+0000 7fbfaeffd700 1 -- 172.21.15.39:0/3008411144 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8021040 con 0x562845caef40 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3008411144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845caef40 msgr2=0x562845e34580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845caef40 0x562845e34580 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8010710 tx=0x7fbfb800a460 comp rx=0 tx=0).stop 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3008411144 shutdown_connections 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845caef40 0x562845e34580 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e2e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/3008411144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845e2bad0 0x562845e2db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3008411144 >> 172.21.15.39:0/3008411144 conn(0x562845e20b60 msgr2=0x562845e2d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3008411144 shutdown_connections 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.907+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/3008411144 wait complete. 2024-05-04T03:23:38.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 1 -- start start 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 1 Processor -- start 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845caef40 0x562845e3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e3c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e3c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41aff90 con 0x562845e2bad0 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41af8c0 con 0x562845caef40 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41acac0 con 0x562845e2c420 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfd9e3cb80 10 client.0 _ll_register_callbacks cb 0x562845ce6a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfc89c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e3c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfc89c3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e3c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47398/0 (socket says 172.21.15.39:47398) 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.908+0000 7fbfc89c3700 1 -- 172.21.15.39:0/1005046512 learned_addr learned my addr 172.21.15.39:0/1005046512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845caef40 0x562845e3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc99c5700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e3c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc89c3700 1 -- 172.21.15.39:0/1005046512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 msgr2=0x562845e3c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e3c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc89c3700 1 -- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845caef40 msgr2=0x562845e3bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845caef40 0x562845e3bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc89c3700 1 -- 172.21.15.39:0/1005046512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562845eb13a0 con 0x562845e2bad0 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc99c5700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e3c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845caef40 0x562845e3bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfc89c3700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e3c070 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0022000 tx=0x7fbfc00c5350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb6a1443753790a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfacff9700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfacff9700 1 -- 172.21.15.39:0/1005046512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0005c60 con 0x562845e2bad0 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfacff9700 1 -- 172.21.15.39:0/1005046512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbfc0005e00 con 0x562845e2bad0 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.909+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/1005046512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb80046f0 con 0x562845e2bad0 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfacff9700 1 -- 172.21.15.39:0/1005046512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbfc00712d0 con 0x562845e2bad0 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfd9e3cb80 10 client.16746 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:38.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/1005046512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562845eb13a0 con 0x562845e2bad0 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfacff9700 1 -- 172.21.15.39:0/1005046512 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfb80046f0 con 0x562845e2bad0 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfacff9700 1 -- 172.21.15.39:0/1005046512 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbfc0011040 con 0x562845e2bad0 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfd9e3cb80 10 client.16746 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfd9e3cb80 10 client.16746 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfd9e3cb80 20 client.16746 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562845e36c40 0x562845eb4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfd9e3cb80 4 client.16746 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fcc00b97", "release": true, "await": true}] 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/1005046512 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fcc00b97", "release": true, "await": true}) -- 0x562845d5c420 con 0x562845e36c40 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.910+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562845e36c40 0x562845eb4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.911+0000 7fbfc91c4700 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562845e36c40 0x562845eb4940 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4300b09b3893a921 server_cookie=2d9811b87f073ae6 in_seq=0 out_seq=0 2024-05-04T03:23:38.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.911+0000 7fbfacff9700 10 client.16746 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfacff9700 1 -- 172.21.15.39:0/1005046512 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562845d5c420 con 0x562845e36c40 2024-05-04T03:23:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfacff9700 10 client.16746 handle_command_reply: tid=0 2024-05-04T03:23:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 client.16746 shutdown 2024-05-04T03:23:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/1005046512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 msgr2=0x562845e3c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e3c070 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0022000 tx=0x7fbfc00c5350 comp rx=0 tx=0).stop 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/1005046512 shutdown_connections 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x562845e36c40 0x562845eb4940 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562845e2c420 0x562845e3c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562845caef40 0x562845e3bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 --2- 172.21.15.39:0/1005046512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562845e2bad0 0x562845e3c070 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfacff9700 0 client.16746 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/1005046512 >> 172.21.15.39:0/1005046512 conn(0x562845e20b60 msgr2=0x562845e2f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/1005046512 shutdown_connections 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 1 -- 172.21.15.39:0/1005046512 wait complete. 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.916+0000 7fbfd9e3cb80 20 client.16746 trim_cache size 0 max 16384 2024-05-04T03:23:38.919 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 36, 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "fcc00b97": { 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 36, 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:23:38.920 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 90.0, 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:23:38.921 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 -- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfb00b2880 msgr2=0x7fbfb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfb00b2880 0x7fbfb00b4d90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8005fd0 tx=0x7fbfb800a560 comp rx=0 tx=0).stop 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 -- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 msgr2=0x7fbfc419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 0x7fbfc419b300 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0007d90 tx=0x7fbfc0007e70 comp rx=0 tx=0).stop 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 -- 172.21.15.39:0/2822713032 shutdown_connections 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfb00b2880 0x7fbfb00b4d90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbfc4108750 0x7fbfc419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbfc41034d0 0x7fbfc419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 --2- 172.21.15.39:0/2822713032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfc4107e00 0x7fbfc419b300 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 -- 172.21.15.39:0/2822713032 >> 172.21.15.39:0/2822713032 conn(0x7fbfc40fef20 msgr2=0x7fbfc4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:38.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 -- 172.21.15.39:0/2822713032 shutdown_connections 2024-05-04T03:23:38.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:23:38.917+0000 7fbfcb428700 1 -- 172.21.15.39:0/2822713032 wait complete. 2024-05-04T03:23:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.979+0000 7f7857b3f640 1 -- 172.21.15.39:0/7181887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850103dc0 msgr2=0x7f78501041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.979+0000 7f7857b3f640 1 --2- 172.21.15.39:0/7181887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850103dc0 0x7f78501041d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f784c009900 tx=0x7f784c02a3f0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.979+0000 7f7857b3f640 1 -- 172.21.15.39:0/7181887 shutdown_connections 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.979+0000 7f7857b3f640 1 --2- 172.21.15.39:0/7181887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7850108ac0 0x7f785010aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.979+0000 7f7857b3f640 1 --2- 172.21.15.39:0/7181887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850103dc0 0x7f78501041d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.979+0000 7f7857b3f640 1 --2- 172.21.15.39:0/7181887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7850102b80 0x7f7850103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.980+0000 7f7857b3f640 1 -- 172.21.15.39:0/7181887 >> 172.21.15.39:0/7181887 conn(0x7f78500fdd90 msgr2=0x7f7850100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.980+0000 7f7857b3f640 1 -- 172.21.15.39:0/7181887 shutdown_connections 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.980+0000 7f7857b3f640 1 -- 172.21.15.39:0/7181887 wait complete. 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.980+0000 7f7857b3f640 1 Processor -- start 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.980+0000 7f7857b3f640 1 -- start start 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f7857b3f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850102b80 0x7f785019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f7857b3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7850103dc0 0x7f785019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f7857b3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7850108ac0 0x7f78501a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f7857b3f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f785010f6c0 con 0x7f7850102b80 2024-05-04T03:23:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f7857b3f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f785010f540 con 0x7f7850108ac0 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f7857b3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f785010f840 con 0x7f7850103dc0 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850102b80 0x7f785019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850102b80 0x7f785019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47420/0 (socket says 172.21.15.39:47420) 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 -- 172.21.15.39:0/1793885679 learned_addr learned my addr 172.21.15.39:0/1793885679 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78550da640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7850103dc0 0x7f785019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78560dc640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7850108ac0 0x7f78501a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 -- 172.21.15.39:0/1793885679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7850103dc0 msgr2=0x7f785019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7850103dc0 0x7f785019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 -- 172.21.15.39:0/1793885679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7850108ac0 msgr2=0x7f78501a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7850108ac0 0x7f78501a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 -- 172.21.15.39:0/1793885679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78501a1d50 con 0x7f7850102b80 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78560dc640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7850108ac0 0x7f78501a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.981+0000 7f78558db640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850102b80 0x7f785019a2c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f784000e9f0 tx=0x7f784000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.982+0000 7f7846ffd640 1 -- 172.21.15.39:0/1793885679 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784000ccc0 con 0x7f7850102b80 2024-05-04T03:23:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.982+0000 7f7846ffd640 1 -- 172.21.15.39:0/1793885679 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78400035f0 con 0x7f7850102b80 2024-05-04T03:23:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.982+0000 7f7857b3f640 1 -- 172.21.15.39:0/1793885679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78501a1fe0 con 0x7f7850102b80 2024-05-04T03:23:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.982+0000 7f7846ffd640 1 -- 172.21.15.39:0/1793885679 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784000ce60 con 0x7f7850102b80 2024-05-04T03:23:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.982+0000 7f7857b3f640 1 -- 172.21.15.39:0/1793885679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78501a24a0 con 0x7f7850102b80 2024-05-04T03:23:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f785e9f6740 1 Processor -- start 2024-05-04T03:23:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f785e9f6740 1 -- start start 2024-05-04T03:23:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b628a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b25e2f0 0x55eb3b628fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b2664d0 0x55eb3b62fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f785e9f6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eb3b58d5d0 con 0x55eb3b25e2f0 2024-05-04T03:23:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f785e9f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eb3b546890 con 0x55eb3b2664d0 2024-05-04T03:23:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f785e9f6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb3b589590 con 0x55eb3b2871b0 2024-05-04T03:23:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.986+0000 7f7846ffd640 1 -- 172.21.15.39:0/1793885679 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7840004000 con 0x7f7850102b80 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78560dc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b628a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78550da640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b2664d0 0x55eb3b62fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78560dc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b628a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:39498/0 (socket says 172.21.15.39:39498) 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78560dc640 1 -- 172.21.15.39:0/3992431219 learned_addr learned my addr 172.21.15.39:0/3992431219 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78550da640 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b2664d0 0x55eb3b62fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:38118/0 (socket says 172.21.15.39:38118) 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78558db640 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b25e2f0 0x55eb3b628fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78550da640 1 -- 172.21.15.39:0/3992431219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 msgr2=0x55eb3b628a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78550da640 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b628a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78550da640 1 -- 172.21.15.39:0/3992431219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b25e2f0 msgr2=0x55eb3b628fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78550da640 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b25e2f0 0x55eb3b628fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78550da640 1 -- 172.21.15.39:0/3992431219 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3b29f210 con 0x55eb3b2664d0 2024-05-04T03:23:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.987+0000 7f78550da640 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b2664d0 0x55eb3b62fee0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f784c02a910 tx=0x7f784c004200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97a59eba961bb881 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.988+0000 7f783bfff640 1 -- 172.21.15.39:0/3992431219 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c035030 con 0x55eb3b2664d0 2024-05-04T03:23:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.989+0000 7f783bfff640 1 -- 172.21.15.39:0/3992431219 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f784c040070 con 0x55eb3b2664d0 2024-05-04T03:23:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.989+0000 7f783bfff640 1 -- 172.21.15.39:0/3992431219 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c031940 con 0x55eb3b2664d0 2024-05-04T03:23:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.989+0000 7f7846ffd640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f783c0b2860 0x7f783c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f7846ffd640 1 -- 172.21.15.39:0/1793885679 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f78400e25a0 con 0x7f7850102b80 2024-05-04T03:23:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f78560dc640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f783c0b2860 0x7f783c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 -- 172.21.15.39:0/3992431219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b2664d0 msgr2=0x55eb3b62fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b2664d0 0x55eb3b62fee0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f784c02a910 tx=0x7f784c004200 comp rx=0 tx=0).stop 2024-05-04T03:23:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 -- 172.21.15.39:0/3992431219 shutdown_connections 2024-05-04T03:23:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b2664d0 0x55eb3b62fee0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b25e2f0 0x55eb3b628fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3992431219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b628a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 -- 172.21.15.39:0/3992431219 >> 172.21.15.39:0/3992431219 conn(0x55eb3b27b300 msgr2=0x55eb3b6243a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f78560dc640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f783c0b2860 0x7f783c0b4d70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55eb3b629c30 tx=0x7f78480109a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 -- 172.21.15.39:0/3992431219 shutdown_connections 2024-05-04T03:23:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.990+0000 7f785e9f6740 1 -- 172.21.15.39:0/3992431219 wait complete. 2024-05-04T03:23:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.991+0000 7f785e9f6740 1 -- start start 2024-05-04T03:23:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.991+0000 7f785e9f6740 1 Processor -- start 2024-05-04T03:23:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.992+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b632650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78558db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b632650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b632b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b639af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78558db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b632650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:38132/0 (socket says 172.21.15.39:38132) 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78558db640 1 -- 172.21.15.39:0/3235700368 learned_addr learned my addr 172.21.15.39:0/3235700368 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55eb3b6308d0 con 0x55eb3b2664d0 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55eb3b54a370 con 0x55eb3b25e2f0 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb3b630a50 con 0x55eb3b2871b0 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78550da640 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b632b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78560dc640 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b639af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f785e9f6740 10 client.0 _ll_register_callbacks cb 0x55eb3b29ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78560dc640 1 -- 172.21.15.39:0/3235700368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 msgr2=0x55eb3b632650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78560dc640 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b632650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78560dc640 1 -- 172.21.15.39:0/3235700368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 msgr2=0x55eb3b632b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78560dc640 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b632b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.993+0000 7f78560dc640 1 -- 172.21.15.39:0/3235700368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3b36b620 con 0x55eb3b2871b0 2024-05-04T03:23:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.994+0000 7f78560dc640 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b639af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7848007e50 tx=0x7f7848004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b68a5c281129e3fe server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.994+0000 7f78397fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:23:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.994+0000 7f78397fa640 1 -- 172.21.15.39:0/3235700368 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784802c060 con 0x55eb3b2871b0 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.994+0000 7f78397fa640 1 -- 172.21.15.39:0/3235700368 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7848027400 con 0x55eb3b2871b0 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.994+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f784c040070 con 0x55eb3b2871b0 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.995+0000 7f78397fa640 1 -- 172.21.15.39:0/3235700368 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7848027600 con 0x55eb3b2871b0 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.995+0000 7f785e9f6740 10 client.26368 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.995+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eb3b63c8a0 con 0x55eb3b2871b0 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.995+0000 7f78397fa640 1 -- 172.21.15.39:0/3235700368 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f784c040070 con 0x55eb3b2871b0 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.995+0000 7f78397fa640 1 -- 172.21.15.39:0/3235700368 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7848032740 con 0x55eb3b2871b0 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.996+0000 7f785e9f6740 10 client.26368 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.996+0000 7f785e9f6740 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.996+0000 7f785e9f6740 20 client.26368 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.996+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb3b245d80 0x55eb3b640060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.996+0000 7f785e9f6740 4 client.26368 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.996+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb3b59b1e0 con 0x55eb3b245d80 2024-05-04T03:23:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.997+0000 7f78558db640 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb3b245d80 0x55eb3b640060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.997+0000 7f78397fa640 10 client.26368 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.997+0000 7f78558db640 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb3b245d80 0x55eb3b640060 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2037c92cdc1290b5 server_cookie=6fdda280c03a75a2 in_seq=0 out_seq=0 2024-05-04T03:23:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.998+0000 7f78397fa640 1 -- 172.21.15.39:0/3235700368 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb3b59b1e0 con 0x55eb3b245d80 2024-05-04T03:23:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.998+0000 7f78397fa640 10 client.26368 handle_command_reply: tid=0 2024-05-04T03:23:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.999+0000 7f785e9f6740 1 client.26368 shutdown 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.999+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 msgr2=0x55eb3b639af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.999+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b639af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7848007e50 tx=0x7f7848004a40 comp rx=0 tx=0).stop 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.999+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 shutdown_connections 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:38.999+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb3b245d80 0x55eb3b640060 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.000+0000 7f78397fa640 0 client.26368 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.000+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2871b0 0x55eb3b639af0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.000+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b632b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.000+0000 7f785e9f6740 1 --2- 172.21.15.39:0/3235700368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b632650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.000+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 >> 172.21.15.39:0/3235700368 conn(0x55eb3b27b300 msgr2=0x55eb3b627300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.000+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 shutdown_connections 2024-05-04T03:23:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.000+0000 7f785e9f6740 1 -- 172.21.15.39:0/3235700368 wait complete. 2024-05-04T03:23:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.000+0000 7f785e9f6740 20 client.26368 trim_cache size 0 max 16384 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.030+0000 7f785e9f6740 1 Processor -- start 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.030+0000 7f785e9f6740 1 -- start start 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.030+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b245d80 0x55eb3b624310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b639a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2664d0 0x55eb3b639f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f785e9f6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78501a8ff0 con 0x55eb3b245d80 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f785e9f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f78501aab30 con 0x55eb3b25e2f0 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78558db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b245d80 0x55eb3b624310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78558db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b245d80 0x55eb3b624310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47454/0 (socket says 172.21.15.39:47454) 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78558db640 1 -- 172.21.15.39:0/4201113445 learned_addr learned my addr 172.21.15.39:0/4201113445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f785e9f6740 1 -- 172.21.15.39:0/4201113445 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78501a96c0 con 0x55eb3b2664d0 2024-05-04T03:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78560dc640 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2664d0 0x55eb3b639f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78550da640 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b639a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78558db640 1 -- 172.21.15.39:0/4201113445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2664d0 msgr2=0x55eb3b639f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78558db640 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2664d0 0x55eb3b639f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78558db640 1 -- 172.21.15.39:0/4201113445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 msgr2=0x55eb3b639a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78558db640 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b639a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.031+0000 7f78558db640 1 -- 172.21.15.39:0/4201113445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3b624d00 con 0x55eb3b245d80 2024-05-04T03:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.032+0000 7f78558db640 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b245d80 0x55eb3b624310 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f78400a5c50 tx=0x7f78400dff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a504d837ae0d9d00 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.032+0000 7f783b7fe640 1 -- 172.21.15.39:0/4201113445 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78400a5360 con 0x55eb3b245d80 2024-05-04T03:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.032+0000 7f783b7fe640 1 -- 172.21.15.39:0/4201113445 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78400b2370 con 0x55eb3b245d80 2024-05-04T03:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.032+0000 7f783b7fe640 1 -- 172.21.15.39:0/4201113445 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78400b2a30 con 0x55eb3b245d80 2024-05-04T03:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.033+0000 7f785e9f6740 1 -- 172.21.15.39:0/4201113445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b245d80 msgr2=0x55eb3b624310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.033+0000 7f785e9f6740 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b245d80 0x55eb3b624310 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f78400a5c50 tx=0x7f78400dff80 comp rx=0 tx=0).stop 2024-05-04T03:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.034+0000 7f785e9f6740 1 -- 172.21.15.39:0/4201113445 shutdown_connections 2024-05-04T03:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.034+0000 7f785e9f6740 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b2664d0 0x55eb3b639f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.034+0000 7f785e9f6740 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b25e2f0 0x55eb3b639a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.034+0000 7f785e9f6740 1 --2- 172.21.15.39:0/4201113445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b245d80 0x55eb3b624310 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.034+0000 7f785e9f6740 1 -- 172.21.15.39:0/4201113445 >> 172.21.15.39:0/4201113445 conn(0x55eb3b27b300 msgr2=0x55eb3b623c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.034+0000 7f785e9f6740 1 -- 172.21.15.39:0/4201113445 shutdown_connections 2024-05-04T03:23:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.034+0000 7f785e9f6740 1 -- 172.21.15.39:0/4201113445 wait complete. 2024-05-04T03:23:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.035+0000 7f785e9f6740 1 -- start start 2024-05-04T03:23:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.035+0000 7f785e9f6740 1 Processor -- start 2024-05-04T03:23:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b245d80 0x55eb3b638010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b25e2f0 0x55eb3b638550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f785e9f6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b639480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f785e9f6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78501aa460 con 0x55eb3b2664d0 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f785e9f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f78501a9d90 con 0x55eb3b245d80 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f785e9f6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78501a6fd0 con 0x55eb3b25e2f0 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f785e9f6740 10 client.0 _ll_register_callbacks cb 0x7f7848010be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f78560dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b639480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f78550da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b25e2f0 0x55eb3b638550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.036+0000 7f78550da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b25e2f0 0x55eb3b638550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:39516/0 (socket says 172.21.15.39:39516) 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78550da640 1 -- 172.21.15.39:0/257369779 learned_addr learned my addr 172.21.15.39:0/257369779 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78558db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b245d80 0x55eb3b638010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78550da640 1 -- 172.21.15.39:0/257369779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b245d80 msgr2=0x55eb3b638010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78550da640 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b245d80 0x55eb3b638010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78550da640 1 -- 172.21.15.39:0/257369779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 msgr2=0x55eb3b639480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78550da640 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b639480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78550da640 1 -- 172.21.15.39:0/257369779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3b639a40 con 0x55eb3b25e2f0 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78558db640 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b245d80 0x55eb3b638010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.037+0000 7f78560dc640 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b639480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.038+0000 7f78397fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.038+0000 7f78550da640 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b25e2f0 0x55eb3b638550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f784c009900 tx=0x7f784c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dad0cb8218c0e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.038+0000 7f78397fa640 1 -- 172.21.15.39:0/257369779 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c055060 con 0x55eb3b25e2f0 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.038+0000 7f78397fa640 1 -- 172.21.15.39:0/257369779 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f784c031570 con 0x55eb3b25e2f0 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.038+0000 7f785e9f6740 1 -- 172.21.15.39:0/257369779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb3b595050 con 0x55eb3b25e2f0 2024-05-04T03:23:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.039+0000 7f78397fa640 1 -- 172.21.15.39:0/257369779 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f784c035070 con 0x55eb3b25e2f0 2024-05-04T03:23:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.039+0000 7f785e9f6740 10 client.26374 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.039+0000 7f785e9f6740 1 -- 172.21.15.39:0/257369779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eb3b6a8970 con 0x55eb3b25e2f0 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.039+0000 7f78397fa640 1 -- 172.21.15.39:0/257369779 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f784c040040 con 0x55eb3b25e2f0 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.040+0000 7f78397fa640 1 -- 172.21.15.39:0/257369779 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f784c050470 con 0x55eb3b25e2f0 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.040+0000 7f785e9f6740 10 client.26374 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.040+0000 7f785e9f6740 10 client.26374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.040+0000 7f785e9f6740 20 client.26374 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.040+0000 7f785e9f6740 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb3b2871b0 0x55eb3b6aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.040+0000 7f785e9f6740 4 client.26374 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.040+0000 7f785e9f6740 1 -- 172.21.15.39:0/257369779 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb3b595050 con 0x55eb3b2871b0 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.040+0000 7f78558db640 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb3b2871b0 0x55eb3b6aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.041+0000 7f78397fa640 10 client.26374 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.041+0000 7f78558db640 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb3b2871b0 0x55eb3b6aaed0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3727f11161eb31a server_cookie=9c3a84d69ae6d6f2 in_seq=0 out_seq=0 2024-05-04T03:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f78397fa640 1 -- 172.21.15.39:0/257369779 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb3b595050 con 0x55eb3b2871b0 2024-05-04T03:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f78397fa640 10 client.26374 handle_command_reply: tid=0 2024-05-04T03:23:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 client.26374 shutdown 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 -- 172.21.15.39:0/257369779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b25e2f0 msgr2=0x55eb3b638550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b25e2f0 0x55eb3b638550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f784c009900 tx=0x7f784c009260 comp rx=0 tx=0).stop 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 -- 172.21.15.39:0/257369779 shutdown_connections 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55eb3b2871b0 0x55eb3b6aaed0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55eb3b2664d0 0x55eb3b639480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb3b25e2f0 0x55eb3b638550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 --2- 172.21.15.39:0/257369779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55eb3b245d80 0x55eb3b638010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f785e9f6740 1 -- 172.21.15.39:0/257369779 >> 172.21.15.39:0/257369779 conn(0x55eb3b27b300 msgr2=0x55eb3b627c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.046+0000 7f78397fa640 0 client.26374 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.047+0000 7f785e9f6740 1 -- 172.21.15.39:0/257369779 shutdown_connections 2024-05-04T03:23:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.047+0000 7f785e9f6740 1 -- 172.21.15.39:0/257369779 wait complete. 2024-05-04T03:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.047+0000 7f785e9f6740 20 client.26374 trim_cache size 0 max 16384 2024-05-04T03:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:23:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 -- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f783c0b2860 msgr2=0x7f783c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f783c0b2860 0x7f783c0b4d70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55eb3b629c30 tx=0x7f78480109a0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 -- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850102b80 msgr2=0x7f785019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850102b80 0x7f785019a2c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f784000e9f0 tx=0x7f784000eeb0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 -- 172.21.15.39:0/1793885679 shutdown_connections 2024-05-04T03:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f783c0b2860 0x7f783c0b4d70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7850108ac0 0x7f78501a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7850103dc0 0x7f785019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 --2- 172.21.15.39:0/1793885679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7850102b80 0x7f785019a2c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 -- 172.21.15.39:0/1793885679 >> 172.21.15.39:0/1793885679 conn(0x7f78500fdd90 msgr2=0x7f7850104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 -- 172.21.15.39:0/1793885679 shutdown_connections 2024-05-04T03:23:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:39.048+0000 7f7857b3f640 1 -- 172.21.15.39:0/1793885679 wait complete. 2024-05-04T03:23:39.734 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fcc00b97', seconds elapsed: 43.3 2024-05-04T03:23:40.677 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:40 smithi167 ceph-mon[33614]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 308 KiB/s wr, 36 op/s 2024-05-04T03:23:40.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:40 smithi039 ceph-mon[29715]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 308 KiB/s wr, 36 op/s 2024-05-04T03:23:40.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:40 smithi066 ceph-mon[33264]: pgmap v772: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 308 KiB/s wr, 36 op/s 2024-05-04T03:23:41.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 590.0 s, 587.7 tps, lat 1322.277 ms stddev 6636.901 2024-05-04T03:23:42.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:42 smithi167 ceph-mon[33614]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1015 KiB/s wr, 73 op/s 2024-05-04T03:23:42.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:42 smithi039 ceph-mon[29715]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1015 KiB/s wr, 73 op/s 2024-05-04T03:23:42.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:42 smithi066 ceph-mon[33264]: pgmap v773: 97 pgs: 97 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1015 KiB/s wr, 73 op/s 2024-05-04T03:23:44.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:44 smithi167 ceph-mon[33614]: pgmap v774: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.5 MiB/s wr, 219 op/s 2024-05-04T03:23:44.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:44 smithi039 ceph-mon[29715]: pgmap v774: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.5 MiB/s wr, 219 op/s 2024-05-04T03:23:44.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:44 smithi066 ceph-mon[33264]: pgmap v774: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.5 MiB/s wr, 219 op/s 2024-05-04T03:23:46.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:46 smithi167 ceph-mon[33614]: pgmap v775: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-05-04T03:23:46.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:46 smithi039 ceph-mon[29715]: pgmap v775: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-05-04T03:23:46.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 595.0 s, 1147.1 tps, lat 97.071 ms stddev 134.321 2024-05-04T03:23:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:46 smithi066 ceph-mon[33264]: pgmap v775: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 393 op/s 2024-05-04T03:23:48.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:48 smithi167 ceph-mon[33614]: pgmap v776: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 667 op/s 2024-05-04T03:23:48.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:48 smithi039 ceph-mon[29715]: pgmap v776: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 667 op/s 2024-05-04T03:23:48.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:48 smithi066 ceph-mon[33264]: pgmap v776: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 667 op/s 2024-05-04T03:23:50.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:50 smithi167 ceph-mon[33614]: pgmap v777: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-05-04T03:23:50.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:50 smithi039 ceph-mon[29715]: pgmap v777: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-05-04T03:23:50.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:50 smithi066 ceph-mon[33264]: pgmap v777: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-05-04T03:23:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 600.0 s, 969.3 tps, lat 103.001 ms stddev 98.020 2024-05-04T03:23:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.208+0000 7f4724d3f640 1 -- 172.21.15.39:0/978710596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f472010bdb0 msgr2=0x7f472010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.208+0000 7f4724d3f640 1 --2- 172.21.15.39:0/978710596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f472010bdb0 0x7f472010e1d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4714009900 tx=0x7f471402a410 comp rx=0 tx=0).stop 2024-05-04T03:23:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.209+0000 7f4724d3f640 1 -- 172.21.15.39:0/978710596 shutdown_connections 2024-05-04T03:23:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.209+0000 7f4724d3f640 1 --2- 172.21.15.39:0/978710596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f472010bdb0 0x7f472010e1d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.209+0000 7f4724d3f640 1 --2- 172.21.15.39:0/978710596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47201070b0 0x7f47201074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.209+0000 7f4724d3f640 1 --2- 172.21.15.39:0/978710596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4720105e70 0x7f4720106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.209+0000 7f4724d3f640 1 -- 172.21.15.39:0/978710596 >> 172.21.15.39:0/978710596 conn(0x7f4720101040 msgr2=0x7f4720103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.209+0000 7f4724d3f640 1 -- 172.21.15.39:0/978710596 shutdown_connections 2024-05-04T03:23:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.209+0000 7f4724d3f640 1 -- 172.21.15.39:0/978710596 wait complete. 2024-05-04T03:23:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.210+0000 7f4724d3f640 1 Processor -- start 2024-05-04T03:23:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.210+0000 7f4724d3f640 1 -- start start 2024-05-04T03:23:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f4724d3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4720105e70 0x7f472019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f4724d3f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47201070b0 0x7f472019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f4724d3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f472010bdb0 0x7f47201a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f471e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4720105e70 0x7f472019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f471e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4720105e70 0x7f472019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:58324/0 (socket says 172.21.15.39:58324) 2024-05-04T03:23:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f471e59c640 1 -- 172.21.15.39:0/4132124480 learned_addr learned my addr 172.21.15.39:0/4132124480 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4720111870 con 0x7f47201070b0 2024-05-04T03:23:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47201116f0 con 0x7f472010bdb0 2024-05-04T03:23:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.211+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47201119f0 con 0x7f4720105e70 2024-05-04T03:23:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.212+0000 7f471ed9d640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f472010bdb0 0x7f47201a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.212+0000 7f471e59c640 1 -- 172.21.15.39:0/4132124480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f472010bdb0 msgr2=0x7f47201a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.212+0000 7f471e59c640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f472010bdb0 0x7f47201a38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.212+0000 7f471e59c640 1 -- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47201070b0 msgr2=0x7f472019c950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:23:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.212+0000 7f471e59c640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47201070b0 0x7f472019c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.212+0000 7f471e59c640 1 -- 172.21.15.39:0/4132124480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47201a3f50 con 0x7f4720105e70 2024-05-04T03:23:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.212+0000 7f471e59c640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4720105e70 0x7f472019c410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f471800eb90 tx=0x7f471800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.213+0000 7f470f7fe640 1 -- 172.21.15.39:0/4132124480 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718003b70 con 0x7f4720105e70 2024-05-04T03:23:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.213+0000 7f470f7fe640 1 -- 172.21.15.39:0/4132124480 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f471800ee90 con 0x7f4720105e70 2024-05-04T03:23:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.213+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47201a4240 con 0x7f4720105e70 2024-05-04T03:23:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.213+0000 7f470f7fe640 1 -- 172.21.15.39:0/4132124480 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47180196e0 con 0x7f4720105e70 2024-05-04T03:23:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.216+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47201a4730 con 0x7f4720105e70 2024-05-04T03:23:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.217+0000 7f470f7fe640 1 -- 172.21.15.39:0/4132124480 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4718004030 con 0x7f4720105e70 2024-05-04T03:23:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.219+0000 7f470f7fe640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f47080b2930 0x7f47080b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.219+0000 7f470f7fe640 1 -- 172.21.15.39:0/4132124480 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f47180e5270 con 0x7f4720105e70 2024-05-04T03:23:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.219+0000 7f471dd9b640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f47080b2930 0x7f47080b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.220+0000 7f472bbf5740 1 Processor -- start 2024-05-04T03:23:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.220+0000 7f472bbf5740 1 -- start start 2024-05-04T03:23:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.220+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c36a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.220+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c36fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.220+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567438744d0 0x556743c3dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.220+0000 7f472bbf5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556743b9b5d0 con 0x5567438744d0 2024-05-04T03:23:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.220+0000 7f472bbf5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556743b54890 con 0x5567438951b0 2024-05-04T03:23:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.220+0000 7f472bbf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556743b97590 con 0x55674386c2f0 2024-05-04T03:23:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c36fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567438744d0 0x556743c3dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c36fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:58336/0 (socket says 172.21.15.39:58336) 2024-05-04T03:23:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 -- 172.21.15.39:0/2234669823 learned_addr learned my addr 172.21.15.39:0/2234669823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471e59c640 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c36a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 -- 172.21.15.39:0/2234669823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 msgr2=0x556743c36a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c36a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 -- 172.21.15.39:0/2234669823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567438744d0 msgr2=0x556743c3dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567438744d0 0x556743c3dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 -- 172.21.15.39:0/2234669823 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567438ad210 con 0x55674386c2f0 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471dd9b640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f47080b2930 0x7f47080b4e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f472019d4b0 tx=0x7f4710005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471e59c640 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c36a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.221+0000 7f471ed9d640 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567438744d0 0x556743c3dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.222+0000 7f471dd9b640 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c36fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4710005d00 tx=0x7f4710019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd75c25a26819f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.222+0000 7f470cff9640 1 -- 172.21.15.39:0/2234669823 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710010cb0 con 0x55674386c2f0 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.222+0000 7f470cff9640 1 -- 172.21.15.39:0/2234669823 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4710010e50 con 0x55674386c2f0 2024-05-04T03:23:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.222+0000 7f470cff9640 1 -- 172.21.15.39:0/2234669823 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710023740 con 0x55674386c2f0 2024-05-04T03:23:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 -- 172.21.15.39:0/2234669823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 msgr2=0x556743c36fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c36fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4710005d00 tx=0x7f4710019000 comp rx=0 tx=0).stop 2024-05-04T03:23:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 -- 172.21.15.39:0/2234669823 shutdown_connections 2024-05-04T03:23:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567438744d0 0x556743c3dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c36fe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 --2- 172.21.15.39:0/2234669823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c36a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 -- 172.21.15.39:0/2234669823 >> 172.21.15.39:0/2234669823 conn(0x556743889300 msgr2=0x556743c323a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 -- 172.21.15.39:0/2234669823 shutdown_connections 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.223+0000 7f472bbf5740 1 -- 172.21.15.39:0/2234669823 wait complete. 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f472bbf5740 1 -- start start 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f472bbf5740 1 Processor -- start 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55674386c2f0 0x556743c406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 0x556743c40bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c47b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f472bbf5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556743c3e930 con 0x55674386c2f0 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f472bbf5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x556743b58370 con 0x5567438951b0 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f472bbf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556743c3eab0 con 0x5567438744d0 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.224+0000 7f471e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55674386c2f0 0x556743c406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f472bbf5740 10 client.0 _ll_register_callbacks cb 0x5567438aca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55674386c2f0 0x556743c406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48798/0 (socket says 172.21.15.39:48798) 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471e59c640 1 -- 172.21.15.39:0/110336975 learned_addr learned my addr 172.21.15.39:0/110336975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471dd9b640 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 0x556743c40bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471ed9d640 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c47b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471e59c640 1 -- 172.21.15.39:0/110336975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 msgr2=0x556743c40bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471e59c640 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 0x556743c40bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471e59c640 1 -- 172.21.15.39:0/110336975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 msgr2=0x556743c47b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471e59c640 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c47b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471e59c640 1 -- 172.21.15.39:0/110336975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556743979620 con 0x55674386c2f0 2024-05-04T03:23:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f471e59c640 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55674386c2f0 0x556743c406b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f47180a96a0 tx=0x7f47180045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e4ae3e8347b612 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.225+0000 7f4705ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.226+0000 7f4705ffb640 1 -- 172.21.15.39:0/110336975 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47180e5070 con 0x55674386c2f0 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.226+0000 7f4705ffb640 1 -- 172.21.15.39:0/110336975 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4718004b60 con 0x55674386c2f0 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.226+0000 7f472bbf5740 1 -- 172.21.15.39:0/110336975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4710010e50 con 0x55674386c2f0 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.226+0000 7f4705ffb640 1 -- 172.21.15.39:0/110336975 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f471800a460 con 0x55674386c2f0 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.226+0000 7f472bbf5740 10 client.16773 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.226+0000 7f472bbf5740 1 -- 172.21.15.39:0/110336975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556743c4a8e0 con 0x55674386c2f0 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.226+0000 7f4705ffb640 1 -- 172.21.15.39:0/110336975 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4710010e50 con 0x55674386c2f0 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.227+0000 7f4705ffb640 1 -- 172.21.15.39:0/110336975 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f47180a8b10 con 0x55674386c2f0 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.227+0000 7f472bbf5740 10 client.16773 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.227+0000 7f472bbf5740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.227+0000 7f472bbf5740 20 client.16773 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.227+0000 7f472bbf5740 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556743853d80 0x556743c4df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.227+0000 7f472bbf5740 4 client.16773 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.227+0000 7f472bbf5740 1 -- 172.21.15.39:0/110336975 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556743ba91e0 con 0x556743853d80 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.227+0000 7f471dd9b640 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556743853d80 0x556743c4df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.228+0000 7f4705ffb640 10 client.16773 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.228+0000 7f471dd9b640 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556743853d80 0x556743c4df50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aabb836ace76389a server_cookie=9d1afd77b9841962 in_seq=0 out_seq=0 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f4705ffb640 1 -- 172.21.15.39:0/110336975 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556743ba91e0 con 0x556743853d80 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f4705ffb640 10 client.16773 handle_command_reply: tid=0 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 client.16773 shutdown 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 -- 172.21.15.39:0/110336975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55674386c2f0 msgr2=0x556743c406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55674386c2f0 0x556743c406b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f47180a96a0 tx=0x7f47180045d0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 -- 172.21.15.39:0/110336975 shutdown_connections 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x556743853d80 0x556743c4df50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438951b0 0x556743c47b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f4705ffb640 0 client.16773 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 0x556743c40bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 --2- 172.21.15.39:0/110336975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55674386c2f0 0x556743c406b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 -- 172.21.15.39:0/110336975 >> 172.21.15.39:0/110336975 conn(0x556743889300 msgr2=0x556743c313a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.229+0000 7f472bbf5740 1 -- 172.21.15.39:0/110336975 shutdown_connections 2024-05-04T03:23:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.230+0000 7f472bbf5740 1 -- 172.21.15.39:0/110336975 wait complete. 2024-05-04T03:23:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.230+0000 7f472bbf5740 20 client.16773 trim_cache size 0 max 16384 2024-05-04T03:23:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.257+0000 7f472bbf5740 1 Processor -- start 2024-05-04T03:23:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.257+0000 7f472bbf5740 1 -- start start 2024-05-04T03:23:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.257+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c31cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.257+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c47ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.257+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438744d0 0x556743c32230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.257+0000 7f472bbf5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f47201a9170 con 0x556743889300 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.257+0000 7f472bbf5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47201aacb0 con 0x5567438744d0 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.257+0000 7f472bbf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47201a9840 con 0x55674386c2f0 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c31cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c31cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48804/0 (socket says 172.21.15.39:48804) 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471e59c640 1 -- 172.21.15.39:0/4222867286 learned_addr learned my addr 172.21.15.39:0/4222867286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471ed9d640 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438744d0 0x556743c32230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471dd9b640 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c47ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471ed9d640 1 -- 172.21.15.39:0/4222867286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 msgr2=0x556743c47ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471ed9d640 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c47ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471ed9d640 1 -- 172.21.15.39:0/4222867286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 msgr2=0x556743c31cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471ed9d640 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c31cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471ed9d640 1 -- 172.21.15.39:0/4222867286 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556743c32d00 con 0x5567438744d0 2024-05-04T03:23:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.258+0000 7f471e59c640 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c31cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.259+0000 7f471ed9d640 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438744d0 0x556743c32230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f471402aa00 tx=0x7f4714004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7381aa54c12d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.259+0000 7f4707fff640 1 -- 172.21.15.39:0/4222867286 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714035040 con 0x5567438744d0 2024-05-04T03:23:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.259+0000 7f4707fff640 1 -- 172.21.15.39:0/4222867286 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4714003a80 con 0x5567438744d0 2024-05-04T03:23:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.259+0000 7f4707fff640 1 -- 172.21.15.39:0/4222867286 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714031960 con 0x5567438744d0 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 -- 172.21.15.39:0/4222867286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438744d0 msgr2=0x556743c32230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438744d0 0x556743c32230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f471402aa00 tx=0x7f4714004310 comp rx=0 tx=0).stop 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 -- 172.21.15.39:0/4222867286 shutdown_connections 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5567438744d0 0x556743c32230 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55674386c2f0 0x556743c47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 --2- 172.21.15.39:0/4222867286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c31cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 -- 172.21.15.39:0/4222867286 >> 172.21.15.39:0/4222867286 conn(0x556743c4e2c0 msgr2=0x556743c3ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 -- 172.21.15.39:0/4222867286 shutdown_connections 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.260+0000 7f472bbf5740 1 -- 172.21.15.39:0/4222867286 wait complete. 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.261+0000 7f472bbf5740 1 -- start start 2024-05-04T03:23:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.261+0000 7f472bbf5740 1 Processor -- start 2024-05-04T03:23:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.261+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55674386c2f0 0x556743c44eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.261+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 0x556743c453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.261+0000 7f472bbf5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.261+0000 7f472bbf5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f47201aa5e0 con 0x556743889300 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.261+0000 7f472bbf5740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47201a9f10 con 0x55674386c2f0 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.261+0000 7f472bbf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47201a7150 con 0x5567438744d0 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f472bbf5740 10 client.0 _ll_register_callbacks cb 0x7f47180a9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55674386c2f0 0x556743c44eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55674386c2f0 0x556743c44eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:53608/0 (socket says 172.21.15.39:53608) 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471e59c640 1 -- 172.21.15.39:0/434810125 learned_addr learned my addr 172.21.15.39:0/434810125 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471ed9d640 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471dd9b640 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 0x556743c453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471ed9d640 1 -- 172.21.15.39:0/434810125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 msgr2=0x556743c453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471ed9d640 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 0x556743c453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471ed9d640 1 -- 172.21.15.39:0/434810125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55674386c2f0 msgr2=0x556743c44eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471ed9d640 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55674386c2f0 0x556743c44eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.262+0000 7f471ed9d640 1 -- 172.21.15.39:0/434810125 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556743c46870 con 0x556743889300 2024-05-04T03:23:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.263+0000 7f471ed9d640 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c462b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f47140057c0 tx=0x7f4714003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30ed6ab7a2728a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.263+0000 7f4705ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:23:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.263+0000 7f4705ffb640 1 -- 172.21.15.39:0/434810125 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714035070 con 0x556743889300 2024-05-04T03:23:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.263+0000 7f4705ffb640 1 -- 172.21.15.39:0/434810125 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4714003800 con 0x556743889300 2024-05-04T03:23:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.263+0000 7f472bbf5740 1 -- 172.21.15.39:0/434810125 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4714003a80 con 0x556743889300 2024-05-04T03:23:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f4705ffb640 1 -- 172.21.15.39:0/434810125 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f47140316e0 con 0x556743889300 2024-05-04T03:23:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f472bbf5740 10 client.16779 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f472bbf5740 1 -- 172.21.15.39:0/434810125 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556743c47b00 con 0x556743889300 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f4705ffb640 1 -- 172.21.15.39:0/434810125 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4714003a80 con 0x556743889300 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f4705ffb640 1 -- 172.21.15.39:0/434810125 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4714044020 con 0x556743889300 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f472bbf5740 10 client.16779 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f472bbf5740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f472bbf5740 20 client.16779 populate_metadata read hostname 'smithi039' 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f472bbf5740 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5567438951b0 0x556743cb87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f472bbf5740 4 client.16779 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.264+0000 7f472bbf5740 1 -- 172.21.15.39:0/434810125 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556743ba3050 con 0x5567438951b0 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.265+0000 7f471e59c640 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5567438951b0 0x556743cb87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.265+0000 7f4705ffb640 10 client.16779 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.266+0000 7f471e59c640 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5567438951b0 0x556743cb87b0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d3332f70407a54 server_cookie=393edd70b77d93af in_seq=0 out_seq=0 2024-05-04T03:23:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.276+0000 7f4705ffb640 1 -- 172.21.15.39:0/434810125 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556743ba3050 con 0x5567438951b0 2024-05-04T03:23:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.276+0000 7f4705ffb640 10 client.16779 handle_command_reply: tid=0 2024-05-04T03:23:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.276+0000 7f472bbf5740 1 client.16779 shutdown 2024-05-04T03:23:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.276+0000 7f472bbf5740 1 -- 172.21.15.39:0/434810125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 msgr2=0x556743c462b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.276+0000 7f472bbf5740 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c462b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f47140057c0 tx=0x7f4714003e60 comp rx=0 tx=0).stop 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.276+0000 7f472bbf5740 1 -- 172.21.15.39:0/434810125 shutdown_connections 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.277+0000 7f472bbf5740 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5567438951b0 0x556743cb87b0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.277+0000 7f4705ffb640 0 client.16779 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.277+0000 7f472bbf5740 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556743889300 0x556743c462b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.277+0000 7f472bbf5740 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567438744d0 0x556743c453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.277+0000 7f472bbf5740 1 --2- 172.21.15.39:0/434810125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55674386c2f0 0x556743c44eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.277+0000 7f472bbf5740 1 -- 172.21.15.39:0/434810125 >> 172.21.15.39:0/434810125 conn(0x556743c4e2c0 msgr2=0x556743c31cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.278+0000 7f472bbf5740 1 -- 172.21.15.39:0/434810125 shutdown_connections 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.278+0000 7f472bbf5740 1 -- 172.21.15.39:0/434810125 wait complete. 2024-05-04T03:23:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.278+0000 7f472bbf5740 20 client.16779 trim_cache size 0 max 16384 2024-05-04T03:23:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:23:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:23:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:23:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:23:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f47080b2930 msgr2=0x7f47080b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f47080b2930 0x7f47080b4e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f472019d4b0 tx=0x7f4710005f90 comp rx=0 tx=0).stop 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4720105e70 msgr2=0x7f472019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4720105e70 0x7f472019c410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f471800eb90 tx=0x7f471800c6a0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 shutdown_connections 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f47080b2930 0x7f47080b4e40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f472010bdb0 0x7f47201a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47201070b0 0x7f472019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 --2- 172.21.15.39:0/4132124480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4720105e70 0x7f472019c410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 >> 172.21.15.39:0/4132124480 conn(0x7f4720101040 msgr2=0x7f4720107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 shutdown_connections 2024-05-04T03:23:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:23:52.279+0000 7f4724d3f640 1 -- 172.21.15.39:0/4132124480 wait complete. 2024-05-04T03:23:52.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:52 smithi167 ceph-mon[33614]: pgmap v778: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-05-04T03:23:52.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:52 smithi039 ceph-mon[29715]: pgmap v778: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-05-04T03:23:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:52 smithi066 ceph-mon[33264]: pgmap v778: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-05-04T03:23:54.679 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:54 smithi167 ceph-mon[33614]: pgmap v779: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2024-05-04T03:23:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:54 smithi039 ceph-mon[29715]: pgmap v779: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2024-05-04T03:23:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:54 smithi066 ceph-mon[33264]: pgmap v779: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2024-05-04T03:23:56.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:56 smithi167 ceph-mon[33614]: pgmap v780: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-04T03:23:56.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:56 smithi039 ceph-mon[29715]: pgmap v780: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-04T03:23:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 605.0 s, 1204.6 tps, lat 83.281 ms stddev 22.608 2024-05-04T03:23:56.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:56 smithi066 ceph-mon[33264]: pgmap v780: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-05-04T03:23:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:23:58 smithi167 ceph-mon[33614]: pgmap v781: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-05-04T03:23:58.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:23:58 smithi039 ceph-mon[29715]: pgmap v781: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-05-04T03:23:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:23:58 smithi066 ceph-mon[33264]: pgmap v781: 97 pgs: 97 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2024-05-04T03:24:00.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:00 smithi167 ceph-mon[33614]: pgmap v782: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-05-04T03:24:00.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:00 smithi039 ceph-mon[29715]: pgmap v782: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-05-04T03:24:00.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:00 smithi066 ceph-mon[33264]: pgmap v782: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-05-04T03:24:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 610.0 s, 1255.8 tps, lat 79.550 ms stddev 20.546 2024-05-04T03:24:02.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:02 smithi167 ceph-mon[33614]: pgmap v783: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-04T03:24:02.691 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:02 smithi039 ceph-mon[29715]: pgmap v783: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-04T03:24:02.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:02 smithi066 ceph-mon[33264]: pgmap v783: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-04T03:24:04.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:04 smithi167 ceph-mon[33614]: pgmap v784: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-05-04T03:24:04.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:04 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:24:04.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:04 smithi039 ceph-mon[29715]: pgmap v784: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-05-04T03:24:04.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:04 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:24:04.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:04 smithi066 ceph-mon[33264]: pgmap v784: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 1.56k op/s 2024-05-04T03:24:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:04 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4121019078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a281043c0 msgr2=0x7f2a281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4121019078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a281043c0 0x7f2a281067e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009900 tx=0x7f2a1802a3f0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4121019078 shutdown_connections 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4121019078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a281098e0 0x7f2a2810bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4121019078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a28106db0 0x7f2a28109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4121019078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a281043c0 0x7f2a281067e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4121019078 >> 172.21.15.39:0/4121019078 conn(0x7f2a280fdd40 msgr2=0x7f2a281001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4121019078 shutdown_connections 2024-05-04T03:24:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.442+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4121019078 wait complete. 2024-05-04T03:24:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.443+0000 7f2a2dfa5640 1 Processor -- start 2024-05-04T03:24:05.446 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.444+0000 7f2a2dfa5640 1 -- start start 2024-05-04T03:24:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.445+0000 7f2a2dfa5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a281043c0 0x7f2a2819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.445+0000 7f2a2dfa5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a28106db0 0x7f2a2819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.445+0000 7f2a2dfa5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a281098e0 0x7f2a281a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.446+0000 7f2a2dfa5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2810f5d0 con 0x7f2a281043c0 2024-05-04T03:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.446+0000 7f2a2dfa5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2810f450 con 0x7f2a28106db0 2024-05-04T03:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.446+0000 7f2a2dfa5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2810f750 con 0x7f2a281098e0 2024-05-04T03:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.446+0000 7f2a26ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a28106db0 0x7f2a2819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.446+0000 7f2a26ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a28106db0 0x7f2a2819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:58242/0 (socket says 172.21.15.39:58242) 2024-05-04T03:24:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.446+0000 7f2a26ffd640 1 -- 172.21.15.39:0/4112069367 learned_addr learned my addr 172.21.15.39:0/4112069367 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.446+0000 7f2a27fff640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a281098e0 0x7f2a281a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.447+0000 7f2a277fe640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a281043c0 0x7f2a2819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a26ffd640 1 -- 172.21.15.39:0/4112069367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a281098e0 msgr2=0x7f2a281a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a281098e0 0x7f2a281a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a26ffd640 1 -- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a281043c0 msgr2=0x7f2a2819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a281043c0 0x7f2a2819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a26ffd640 1 -- 172.21.15.39:0/4112069367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a281a1da0 con 0x7f2a28106db0 2024-05-04T03:24:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a277fe640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a281043c0 0x7f2a2819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:24:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a28106db0 0x7f2a2819a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00ea60 tx=0x7f2a1c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a24ff9640 1 -- 172.21.15.39:0/4112069367 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00cde0 con 0x7f2a28106db0 2024-05-04T03:24:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a24ff9640 1 -- 172.21.15.39:0/4112069367 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a1c0108c0 con 0x7f2a28106db0 2024-05-04T03:24:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.448+0000 7f2a24ff9640 1 -- 172.21.15.39:0/4112069367 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c018880 con 0x7f2a28106db0 2024-05-04T03:24:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.449+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4112069367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a281a2090 con 0x7f2a28106db0 2024-05-04T03:24:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.449+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4112069367 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a281a2530 con 0x7f2a28106db0 2024-05-04T03:24:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.452+0000 7f2a34e56740 1 Processor -- start 2024-05-04T03:24:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.452+0000 7f2a34e56740 1 -- start start 2024-05-04T03:24:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.452+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd0355a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.452+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0355ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.452+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd035cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.452+0000 7f2a34e56740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55afd02a95d0 con 0x55afcff824d0 2024-05-04T03:24:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.452+0000 7f2a34e56740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55afd0262890 con 0x55afcffa31b0 2024-05-04T03:24:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.452+0000 7f2a34e56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55afd02a5590 con 0x55afcff7a2f0 2024-05-04T03:24:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.453+0000 7f2a277fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0355ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.453+0000 7f2a277fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0355ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:35400/0 (socket says 172.21.15.39:35400) 2024-05-04T03:24:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.453+0000 7f2a277fe640 1 -- 172.21.15.39:0/1013262792 learned_addr learned my addr 172.21.15.39:0/1013262792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.453+0000 7f2a27fff640 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd0355a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.453+0000 7f2a277fe640 1 -- 172.21.15.39:0/1013262792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 msgr2=0x55afd0355a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.453+0000 7f2a277fe640 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd0355a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.454+0000 7f2a277fe640 1 -- 172.21.15.39:0/1013262792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 msgr2=0x55afd035cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.454+0000 7f2a277fe640 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd035cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.454+0000 7f2a277fe640 1 -- 172.21.15.39:0/1013262792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afcffbb210 con 0x55afcff7a2f0 2024-05-04T03:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.454+0000 7f2a27fff640 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd0355a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.454+0000 7f2a277fe640 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0355ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2a18000c00 tx=0x7f2a18005330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f34d8c4703594bd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.455+0000 7f2a0dffb640 1 -- 172.21.15.39:0/1013262792 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18035020 con 0x55afcff7a2f0 2024-05-04T03:24:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.455+0000 7f2a0dffb640 1 -- 172.21.15.39:0/1013262792 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a180044d0 con 0x55afcff7a2f0 2024-05-04T03:24:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.455+0000 7f2a24ff9640 1 -- 172.21.15.39:0/4112069367 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2a1c010430 con 0x7f2a28106db0 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.455+0000 7f2a0dffb640 1 -- 172.21.15.39:0/1013262792 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18004e80 con 0x55afcff7a2f0 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a24ff9640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a100b29b0 0x7f2a100b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a24ff9640 1 -- 172.21.15.39:0/4112069367 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2a1c0e4720 con 0x7f2a28106db0 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a34e56740 1 -- 172.21.15.39:0/1013262792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 msgr2=0x55afd0355ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0355ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2a18000c00 tx=0x7f2a18005330 comp rx=0 tx=0).stop 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a27fff640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a100b29b0 0x7f2a100b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a34e56740 1 -- 172.21.15.39:0/1013262792 shutdown_connections 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd035cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0355ff0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1013262792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd0355a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a34e56740 1 -- 172.21.15.39:0/1013262792 >> 172.21.15.39:0/1013262792 conn(0x55afcff97300 msgr2=0x55afd03403a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.456+0000 7f2a34e56740 1 -- 172.21.15.39:0/1013262792 shutdown_connections 2024-05-04T03:24:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.457+0000 7f2a34e56740 1 -- 172.21.15.39:0/1013262792 wait complete. 2024-05-04T03:24:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.457+0000 7f2a27fff640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a100b29b0 0x7f2a100b4ec0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009d10 tx=0x7f2a200096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.457+0000 7f2a34e56740 1 -- start start 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.457+0000 7f2a34e56740 1 Processor -- start 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcff7a2f0 0x55afd035f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd035fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcffa31b0 0x55afd0366bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a34e56740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55afd035d980 con 0x55afcff824d0 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a34e56740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55afd0266370 con 0x55afcff7a2f0 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a34e56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55afd035db00 con 0x55afcffa31b0 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a277fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcff7a2f0 0x55afd035f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a34e56740 10 client.0 _ll_register_callbacks cb 0x55afcffbaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a277fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcff7a2f0 0x55afd035f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:58266/0 (socket says 172.21.15.39:58266) 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a277fe640 1 -- 172.21.15.39:0/1897875193 learned_addr learned my addr 172.21.15.39:0/1897875193 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd035fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.458+0000 7f2a27fff640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcffa31b0 0x55afd0366bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a277fe640 1 -- 172.21.15.39:0/1897875193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcffa31b0 msgr2=0x55afd0366bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a277fe640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcffa31b0 0x55afd0366bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a277fe640 1 -- 172.21.15.39:0/1897875193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 msgr2=0x55afd035fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a277fe640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd035fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a277fe640 1 -- 172.21.15.39:0/1897875193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd0087620 con 0x55afcff7a2f0 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd035fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a27fff640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcffa31b0 0x55afd0366bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a277fe640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcff7a2f0 0x55afd035f750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a1802a430 tx=0x7f2a180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5eee6e4573979f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.459+0000 7f2a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.460+0000 7f2a06ffd640 1 -- 172.21.15.39:0/1897875193 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18035070 con 0x55afcff7a2f0 2024-05-04T03:24:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.460+0000 7f2a06ffd640 1 -- 172.21.15.39:0/1897875193 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2a18004c00 con 0x55afcff7a2f0 2024-05-04T03:24:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.460+0000 7f2a34e56740 1 -- 172.21.15.39:0/1897875193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a180044d0 con 0x55afcff7a2f0 2024-05-04T03:24:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.460+0000 7f2a06ffd640 1 -- 172.21.15.39:0/1897875193 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2a18004e00 con 0x55afcff7a2f0 2024-05-04T03:24:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.460+0000 7f2a34e56740 10 client.26360 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.460+0000 7f2a34e56740 1 -- 172.21.15.39:0/1897875193 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55afd036df30 con 0x55afcff7a2f0 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.460+0000 7f2a06ffd640 1 -- 172.21.15.39:0/1897875193 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a180044d0 con 0x55afcff7a2f0 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a06ffd640 1 -- 172.21.15.39:0/1897875193 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2a1803a040 con 0x55afcff7a2f0 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a34e56740 10 client.26360 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a34e56740 10 client.26360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a34e56740 20 client.26360 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55afcff61d80 0x55afd0371290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a34e56740 4 client.26360 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a34e56740 1 -- 172.21.15.39:0/1897875193 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afd02b71e0 con 0x55afcff61d80 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55afcff61d80 0x55afd0371290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.461+0000 7f2a06ffd640 10 client.26360 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.462+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55afcff61d80 0x55afd0371290 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d2d4583b9d73ce server_cookie=fad198de41d3df3b in_seq=0 out_seq=0 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a06ffd640 1 -- 172.21.15.39:0/1897875193 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55afd02b71e0 con 0x55afcff61d80 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a06ffd640 10 client.26360 handle_command_reply: tid=0 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 client.26360 shutdown 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 -- 172.21.15.39:0/1897875193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcff7a2f0 msgr2=0x55afd035f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcff7a2f0 0x55afd035f750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a1802a430 tx=0x7f2a180046c0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 -- 172.21.15.39:0/1897875193 shutdown_connections 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55afcff61d80 0x55afd0371290 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcffa31b0 0x55afd0366bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd035fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 --2- 172.21.15.39:0/1897875193 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcff7a2f0 0x55afd035f750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a34e56740 1 -- 172.21.15.39:0/1897875193 >> 172.21.15.39:0/1897875193 conn(0x55afcff97300 msgr2=0x55afd0343300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.463+0000 7f2a06ffd640 0 client.26360 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.464+0000 7f2a34e56740 1 -- 172.21.15.39:0/1897875193 shutdown_connections 2024-05-04T03:24:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.464+0000 7f2a34e56740 1 -- 172.21.15.39:0/1897875193 wait complete. 2024-05-04T03:24:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.464+0000 7f2a34e56740 20 client.26360 trim_cache size 0 max 16384 2024-05-04T03:24:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.491+0000 7f2a34e56740 1 Processor -- start 2024-05-04T03:24:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.491+0000 7f2a34e56740 1 -- start start 2024-05-04T03:24:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.491+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0340470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd0366b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd035f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a34e56740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a280730a0 con 0x55afcff824d0 2024-05-04T03:24:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a34e56740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28074be0 con 0x55afcffa31b0 2024-05-04T03:24:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a34e56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28073770 con 0x55afcff7a2f0 2024-05-04T03:24:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a27fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd035f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a27fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd035f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:58274/0 (socket says 172.21.15.39:58274) 2024-05-04T03:24:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a27fff640 1 -- 172.21.15.39:0/2696355187 learned_addr learned my addr 172.21.15.39:0/2696355187 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.492+0000 7f2a277fe640 1 --2- 172.21.15.39:0/2696355187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0340470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a27fff640 1 -- 172.21.15.39:0/2696355187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 msgr2=0x55afd0340470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a27fff640 1 --2- 172.21.15.39:0/2696355187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0340470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a27fff640 1 -- 172.21.15.39:0/2696355187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 msgr2=0x55afd0366b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a27fff640 1 --2- 172.21.15.39:0/2696355187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd0366b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a27fff640 1 -- 172.21.15.39:0/2696355187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afcff61fb0 con 0x55afcffa31b0 2024-05-04T03:24:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a27fff640 1 --2- 172.21.15.39:0/2696355187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd035f320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a20020790 tx=0x7f2a20016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea750899a966496 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a0d7fa640 1 -- 172.21.15.39:0/2696355187 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2002a420 con 0x55afcffa31b0 2024-05-04T03:24:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a0d7fa640 1 -- 172.21.15.39:0/2696355187 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a2002aa50 con 0x55afcffa31b0 2024-05-04T03:24:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.493+0000 7f2a0d7fa640 1 -- 172.21.15.39:0/2696355187 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2001d880 con 0x55afcffa31b0 2024-05-04T03:24:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.495+0000 7f2a34e56740 1 -- 172.21.15.39:0/2696355187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 msgr2=0x55afd035f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.495+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2696355187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd035f320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a20020790 tx=0x7f2a20016000 comp rx=0 tx=0).stop 2024-05-04T03:24:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.495+0000 7f2a34e56740 1 -- 172.21.15.39:0/2696355187 shutdown_connections 2024-05-04T03:24:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.495+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2696355187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd035f320 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.495+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2696355187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff824d0 0x55afd0366b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.495+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2696355187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff7a2f0 0x55afd0340470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.495+0000 7f2a34e56740 1 -- 172.21.15.39:0/2696355187 >> 172.21.15.39:0/2696355187 conn(0x55afcff97300 msgr2=0x55afd033fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.495+0000 7f2a34e56740 1 -- 172.21.15.39:0/2696355187 shutdown_connections 2024-05-04T03:24:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.496+0000 7f2a34e56740 1 -- 172.21.15.39:0/2696355187 wait complete. 2024-05-04T03:24:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.496+0000 7f2a34e56740 1 -- start start 2024-05-04T03:24:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.497+0000 7f2a34e56740 1 Processor -- start 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.497+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff7a2f0 0x55afd0363780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.497+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff824d0 0x55afd0363cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.497+0000 7f2a34e56740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd03d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.497+0000 7f2a34e56740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28074510 con 0x55afcff7a2f0 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.497+0000 7f2a34e56740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28073e40 con 0x55afcffa31b0 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.497+0000 7f2a34e56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28071080 con 0x55afcff824d0 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.497+0000 7f2a34e56740 10 client.0 _ll_register_callbacks cb 0x7f2a20009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a277fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff7a2f0 0x55afd0363780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a27fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd03d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a27fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd03d7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:58278/0 (socket says 172.21.15.39:58278) 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a27fff640 1 -- 172.21.15.39:0/2725867241 learned_addr learned my addr 172.21.15.39:0/2725867241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff824d0 0x55afd0363cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a277fe640 1 -- 172.21.15.39:0/2725867241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff824d0 msgr2=0x55afd0363cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a277fe640 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff824d0 0x55afd0363cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a277fe640 1 -- 172.21.15.39:0/2725867241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 msgr2=0x55afd03d7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a277fe640 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd03d7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a277fe640 1 -- 172.21.15.39:0/2725867241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd0366c80 con 0x55afcff7a2f0 2024-05-04T03:24:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.498+0000 7f2a27fff640 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd03d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.499+0000 7f2a277fe640 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff7a2f0 0x55afd0363780 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2a1802a910 tx=0x7f2a18005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f60e992a98b25ed server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.499+0000 7f2a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.499+0000 7f2a06ffd640 1 -- 172.21.15.39:0/2725867241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18035040 con 0x55afcff7a2f0 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.499+0000 7f2a06ffd640 1 -- 172.21.15.39:0/2725867241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2a1804b040 con 0x55afcff7a2f0 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.500+0000 7f2a34e56740 1 -- 172.21.15.39:0/2725867241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a2002aa50 con 0x55afcff7a2f0 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.500+0000 7f2a06ffd640 1 -- 172.21.15.39:0/2725867241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2a1803d6a0 con 0x55afcff7a2f0 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.500+0000 7f2a34e56740 10 client.16791 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.500+0000 7f2a34e56740 1 -- 172.21.15.39:0/2725867241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55afd03d87e0 con 0x55afcff7a2f0 2024-05-04T03:24:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.500+0000 7f2a06ffd640 1 -- 172.21.15.39:0/2725867241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a2002aa50 con 0x55afcff7a2f0 2024-05-04T03:24:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a06ffd640 1 -- 172.21.15.39:0/2725867241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2a18031a20 con 0x55afcff7a2f0 2024-05-04T03:24:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a34e56740 10 client.16791 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a34e56740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a34e56740 20 client.16791 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55afcff371c0 0x55afd03db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a34e56740 4 client.16791 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a34e56740 1 -- 172.21.15.39:0/2725867241 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afd02b1050 con 0x55afcff371c0 2024-05-04T03:24:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55afcff371c0 0x55afd03db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.501+0000 7f2a06ffd640 10 client.16791 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.502+0000 7f2a26ffd640 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55afcff371c0 0x55afd03db220 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a81ffa7ef719e3 server_cookie=36e4e64a80036ca4 in_seq=0 out_seq=0 2024-05-04T03:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.511+0000 7f2a06ffd640 1 -- 172.21.15.39:0/2725867241 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afd02b1050 con 0x55afcff371c0 2024-05-04T03:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.511+0000 7f2a06ffd640 10 client.16791 handle_command_reply: tid=0 2024-05-04T03:24:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.511+0000 7f2a34e56740 1 client.16791 shutdown 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 -- 172.21.15.39:0/2725867241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff7a2f0 msgr2=0x55afd0363780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff7a2f0 0x55afd0363780 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2a1802a910 tx=0x7f2a18005930 comp rx=0 tx=0).stop 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 -- 172.21.15.39:0/2725867241 shutdown_connections 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55afcff371c0 0x55afd03db220 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55afcffa31b0 0x55afd03d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55afcff824d0 0x55afd0363cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a06ffd640 0 client.16791 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 --2- 172.21.15.39:0/2725867241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55afcff7a2f0 0x55afd0363780 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 -- 172.21.15.39:0/2725867241 >> 172.21.15.39:0/2725867241 conn(0x55afcff97300 msgr2=0x55afd0343270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 -- 172.21.15.39:0/2725867241 shutdown_connections 2024-05-04T03:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.512+0000 7f2a34e56740 1 -- 172.21.15.39:0/2725867241 wait complete. 2024-05-04T03:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.513+0000 7f2a34e56740 20 client.16791 trim_cache size 0 max 16384 2024-05-04T03:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:24:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:24:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:24:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:24:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a100b29b0 msgr2=0x7f2a100b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a100b29b0 0x7f2a100b4ec0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009d10 tx=0x7f2a200096a0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4112069367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a28106db0 msgr2=0x7f2a2819a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a28106db0 0x7f2a2819a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00ea60 tx=0x7f2a1c00ef20 comp rx=0 tx=0).stop 2024-05-04T03:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4112069367 shutdown_connections 2024-05-04T03:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f2a100b29b0 0x7f2a100b4ec0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a281098e0 0x7f2a281a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a28106db0 0x7f2a2819a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 --2- 172.21.15.39:0/4112069367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a281043c0 0x7f2a2819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4112069367 >> 172.21.15.39:0/4112069367 conn(0x7f2a280fdd40 msgr2=0x7f2a28105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4112069367 shutdown_connections 2024-05-04T03:24:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:05.514+0000 7f2a2dfa5640 1 -- 172.21.15.39:0/4112069367 wait complete. 2024-05-04T03:24:05.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:24:05.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:24:05.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:24:05.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:24:05.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:24:05.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:24:05.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:24:05.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:24:05.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:24:06.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:06 smithi167 ceph-mon[33614]: pgmap v785: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:06 smithi039 ceph-mon[29715]: pgmap v785: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 615.0 s, 1091.4 tps, lat 91.566 ms stddev 151.512 2024-05-04T03:24:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:06 smithi066 ceph-mon[33264]: pgmap v785: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:08.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:08 smithi167 ceph-mon[33614]: pgmap v786: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:08.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:08 smithi039 ceph-mon[29715]: pgmap v786: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:08.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:08 smithi066 ceph-mon[33264]: pgmap v786: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:10.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:10 smithi167 ceph-mon[33614]: pgmap v787: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 1.59k op/s 2024-05-04T03:24:10.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:10 smithi039 ceph-mon[29715]: pgmap v787: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 1.59k op/s 2024-05-04T03:24:10.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:10 smithi066 ceph-mon[33264]: pgmap v787: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 1.59k op/s 2024-05-04T03:24:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 620.0 s, 1228.8 tps, lat 81.364 ms stddev 24.266 2024-05-04T03:24:12.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:12 smithi167 ceph-mon[33614]: pgmap v788: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-05-04T03:24:12.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:12 smithi039 ceph-mon[29715]: pgmap v788: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-05-04T03:24:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:12 smithi066 ceph-mon[33264]: pgmap v788: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-05-04T03:24:14.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:14 smithi039 ceph-mon[29715]: pgmap v789: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.58k op/s 2024-05-04T03:24:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:14 smithi066 ceph-mon[33264]: pgmap v789: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.58k op/s 2024-05-04T03:24:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:14 smithi167 ceph-mon[33614]: pgmap v789: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 1.58k op/s 2024-05-04T03:24:16.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:16 smithi039 ceph-mon[29715]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:16.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 625.0 s, 1172.6 tps, lat 85.149 ms stddev 26.176 2024-05-04T03:24:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:16 smithi066 ceph-mon[33264]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:16.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:16 smithi167 ceph-mon[33614]: pgmap v790: 97 pgs: 97 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.674+0000 7fa2a2ffd640 1 -- 172.21.15.39:0/207719277 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c030c60 con 0x7fa2a4077a40 2024-05-04T03:24:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 -- 172.21.15.39:0/207719277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a4077a40 msgr2=0x7fa2a4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/207719277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a4077a40 0x7fa2a4078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009be0 tx=0x7fa29c02a6f0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 -- 172.21.15.39:0/207719277 shutdown_connections 2024-05-04T03:24:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/207719277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a40799d0 0x7fa2a4079de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/207719277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a4078cd0 0x7fa2a40790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/207719277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a4077a40 0x7fa2a4078790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 -- 172.21.15.39:0/207719277 >> 172.21.15.39:0/207719277 conn(0x7fa2a4101060 msgr2=0x7fa2a4103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 -- 172.21.15.39:0/207719277 shutdown_connections 2024-05-04T03:24:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.675+0000 7fa2aa46e640 1 -- 172.21.15.39:0/207719277 wait complete. 2024-05-04T03:24:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.676+0000 7fa2aa46e640 1 Processor -- start 2024-05-04T03:24:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.676+0000 7fa2aa46e640 1 -- start start 2024-05-04T03:24:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2aa46e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a4077a40 0x7fa2a41a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2aa46e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a4078cd0 0x7fa2a41a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2aa46e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40799d0 0x7fa2a41a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2aa46e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4115cb0 con 0x7fa2a4078cd0 2024-05-04T03:24:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2aa46e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4115b30 con 0x7fa2a40799d0 2024-05-04T03:24:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2aa46e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4115e30 con 0x7fa2a4077a40 2024-05-04T03:24:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a8a0b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40799d0 0x7fa2a41a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a4077a40 0x7fa2a41a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a8a0b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40799d0 0x7fa2a41a7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48536/0 (socket says 172.21.15.39:48536) 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a8a0b640 1 -- 172.21.15.39:0/908936330 learned_addr learned my addr 172.21.15.39:0/908936330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a8a0b640 1 -- 172.21.15.39:0/908936330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a4077a40 msgr2=0x7fa2a41a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a8a0b640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a4077a40 0x7fa2a41a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a8a0b640 1 -- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a4078cd0 msgr2=0x7fa2a41a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a8a0b640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a4078cd0 0x7fa2a41a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.677+0000 7fa2a8a0b640 1 -- 172.21.15.39:0/908936330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a41a8480 con 0x7fa2a40799d0 2024-05-04T03:24:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.678+0000 7fa2a8a0b640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40799d0 0x7fa2a41a7e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa29800b3f0 tx=0x7fa29800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.678+0000 7fa2a17fa640 1 -- 172.21.15.39:0/908936330 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298015020 con 0x7fa2a40799d0 2024-05-04T03:24:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.678+0000 7fa2a17fa640 1 -- 172.21.15.39:0/908936330 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa298011cc0 con 0x7fa2a40799d0 2024-05-04T03:24:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.678+0000 7fa2aa46e640 1 -- 172.21.15.39:0/908936330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a41a8770 con 0x7fa2a40799d0 2024-05-04T03:24:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.678+0000 7fa2a17fa640 1 -- 172.21.15.39:0/908936330 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298010ce0 con 0x7fa2a40799d0 2024-05-04T03:24:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.678+0000 7fa2aa46e640 1 -- 172.21.15.39:0/908936330 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a41a8cc0 con 0x7fa2a40799d0 2024-05-04T03:24:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.682+0000 7fa2b1400740 1 Processor -- start 2024-05-04T03:24:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.683+0000 7fa2a17fa640 1 -- 172.21.15.39:0/908936330 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa298018020 con 0x7fa2a40799d0 2024-05-04T03:24:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.683+0000 7fa2b1400740 1 -- start start 2024-05-04T03:24:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.683+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4691b0 0x55988d80aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.683+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 0x55988d80afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.683+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d811ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.683+0000 7fa2b1400740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55988d76f5d0 con 0x55988d4402f0 2024-05-04T03:24:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.683+0000 7fa2b1400740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55988d728890 con 0x55988d4484d0 2024-05-04T03:24:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.683+0000 7fa2b1400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55988d76b590 con 0x55988d4691b0 2024-05-04T03:24:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.684+0000 7fa2a17fa640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa28c0b28e0 0x7fa28c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4691b0 0x55988d80aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4691b0 0x55988d80aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53708/0 (socket says 172.21.15.39:53708) 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 -- 172.21.15.39:0/1629177134 learned_addr learned my addr 172.21.15.39:0/1629177134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d811ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a3fff640 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 0x55988d80afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a17fa640 1 -- 172.21.15.39:0/908936330 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa2980e1b40 con 0x7fa2a40799d0 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 -- 172.21.15.39:0/1629177134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 msgr2=0x55988d811ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d811ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 -- 172.21.15.39:0/1629177134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 msgr2=0x55988d80afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 0x55988d80afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 -- 172.21.15.39:0/1629177134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988d481210 con 0x55988d4691b0 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a8a0b640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa28c0b28e0 0x7fa28c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.685+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4691b0 0x55988d80aa60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa294013820 tx=0x7fa294009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=224698cd8899fa5c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.686+0000 7fa28a7fc640 1 -- 172.21.15.39:0/1629177134 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294013e60 con 0x55988d4691b0 2024-05-04T03:24:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.686+0000 7fa28a7fc640 1 -- 172.21.15.39:0/1629177134 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2940044c0 con 0x55988d4691b0 2024-05-04T03:24:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.686+0000 7fa28a7fc640 1 -- 172.21.15.39:0/1629177134 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29400edd0 con 0x55988d4691b0 2024-05-04T03:24:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.687+0000 7fa2b1400740 1 -- 172.21.15.39:0/1629177134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4691b0 msgr2=0x55988d80aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.687+0000 7fa2a8a0b640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa28c0b28e0 0x7fa28c0b4df0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2980a90a0 tx=0x7fa2980046e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.687+0000 7fa2b1400740 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4691b0 0x55988d80aa60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa294013820 tx=0x7fa294009660 comp rx=0 tx=0).stop 2024-05-04T03:24:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.687+0000 7fa2b1400740 1 -- 172.21.15.39:0/1629177134 shutdown_connections 2024-05-04T03:24:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.687+0000 7fa2b1400740 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d811ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.687+0000 7fa2b1400740 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 0x55988d80afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.687+0000 7fa2b1400740 1 --2- 172.21.15.39:0/1629177134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4691b0 0x55988d80aa60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.687+0000 7fa2b1400740 1 -- 172.21.15.39:0/1629177134 >> 172.21.15.39:0/1629177134 conn(0x55988d45d300 msgr2=0x55988d8063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.688+0000 7fa2b1400740 1 -- 172.21.15.39:0/1629177134 shutdown_connections 2024-05-04T03:24:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.688+0000 7fa2b1400740 1 -- 172.21.15.39:0/1629177134 wait complete. 2024-05-04T03:24:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.688+0000 7fa2b1400740 1 -- start start 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.688+0000 7fa2b1400740 1 Processor -- start 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 0x55988d814610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d814b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 0x55988d81bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2b1400740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55988d8128d0 con 0x55988d4691b0 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d814b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d814b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48554/0 (socket says 172.21.15.39:48554) 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2a3fff640 1 -- 172.21.15.39:0/2310296887 learned_addr learned my addr 172.21.15.39:0/2310296887 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55988d72c370 con 0x55988d4484d0 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 0x55988d814610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55988d812a50 con 0x55988d4402f0 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 0x55988d81bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.689+0000 7fa2b1400740 10 client.0 _ll_register_callbacks cb 0x55988d480a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2a3fff640 1 -- 172.21.15.39:0/2310296887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 msgr2=0x55988d814610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2a3fff640 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 0x55988d814610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2a3fff640 1 -- 172.21.15.39:0/2310296887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 msgr2=0x55988d81bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2a3fff640 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 0x55988d81bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2a3fff640 1 -- 172.21.15.39:0/2310296887 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988d54d620 con 0x55988d4484d0 2024-05-04T03:24:18.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:18 smithi039 ceph-mon[29715]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2a3fff640 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d814b50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa29c000f80 tx=0x7fa29c030b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5560baab2f0d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2837fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2837fe640 1 -- 172.21.15.39:0/2310296887 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c043030 con 0x55988d4484d0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2837fe640 1 -- 172.21.15.39:0/2310296887 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa29c003b80 con 0x55988d4484d0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.690+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55988d77d1e0 con 0x55988d4484d0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.691+0000 7fa2837fe640 1 -- 172.21.15.39:0/2310296887 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa29c036040 con 0x55988d4484d0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.691+0000 7fa2b1400740 10 client.26378 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.691+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55988d81e7e0 con 0x55988d4484d0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.691+0000 7fa2837fe640 1 -- 172.21.15.39:0/2310296887 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55988d77d1e0 con 0x55988d4484d0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.691+0000 7fa2837fe640 1 -- 172.21.15.39:0/2310296887 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa29c03c050 con 0x55988d4484d0 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.691+0000 7fa2b1400740 10 client.26378 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.691+0000 7fa2b1400740 10 client.26378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.692+0000 7fa2b1400740 20 client.26378 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.692+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55988d427d80 0x55988d822370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.692+0000 7fa2b1400740 4 client.26378 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.692+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55988d777050 con 0x55988d427d80 2024-05-04T03:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.692+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55988d427d80 0x55988d822370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.692+0000 7fa2837fe640 10 client.26378 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.693+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55988d427d80 0x55988d822370 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179f81a573738439 server_cookie=2c34e218d6fcd6df in_seq=0 out_seq=0 2024-05-04T03:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.694+0000 7fa2837fe640 1 -- 172.21.15.39:0/2310296887 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55988d777050 con 0x55988d427d80 2024-05-04T03:24:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.694+0000 7fa2837fe640 10 client.26378 handle_command_reply: tid=0 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.694+0000 7fa2b1400740 1 client.26378 shutdown 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.694+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 msgr2=0x55988d814b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.694+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d814b50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa29c000f80 tx=0x7fa29c030b90 comp rx=0 tx=0).stop 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 shutdown_connections 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55988d427d80 0x55988d822370 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 0x55988d81bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d814b50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2837fe640 0 client.26378 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2310296887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 0x55988d814610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 >> 172.21.15.39:0/2310296887 conn(0x55988d45d300 msgr2=0x55988d809300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 shutdown_connections 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 1 -- 172.21.15.39:0/2310296887 wait complete. 2024-05-04T03:24:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.695+0000 7fa2b1400740 20 client.26378 trim_cache size 0 max 16384 2024-05-04T03:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.728+0000 7fa2b1400740 1 Processor -- start 2024-05-04T03:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.728+0000 7fa2b1400740 1 -- start start 2024-05-04T03:24:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.729+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 0x55988d813500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.729+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 0x55988d813500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.729+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4484d0 0x55988d827b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.730+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4691b0 0x55988d8280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.730+0000 7fa2b1400740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41b19b0 con 0x55988d4402f0 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.730+0000 7fa2b1400740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41b34f0 con 0x55988d4691b0 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.730+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4691b0 0x55988d8280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.730+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4691b0 0x55988d8280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48560/0 (socket says 172.21.15.39:48560) 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.730+0000 7fa2a37fe640 1 -- 172.21.15.39:0/716695701 learned_addr learned my addr 172.21.15.39:0/716695701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:18.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.730+0000 7fa2b1400740 1 -- 172.21.15.39:0/716695701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41b2080 con 0x55988d4484d0 2024-05-04T03:24:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.730+0000 7fa2a3fff640 1 --2- 172.21.15.39:0/716695701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4484d0 0x55988d827b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.732+0000 7fa2a37fe640 1 -- 172.21.15.39:0/716695701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4484d0 msgr2=0x55988d827b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.732+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/716695701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4484d0 0x55988d827b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.732+0000 7fa2a37fe640 1 -- 172.21.15.39:0/716695701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 msgr2=0x55988d813500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.732+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/716695701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 0x55988d813500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.732+0000 7fa2a37fe640 1 -- 172.21.15.39:0/716695701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988d812220 con 0x55988d4691b0 2024-05-04T03:24:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.733+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/716695701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4691b0 0x55988d8280b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa294009d70 tx=0x7fa294022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef02c5be02012a39 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.733+0000 7fa289ffb640 1 -- 172.21.15.39:0/716695701 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294013150 con 0x55988d4691b0 2024-05-04T03:24:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.734+0000 7fa289ffb640 1 -- 172.21.15.39:0/716695701 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2940132f0 con 0x55988d4691b0 2024-05-04T03:24:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.734+0000 7fa289ffb640 1 -- 172.21.15.39:0/716695701 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29402c980 con 0x55988d4691b0 2024-05-04T03:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.735+0000 7fa2b1400740 1 -- 172.21.15.39:0/716695701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4691b0 msgr2=0x55988d8280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.735+0000 7fa2b1400740 1 --2- 172.21.15.39:0/716695701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4691b0 0x55988d8280b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa294009d70 tx=0x7fa294022040 comp rx=0 tx=0).stop 2024-05-04T03:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.735+0000 7fa2b1400740 1 -- 172.21.15.39:0/716695701 shutdown_connections 2024-05-04T03:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.735+0000 7fa2b1400740 1 --2- 172.21.15.39:0/716695701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4691b0 0x55988d8280b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.735+0000 7fa2b1400740 1 --2- 172.21.15.39:0/716695701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4484d0 0x55988d827b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.735+0000 7fa2b1400740 1 --2- 172.21.15.39:0/716695701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4402f0 0x55988d813500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.735+0000 7fa2b1400740 1 -- 172.21.15.39:0/716695701 >> 172.21.15.39:0/716695701 conn(0x55988d45d300 msgr2=0x55988d812df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.736+0000 7fa2b1400740 1 -- 172.21.15.39:0/716695701 shutdown_connections 2024-05-04T03:24:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.736+0000 7fa2b1400740 1 -- 172.21.15.39:0/716695701 wait complete. 2024-05-04T03:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.737+0000 7fa2b1400740 1 -- start start 2024-05-04T03:24:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.737+0000 7fa2b1400740 1 Processor -- start 2024-05-04T03:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 0x55988d81ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d81b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2b1400740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 0x55988d81b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 0x55988d81ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2b1400740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41b2e20 con 0x55988d4691b0 2024-05-04T03:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2b1400740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41b2750 con 0x55988d4484d0 2024-05-04T03:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2b1400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a41af990 con 0x55988d4402f0 2024-05-04T03:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d81b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2b1400740 10 client.0 _ll_register_callbacks cb 0x7fa29c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.738+0000 7fa2a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d81b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:48566/0 (socket says 172.21.15.39:48566) 2024-05-04T03:24:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2a3fff640 1 -- 172.21.15.39:0/2726128052 learned_addr learned my addr 172.21.15.39:0/2726128052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2a3fff640 1 -- 172.21.15.39:0/2726128052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 msgr2=0x55988d81ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2a3fff640 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 0x55988d81ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2a3fff640 1 -- 172.21.15.39:0/2726128052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 msgr2=0x55988d81b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2a3fff640 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 0x55988d81b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2a3fff640 1 -- 172.21.15.39:0/2726128052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988d888890 con 0x55988d4484d0 2024-05-04T03:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2a3fff640 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d81b260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa29c031040 tx=0x7fa29c0307c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae8b3fcfb613aa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2837fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:24:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.739+0000 7fa2837fe640 1 -- 172.21.15.39:0/2726128052 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c043040 con 0x55988d4484d0 2024-05-04T03:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.740+0000 7fa2b1400740 1 -- 172.21.15.39:0/2726128052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2940132f0 con 0x55988d4484d0 2024-05-04T03:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.740+0000 7fa2837fe640 1 -- 172.21.15.39:0/2726128052 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa29c031c80 con 0x55988d4484d0 2024-05-04T03:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.740+0000 7fa2837fe640 1 -- 172.21.15.39:0/2726128052 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa29c03f480 con 0x55988d4484d0 2024-05-04T03:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.740+0000 7fa2b1400740 10 client.26384 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.740+0000 7fa2b1400740 1 -- 172.21.15.39:0/2726128052 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55988d889ba0 con 0x55988d4484d0 2024-05-04T03:24:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.740+0000 7fa2837fe640 1 -- 172.21.15.39:0/2726128052 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2940132f0 con 0x55988d4484d0 2024-05-04T03:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.740+0000 7fa2837fe640 1 -- 172.21.15.39:0/2726128052 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa29c03c040 con 0x55988d4484d0 2024-05-04T03:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.741+0000 7fa2b1400740 10 client.26384 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.741+0000 7fa2b1400740 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.741+0000 7fa2b1400740 20 client.26384 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.741+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55988d3fd1c0 0x55988d88c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.741+0000 7fa2b1400740 4 client.26384 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.741+0000 7fa2b1400740 1 -- 172.21.15.39:0/2726128052 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55988d6f25f0 con 0x55988d3fd1c0 2024-05-04T03:24:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.741+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55988d3fd1c0 0x55988d88c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.742+0000 7fa2837fe640 10 client.26384 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.742+0000 7fa2a37fe640 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55988d3fd1c0 0x55988d88c5b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7d63a2243d118e server_cookie=2ba38ea94f4b7bc in_seq=0 out_seq=0 2024-05-04T03:24:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.753+0000 7fa2837fe640 1 -- 172.21.15.39:0/2726128052 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55988d6f25f0 con 0x55988d3fd1c0 2024-05-04T03:24:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.753+0000 7fa2837fe640 10 client.26384 handle_command_reply: tid=0 2024-05-04T03:24:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 client.26384 shutdown 2024-05-04T03:24:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 -- 172.21.15.39:0/2726128052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 msgr2=0x55988d81b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d81b260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa29c031040 tx=0x7fa29c0307c0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 -- 172.21.15.39:0/2726128052 shutdown_connections 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55988d3fd1c0 0x55988d88c5b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55988d4691b0 0x55988d81b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55988d4484d0 0x55988d81b260 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2837fe640 0 client.26384 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 --2- 172.21.15.39:0/2726128052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55988d4402f0 0x55988d81ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 -- 172.21.15.39:0/2726128052 >> 172.21.15.39:0/2726128052 conn(0x55988d45d300 msgr2=0x55988d8091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.754+0000 7fa2b1400740 1 -- 172.21.15.39:0/2726128052 shutdown_connections 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.755+0000 7fa2b1400740 1 -- 172.21.15.39:0/2726128052 wait complete. 2024-05-04T03:24:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.755+0000 7fa2b1400740 20 client.26384 trim_cache size 0 max 16384 2024-05-04T03:24:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:24:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:24:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:24:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 -- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa28c0b28e0 msgr2=0x7fa28c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa28c0b28e0 0x7fa28c0b4df0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2980a90a0 tx=0x7fa2980046e0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 -- 172.21.15.39:0/908936330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40799d0 msgr2=0x7fa2a41a7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40799d0 0x7fa2a41a7e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa29800b3f0 tx=0x7fa29800b8b0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 -- 172.21.15.39:0/908936330 shutdown_connections 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa28c0b28e0 0x7fa28c0b4df0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2a40799d0 0x7fa2a41a7e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a4078cd0 0x7fa2a41a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.756+0000 7fa2aa46e640 1 --2- 172.21.15.39:0/908936330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a4077a40 0x7fa2a41a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.757+0000 7fa2aa46e640 1 -- 172.21.15.39:0/908936330 >> 172.21.15.39:0/908936330 conn(0x7fa2a4101060 msgr2=0x7fa2a410fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.757+0000 7fa2aa46e640 1 -- 172.21.15.39:0/908936330 shutdown_connections 2024-05-04T03:24:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:18.757+0000 7fa2aa46e640 1 -- 172.21.15.39:0/908936330 wait complete. 2024-05-04T03:24:18.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:18 smithi066 ceph-mon[33264]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-05-04T03:24:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:18 smithi167 ceph-mon[33614]: pgmap v791: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-05-04T03:24:20.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:20 smithi039 ceph-mon[29715]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-04T03:24:20.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:20 smithi066 ceph-mon[33264]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-04T03:24:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:20 smithi167 ceph-mon[33614]: pgmap v792: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-04T03:24:21.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 630.0 s, 1143.8 tps, lat 87.847 ms stddev 25.104 2024-05-04T03:24:22.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:22 smithi066 ceph-mon[33264]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:22.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:22 smithi167 ceph-mon[33614]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:22.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:22 smithi039 ceph-mon[29715]: pgmap v793: 97 pgs: 97 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:24.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:24 smithi167 ceph-mon[33614]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:24 smithi066 ceph-mon[33264]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:24.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:24 smithi039 ceph-mon[29715]: pgmap v794: 97 pgs: 97 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:26.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 635.0 s, 1181.6 tps, lat 84.408 ms stddev 21.544 2024-05-04T03:24:26.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:26 smithi066 ceph-mon[33264]: pgmap v795: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-05-04T03:24:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:26 smithi167 ceph-mon[33614]: pgmap v795: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-05-04T03:24:26.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:26 smithi039 ceph-mon[29715]: pgmap v795: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-05-04T03:24:28.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:28 smithi066 ceph-mon[33264]: pgmap v796: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:28.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:28 smithi167 ceph-mon[33614]: pgmap v796: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:28.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:28 smithi039 ceph-mon[29715]: pgmap v796: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 16 MiB/s wr, 1.55k op/s 2024-05-04T03:24:30.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:30 smithi066 ceph-mon[33264]: pgmap v797: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-04T03:24:30.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:30 smithi167 ceph-mon[33614]: pgmap v797: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-04T03:24:30.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:30 smithi039 ceph-mon[29715]: pgmap v797: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-04T03:24:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 640.0 s, 1214.4 tps, lat 82.103 ms stddev 28.127 2024-05-04T03:24:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.920+0000 7f266aa6e640 1 -- 172.21.15.39:0/1724880378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664105e90 msgr2=0x7f2664106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:31.922 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.920+0000 7f266aa6e640 1 --2- 172.21.15.39:0/1724880378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664105e90 0x7f2664106b90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2654009900 tx=0x7f265402a410 comp rx=0 tx=0).stop 2024-05-04T03:24:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.922+0000 7f266aa6e640 1 -- 172.21.15.39:0/1724880378 shutdown_connections 2024-05-04T03:24:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.922+0000 7f266aa6e640 1 --2- 172.21.15.39:0/1724880378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f266410bdd0 0x7f266410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.922+0000 7f266aa6e640 1 --2- 172.21.15.39:0/1724880378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26641070d0 0x7f26641074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.922+0000 7f266aa6e640 1 --2- 172.21.15.39:0/1724880378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2664105e90 0x7f2664106b90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.922+0000 7f266aa6e640 1 -- 172.21.15.39:0/1724880378 >> 172.21.15.39:0/1724880378 conn(0x7f2664101060 msgr2=0x7f2664103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.923+0000 7f266aa6e640 1 -- 172.21.15.39:0/1724880378 shutdown_connections 2024-05-04T03:24:31.925 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.923+0000 7f266aa6e640 1 -- 172.21.15.39:0/1724880378 wait complete. 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.924+0000 7f266aa6e640 1 Processor -- start 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.924+0000 7f266aa6e640 1 -- start start 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.924+0000 7f266aa6e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664105e90 0x7f26641a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.924+0000 7f266aa6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26641070d0 0x7f26641a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.924+0000 7f266aa6e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f266410bdd0 0x7f26641a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.925+0000 7f266880a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664105e90 0x7f26641a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.925+0000 7f266880a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664105e90 0x7f26641a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:36626/0 (socket says 172.21.15.39:36626) 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.925+0000 7f266880a640 1 -- 172.21.15.39:0/4174088698 learned_addr learned my addr 172.21.15.39:0/4174088698 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.925+0000 7f2663fff640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26641070d0 0x7f26641a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.925+0000 7f266900b640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f266410bdd0 0x7f26641a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.925+0000 7f266aa6e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2664078120 con 0x7f26641070d0 2024-05-04T03:24:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.925+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2664077fa0 con 0x7f2664105e90 2024-05-04T03:24:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.925+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26640782a0 con 0x7f266410bdd0 2024-05-04T03:24:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.926+0000 7f266880a640 1 -- 172.21.15.39:0/4174088698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f266410bdd0 msgr2=0x7f26641a7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.926+0000 7f266880a640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f266410bdd0 0x7f26641a7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.926+0000 7f266880a640 1 -- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26641070d0 msgr2=0x7f26641a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.926+0000 7f266880a640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26641070d0 0x7f26641a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.926+0000 7f266880a640 1 -- 172.21.15.39:0/4174088698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26641a8430 con 0x7f2664105e90 2024-05-04T03:24:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.926+0000 7f266900b640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f266410bdd0 0x7f26641a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.926+0000 7f266880a640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664105e90 0x7f26641a08f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2654005760 tx=0x7f2654005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.926+0000 7f2661ffb640 1 -- 172.21.15.39:0/4174088698 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265403f030 con 0x7f2664105e90 2024-05-04T03:24:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.927+0000 7f2661ffb640 1 -- 172.21.15.39:0/4174088698 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26540043d0 con 0x7f2664105e90 2024-05-04T03:24:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.927+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26641a86c0 con 0x7f2664105e90 2024-05-04T03:24:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.927+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26641a8b40 con 0x7f2664105e90 2024-05-04T03:24:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.927+0000 7f2661ffb640 1 -- 172.21.15.39:0/4174088698 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654035070 con 0x7f2664105e90 2024-05-04T03:24:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.930+0000 7f2661ffb640 1 -- 172.21.15.39:0/4174088698 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2654042030 con 0x7f2664105e90 2024-05-04T03:24:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.931+0000 7f2671a01740 1 Processor -- start 2024-05-04T03:24:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.931+0000 7f2671a01740 1 -- start start 2024-05-04T03:24:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.931+0000 7f2671a01740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478fca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.931+0000 7f2671a01740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d478fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.931+0000 7f2671a01740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 0x557d47903ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.931+0000 7f2661ffb640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f264c0b2860 0x7f264c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f2671a01740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557d478615d0 con 0x557d4753a4d0 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f2671a01740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557d4781a890 con 0x557d4755b1b0 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f2671a01740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557d4785d590 con 0x557d475322f0 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f2663fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 0x557d47903ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f2661ffb640 1 -- 172.21.15.39:0/4174088698 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2654102510 con 0x7f2664105e90 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f266900b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f266900b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478fca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:36638/0 (socket says 172.21.15.39:36638) 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f266900b640 1 -- 172.21.15.39:0/2935052386 learned_addr learned my addr 172.21.15.39:0/2935052386 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.932+0000 7f266900b640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f264c0b2860 0x7f264c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.933+0000 7f266880a640 1 --2- 172.21.15.39:0/2935052386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d478fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.933+0000 7f266900b640 1 -- 172.21.15.39:0/2935052386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 msgr2=0x557d478fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.933+0000 7f266900b640 1 --2- 172.21.15.39:0/2935052386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d478fcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.933+0000 7f266900b640 1 -- 172.21.15.39:0/2935052386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 msgr2=0x557d47903ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.933+0000 7f266900b640 1 --2- 172.21.15.39:0/2935052386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 0x557d47903ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.933+0000 7f266900b640 1 -- 172.21.15.39:0/2935052386 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d47573210 con 0x557d4755b1b0 2024-05-04T03:24:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.935+0000 7f266900b640 1 --2- 172.21.15.39:0/2935052386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478fca60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f265c011fd0 tx=0x7f265c00fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270dbe6fd26cc616 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.935+0000 7f266900b640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f264c0b2860 0x7f264c0b4d70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f265c00ff50 tx=0x7f265c01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.935+0000 7f264affd640 1 -- 172.21.15.39:0/2935052386 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c02b070 con 0x557d4755b1b0 2024-05-04T03:24:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.935+0000 7f264affd640 1 -- 172.21.15.39:0/2935052386 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265c011630 con 0x557d4755b1b0 2024-05-04T03:24:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.935+0000 7f264affd640 1 -- 172.21.15.39:0/2935052386 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c022070 con 0x557d4755b1b0 2024-05-04T03:24:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 -- 172.21.15.39:0/2935052386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 msgr2=0x557d478fca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 --2- 172.21.15.39:0/2935052386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478fca60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f265c011fd0 tx=0x7f265c00fa50 comp rx=0 tx=0).stop 2024-05-04T03:24:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 -- 172.21.15.39:0/2935052386 shutdown_connections 2024-05-04T03:24:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 --2- 172.21.15.39:0/2935052386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 0x557d47903ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 --2- 172.21.15.39:0/2935052386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d478fcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 --2- 172.21.15.39:0/2935052386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478fca60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 -- 172.21.15.39:0/2935052386 >> 172.21.15.39:0/2935052386 conn(0x557d4754f300 msgr2=0x557d478f83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 -- 172.21.15.39:0/2935052386 shutdown_connections 2024-05-04T03:24:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.936+0000 7f2671a01740 1 -- 172.21.15.39:0/2935052386 wait complete. 2024-05-04T03:24:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.937+0000 7f2671a01740 1 -- start start 2024-05-04T03:24:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.937+0000 7f2671a01740 1 Processor -- start 2024-05-04T03:24:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.937+0000 7f2671a01740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d479176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.937+0000 7f2671a01740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 0x557d47917c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f2671a01740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d4791eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f2671a01740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557d47915930 con 0x557d4753a4d0 2024-05-04T03:24:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f2671a01740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557d4781e370 con 0x557d4755b1b0 2024-05-04T03:24:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f2671a01740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557d47915ab0 con 0x557d475322f0 2024-05-04T03:24:31.940 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f2671a01740 10 client.0 _ll_register_callbacks cb 0x557d47572a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f266900b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d4791eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f266900b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d4791eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:36648/0 (socket says 172.21.15.39:36648) 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f266900b640 1 -- 172.21.15.39:0/2730783999 learned_addr learned my addr 172.21.15.39:0/2730783999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f2663fff640 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 0x557d47917c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f266880a640 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d479176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f266900b640 1 -- 172.21.15.39:0/2730783999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 msgr2=0x557d479176d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f266900b640 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d479176d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f266900b640 1 -- 172.21.15.39:0/2730783999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 msgr2=0x557d47917c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.938+0000 7f266900b640 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 0x557d47917c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.939+0000 7f266900b640 1 -- 172.21.15.39:0/2730783999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4763f620 con 0x557d4755b1b0 2024-05-04T03:24:31.941 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.939+0000 7f266880a640 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d479176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.939+0000 7f266900b640 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d4791eb70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f265c006010 tx=0x7f265c003e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4418adefd025222d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.939+0000 7f2643fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.939+0000 7f2643fff640 1 -- 172.21.15.39:0/2730783999 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c034060 con 0x557d4755b1b0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.939+0000 7f2643fff640 1 -- 172.21.15.39:0/2730783999 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f265c011ce0 con 0x557d4755b1b0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.939+0000 7f2671a01740 1 -- 172.21.15.39:0/2730783999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265c011630 con 0x557d4755b1b0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.940+0000 7f2643fff640 1 -- 172.21.15.39:0/2730783999 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f265c02b040 con 0x557d4755b1b0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.940+0000 7f2671a01740 10 client.26396 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.940+0000 7f2671a01740 1 -- 172.21.15.39:0/2730783999 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557d47921b10 con 0x557d4755b1b0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.940+0000 7f2643fff640 1 -- 172.21.15.39:0/2730783999 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265c011630 con 0x557d4755b1b0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.940+0000 7f2643fff640 1 -- 172.21.15.39:0/2730783999 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f265c02e070 con 0x557d4755b1b0 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.941+0000 7f2671a01740 10 client.26396 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.941+0000 7f2671a01740 10 client.26396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.941+0000 7f2671a01740 20 client.26396 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.941+0000 7f2671a01740 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557d47519d80 0x557d479250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.941+0000 7f2671a01740 4 client.26396 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.941+0000 7f2671a01740 1 -- 172.21.15.39:0/2730783999 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d4786f1e0 con 0x557d47519d80 2024-05-04T03:24:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.941+0000 7f266880a640 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557d47519d80 0x557d479250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.942+0000 7f266880a640 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557d47519d80 0x557d479250a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77dda6a41259da9a server_cookie=24b7cf54ac50a2 in_seq=0 out_seq=0 2024-05-04T03:24:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.942+0000 7f2643fff640 10 client.26396 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2643fff640 1 -- 172.21.15.39:0/2730783999 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d4786f1e0 con 0x557d47519d80 2024-05-04T03:24:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2643fff640 10 client.26396 handle_command_reply: tid=0 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 client.26396 shutdown 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 -- 172.21.15.39:0/2730783999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 msgr2=0x557d4791eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d4791eb70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f265c006010 tx=0x7f265c003e50 comp rx=0 tx=0).stop 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 -- 172.21.15.39:0/2730783999 shutdown_connections 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557d47519d80 0x557d479250a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d4791eb70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d4753a4d0 0x557d47917c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 --2- 172.21.15.39:0/2730783999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d475322f0 0x557d479176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2671a01740 1 -- 172.21.15.39:0/2730783999 >> 172.21.15.39:0/2730783999 conn(0x557d4754f300 msgr2=0x557d478f73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.943+0000 7f2643fff640 0 client.26396 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.944+0000 7f2671a01740 1 -- 172.21.15.39:0/2730783999 shutdown_connections 2024-05-04T03:24:31.946 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.944+0000 7f2671a01740 1 -- 172.21.15.39:0/2730783999 wait complete. 2024-05-04T03:24:31.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.944+0000 7f2671a01740 20 client.26396 trim_cache size 0 max 16384 2024-05-04T03:24:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.972+0000 7f2671a01740 1 Processor -- start 2024-05-04T03:24:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.972+0000 7f2671a01740 1 -- start start 2024-05-04T03:24:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f2671a01740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d478f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f2671a01740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4753a4d0 0x557d4791ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f2671a01740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478f8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f266880a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d478f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f266880a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d478f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46540/0 (socket says 172.21.15.39:46540) 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f2663fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4753a4d0 0x557d4791ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f2663fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4753a4d0 0x557d4791ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:54282/0 (socket says 172.21.15.39:54282) 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f266900b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f266900b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478f8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:36650/0 (socket says 172.21.15.39:36650) 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f266880a640 1 -- 172.21.15.39:0/1784476843 learned_addr learned my addr 172.21.15.39:0/1784476843 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.973+0000 7f2671a01740 1 -- 172.21.15.39:0/1784476843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26641a9090 con 0x557d475322f0 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.974+0000 7f2671a01740 1 -- 172.21.15.39:0/1784476843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26641aabd0 con 0x557d4755b1b0 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.974+0000 7f2671a01740 1 -- 172.21.15.39:0/1784476843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26641a9760 con 0x557d4753a4d0 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.974+0000 7f266900b640 1 -- 172.21.15.39:0/1784476843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4753a4d0 msgr2=0x557d4791ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.974+0000 7f266900b640 1 --2- 172.21.15.39:0/1784476843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4753a4d0 0x557d4791ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.974+0000 7f266900b640 1 -- 172.21.15.39:0/1784476843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 msgr2=0x557d478f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.974+0000 7f266900b640 1 --2- 172.21.15.39:0/1784476843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d478f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.974+0000 7f266900b640 1 -- 172.21.15.39:0/1784476843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d478f8d00 con 0x557d4755b1b0 2024-05-04T03:24:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.974+0000 7f266900b640 1 --2- 172.21.15.39:0/1784476843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478f8230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f265c0044c0 tx=0x7f265c00ff80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f004b427c9919c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.975+0000 7f264a7fc640 1 -- 172.21.15.39:0/1784476843 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c03f020 con 0x557d4755b1b0 2024-05-04T03:24:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.975+0000 7f264a7fc640 1 -- 172.21.15.39:0/1784476843 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265c004730 con 0x557d4755b1b0 2024-05-04T03:24:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.975+0000 7f264a7fc640 1 -- 172.21.15.39:0/1784476843 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c034040 con 0x557d4755b1b0 2024-05-04T03:24:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.976+0000 7f2671a01740 1 -- 172.21.15.39:0/1784476843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 msgr2=0x557d478f8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.976+0000 7f2671a01740 1 --2- 172.21.15.39:0/1784476843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478f8230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f265c0044c0 tx=0x7f265c00ff80 comp rx=0 tx=0).stop 2024-05-04T03:24:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.976+0000 7f2671a01740 1 -- 172.21.15.39:0/1784476843 shutdown_connections 2024-05-04T03:24:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.976+0000 7f2671a01740 1 --2- 172.21.15.39:0/1784476843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4755b1b0 0x557d478f8230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.977+0000 7f2671a01740 1 --2- 172.21.15.39:0/1784476843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4753a4d0 0x557d4791ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.977+0000 7f2671a01740 1 --2- 172.21.15.39:0/1784476843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d478f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.977+0000 7f2671a01740 1 -- 172.21.15.39:0/1784476843 >> 172.21.15.39:0/1784476843 conn(0x557d4754f300 msgr2=0x557d47900b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.978+0000 7f2671a01740 1 -- 172.21.15.39:0/1784476843 shutdown_connections 2024-05-04T03:24:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.980+0000 7f2671a01740 1 -- 172.21.15.39:0/1784476843 wait complete. 2024-05-04T03:24:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.981+0000 7f2671a01740 1 -- start start 2024-05-04T03:24:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.981+0000 7f2671a01740 1 Processor -- start 2024-05-04T03:24:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.981+0000 7f2671a01740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d4791b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f2671a01740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4753a4d0 0x557d4791bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f2671a01740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4755b1b0 0x557d4798b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f2671a01740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26641aa500 con 0x557d475322f0 2024-05-04T03:24:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f2671a01740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f26641a9e30 con 0x557d4753a4d0 2024-05-04T03:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f2671a01740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26641172a0 con 0x557d4755b1b0 2024-05-04T03:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f2671a01740 10 client.0 _ll_register_callbacks cb 0x7f265c029070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f266880a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d4791b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f266880a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d4791b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46546/0 (socket says 172.21.15.39:46546) 2024-05-04T03:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f266880a640 1 -- 172.21.15.39:0/844777300 learned_addr learned my addr 172.21.15.39:0/844777300 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.982+0000 7f2663fff640 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4753a4d0 0x557d4791bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.983+0000 7f266900b640 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4755b1b0 0x557d4798b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.983+0000 7f266880a640 1 -- 172.21.15.39:0/844777300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4755b1b0 msgr2=0x557d4798b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.983+0000 7f266880a640 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4755b1b0 0x557d4798b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.983+0000 7f266880a640 1 -- 172.21.15.39:0/844777300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4753a4d0 msgr2=0x557d4791bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.983+0000 7f266880a640 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4753a4d0 0x557d4791bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.983+0000 7f266880a640 1 -- 172.21.15.39:0/844777300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4791eb70 con 0x557d475322f0 2024-05-04T03:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.984+0000 7f266880a640 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d4791b6c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f265402a450 tx=0x7f26540c5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113561834a56db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.984+0000 7f2643fff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.984+0000 7f2643fff640 1 -- 172.21.15.39:0/844777300 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654049040 con 0x557d475322f0 2024-05-04T03:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.985+0000 7f2671a01740 1 -- 172.21.15.39:0/844777300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265c004730 con 0x557d475322f0 2024-05-04T03:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.985+0000 7f2643fff640 1 -- 172.21.15.39:0/844777300 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f26540c5e20 con 0x557d475322f0 2024-05-04T03:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.985+0000 7f2643fff640 1 -- 172.21.15.39:0/844777300 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26540e41d0 con 0x557d475322f0 2024-05-04T03:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.985+0000 7f2671a01740 10 client.16821 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.985+0000 7f2671a01740 1 -- 172.21.15.39:0/844777300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557d479047d0 con 0x557d475322f0 2024-05-04T03:24:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2643fff640 1 -- 172.21.15.39:0/844777300 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265c004730 con 0x557d475322f0 2024-05-04T03:24:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2643fff640 1 -- 172.21.15.39:0/844777300 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f26540f46e0 con 0x557d475322f0 2024-05-04T03:24:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2671a01740 10 client.16821 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2671a01740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2671a01740 20 client.16821 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2671a01740 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557d474ef1c0 0x557d479071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2671a01740 4 client.16821 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2671a01740 1 -- 172.21.15.39:0/844777300 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d47869050 con 0x557d474ef1c0 2024-05-04T03:24:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.986+0000 7f2663fff640 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557d474ef1c0 0x557d479071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.987+0000 7f2643fff640 10 client.16821 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.987+0000 7f2663fff640 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557d474ef1c0 0x557d479071c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de13c3e24ef6fb42 server_cookie=400f776d6e9694d0 in_seq=0 out_seq=0 2024-05-04T03:24:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2643fff640 1 -- 172.21.15.39:0/844777300 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d47869050 con 0x557d474ef1c0 2024-05-04T03:24:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2643fff640 10 client.16821 handle_command_reply: tid=0 2024-05-04T03:24:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 client.16821 shutdown 2024-05-04T03:24:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 -- 172.21.15.39:0/844777300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 msgr2=0x557d4791b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d4791b6c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f265402a450 tx=0x7f26540c5b70 comp rx=0 tx=0).stop 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 -- 172.21.15.39:0/844777300 shutdown_connections 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557d474ef1c0 0x557d479071c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557d4755b1b0 0x557d4798b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d4753a4d0 0x557d4791bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 --2- 172.21.15.39:0/844777300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557d475322f0 0x557d4791b6c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.996+0000 7f2671a01740 1 -- 172.21.15.39:0/844777300 >> 172.21.15.39:0/844777300 conn(0x557d4754f300 msgr2=0x557d478f7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.997+0000 7f2643fff640 0 client.16821 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.997+0000 7f2671a01740 1 -- 172.21.15.39:0/844777300 shutdown_connections 2024-05-04T03:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.997+0000 7f2671a01740 1 -- 172.21.15.39:0/844777300 wait complete. 2024-05-04T03:24:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.997+0000 7f2671a01740 20 client.16821 trim_cache size 0 max 16384 2024-05-04T03:24:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:24:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:24:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:24:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:24:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.998+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f264c0b2860 msgr2=0x7f264c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.999+0000 7f266aa6e640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f264c0b2860 0x7f264c0b4d70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f265c00ff50 tx=0x7f265c01e000 comp rx=0 tx=0).stop 2024-05-04T03:24:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.999+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664105e90 msgr2=0x7f26641a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:31.999+0000 7f266aa6e640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664105e90 0x7f26641a08f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2654005760 tx=0x7f2654005e20 comp rx=0 tx=0).stop 2024-05-04T03:24:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:32.000+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 shutdown_connections 2024-05-04T03:24:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:32.000+0000 7f266aa6e640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f264c0b2860 0x7f264c0b4d70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:32.000+0000 7f266aa6e640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f266410bdd0 0x7f26641a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:32.000+0000 7f266aa6e640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26641070d0 0x7f26641a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:32.000+0000 7f266aa6e640 1 --2- 172.21.15.39:0/4174088698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664105e90 0x7f26641a08f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:32.000+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 >> 172.21.15.39:0/4174088698 conn(0x7f2664101060 msgr2=0x7f2664078af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:32.001+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 shutdown_connections 2024-05-04T03:24:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:32.001+0000 7f266aa6e640 1 -- 172.21.15.39:0/4174088698 wait complete. 2024-05-04T03:24:32.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:32 smithi066 ceph-mon[33264]: pgmap v798: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.50k op/s 2024-05-04T03:24:32.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:32 smithi167 ceph-mon[33614]: pgmap v798: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.50k op/s 2024-05-04T03:24:32.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:32 smithi039 ceph-mon[29715]: pgmap v798: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 MiB/s wr, 1.50k op/s 2024-05-04T03:24:34.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:34 smithi066 ceph-mon[33264]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-05-04T03:24:34.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:34 smithi167 ceph-mon[33614]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-05-04T03:24:34.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:34 smithi039 ceph-mon[29715]: pgmap v799: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-05-04T03:24:36.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 645.0 s, 1053.4 tps, lat 95.209 ms stddev 413.933 2024-05-04T03:24:36.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:36 smithi066 ceph-mon[33264]: pgmap v800: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-05-04T03:24:36.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:36 smithi167 ceph-mon[33614]: pgmap v800: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-05-04T03:24:36.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:36 smithi039 ceph-mon[29715]: pgmap v800: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-05-04T03:24:38.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:38 smithi066 ceph-mon[33264]: pgmap v801: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-05-04T03:24:38.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:38 smithi167 ceph-mon[33614]: pgmap v801: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-05-04T03:24:38.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:38 smithi039 ceph-mon[29715]: pgmap v801: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.57k op/s 2024-05-04T03:24:40.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:40 smithi066 ceph-mon[33264]: pgmap v802: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.53k op/s 2024-05-04T03:24:40.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:40 smithi167 ceph-mon[33614]: pgmap v802: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.53k op/s 2024-05-04T03:24:40.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:40 smithi039 ceph-mon[29715]: pgmap v802: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.53k op/s 2024-05-04T03:24:41.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 650.0 s, 1024.0 tps, lat 97.499 ms stddev 99.401 2024-05-04T03:24:42.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:42 smithi066 ceph-mon[33264]: pgmap v803: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-05-04T03:24:42.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:42 smithi167 ceph-mon[33614]: pgmap v803: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-05-04T03:24:42.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:42 smithi039 ceph-mon[29715]: pgmap v803: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-05-04T03:24:44.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:44 smithi066 ceph-mon[33264]: pgmap v804: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.54k op/s 2024-05-04T03:24:44.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:44 smithi167 ceph-mon[33614]: pgmap v804: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.54k op/s 2024-05-04T03:24:44.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:44 smithi039 ceph-mon[29715]: pgmap v804: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.54k op/s 2024-05-04T03:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.163+0000 7f6d77fff640 1 -- 172.21.15.39:0/3152847582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d78104ce0 msgr2=0x7f6d781050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.163+0000 7f6d77fff640 1 --2- 172.21.15.39:0/3152847582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d78104ce0 0x7f6d781050f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009860 tx=0x7f6d6c02a340 comp rx=0 tx=0).stop 2024-05-04T03:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.163+0000 7f6d77fff640 1 -- 172.21.15.39:0/3152847582 shutdown_connections 2024-05-04T03:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.163+0000 7f6d77fff640 1 --2- 172.21.15.39:0/3152847582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d781099e0 0x7f6d7810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.163+0000 7f6d77fff640 1 --2- 172.21.15.39:0/3152847582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d78104ce0 0x7f6d781050f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.163+0000 7f6d77fff640 1 --2- 172.21.15.39:0/3152847582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78103aa0 0x7f6d781047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.163+0000 7f6d77fff640 1 -- 172.21.15.39:0/3152847582 >> 172.21.15.39:0/3152847582 conn(0x7f6d780fecd0 msgr2=0x7f6d78101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.164+0000 7f6d77fff640 1 -- 172.21.15.39:0/3152847582 shutdown_connections 2024-05-04T03:24:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.164+0000 7f6d77fff640 1 -- 172.21.15.39:0/3152847582 wait complete. 2024-05-04T03:24:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.165+0000 7f6d77fff640 1 Processor -- start 2024-05-04T03:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d77fff640 1 -- start start 2024-05-04T03:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d78103aa0 0x7f6d78077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78104ce0 0x7f6d78076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d781099e0 0x7f6d78076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d77fff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7810f450 con 0x7f6d78104ce0 2024-05-04T03:24:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d77fff640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7810f2d0 con 0x7f6d78103aa0 2024-05-04T03:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d77fff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7810f5d0 con 0x7f6d781099e0 2024-05-04T03:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d767fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78104ce0 0x7f6d78076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d767fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78104ce0 0x7f6d78076130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57762/0 (socket says 172.21.15.39:57762) 2024-05-04T03:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d767fc640 1 -- 172.21.15.39:0/1111818437 learned_addr learned my addr 172.21.15.39:0/1111818437 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d78103aa0 0x7f6d78077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.166+0000 7f6d777fe640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d781099e0 0x7f6d78076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.167+0000 7f6d767fc640 1 -- 172.21.15.39:0/1111818437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d781099e0 msgr2=0x7f6d78076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.167+0000 7f6d767fc640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d781099e0 0x7f6d78076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.167+0000 7f6d767fc640 1 -- 172.21.15.39:0/1111818437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d78103aa0 msgr2=0x7f6d78077a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.167+0000 7f6d767fc640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d78103aa0 0x7f6d78077a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.167+0000 7f6d767fc640 1 -- 172.21.15.39:0/1111818437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d78076da0 con 0x7f6d78104ce0 2024-05-04T03:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.167+0000 7f6d777fe640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d781099e0 0x7f6d78076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.167+0000 7f6d767fc640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78104ce0 0x7f6d78076130 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c02a930 tx=0x7f6d6c02ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.167+0000 7f6d7c978640 1 -- 172.21.15.39:0/1111818437 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c040030 con 0x7f6d78104ce0 2024-05-04T03:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.168+0000 7f6d7c978640 1 -- 172.21.15.39:0/1111818437 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c0305f0 con 0x7f6d78104ce0 2024-05-04T03:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.168+0000 7f6d77fff640 1 -- 172.21.15.39:0/1111818437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d781a4180 con 0x7f6d78104ce0 2024-05-04T03:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.168+0000 7f6d7c978640 1 -- 172.21.15.39:0/1111818437 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c037040 con 0x7f6d78104ce0 2024-05-04T03:24:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.168+0000 7f6d77fff640 1 -- 172.21.15.39:0/1111818437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d781a4530 con 0x7f6d78104ce0 2024-05-04T03:24:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d8482a740 1 Processor -- start 2024-05-04T03:24:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d8482a740 1 -- start start 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36d9bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d7c978640 1 -- 172.21.15.39:0/1111818437 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f6d6c043070 con 0x7f6d78104ce0 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d9c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36da30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d777fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36d9bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d777fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36d9bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:57400/0 (socket says 172.21.15.39:57400) 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d777fe640 1 -- 172.21.15.39:0/295237788 learned_addr learned my addr 172.21.15.39:0/295237788 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d8482a740 1 -- 172.21.15.39:0/295237788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cb36d005d0 con 0x55cb369d12f0 2024-05-04T03:24:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d8482a740 1 -- 172.21.15.39:0/295237788 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb36cb9890 con 0x55cb369fa1b0 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d8482a740 1 -- 172.21.15.39:0/295237788 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cb36cfc590 con 0x55cb369d94d0 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.172+0000 7f6d767fc640 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36da30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.173+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d9c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.173+0000 7f6d767fc640 1 -- 172.21.15.39:0/295237788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 msgr2=0x55cb36d9bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.173+0000 7f6d767fc640 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36d9bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.173+0000 7f6d767fc640 1 -- 172.21.15.39:0/295237788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 msgr2=0x55cb36d9c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.173+0000 7f6d767fc640 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d9c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.173+0000 7f6d767fc640 1 -- 172.21.15.39:0/295237788 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb36a12210 con 0x55cb369d94d0 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.173+0000 7f6d777fe640 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36d9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.173+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d9c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.174+0000 7f6d7c978640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b2860 0x7f6d600b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.174+0000 7f6d767fc640 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36da30b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0ffa00 tx=0x7f6d6c105f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3684ea41c64c384d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.174+0000 7f6d777fe640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b2860 0x7f6d600b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.174+0000 7f6d5dffb640 1 -- 172.21.15.39:0/295237788 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0c5c60 con 0x55cb369d94d0 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.174+0000 7f6d5dffb640 1 -- 172.21.15.39:0/295237788 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c10f070 con 0x55cb369d94d0 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.174+0000 7f6d5dffb640 1 -- 172.21.15.39:0/295237788 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0c8d80 con 0x55cb369d94d0 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.174+0000 7f6d7c978640 1 -- 172.21.15.39:0/1111818437 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d6c101fb0 con 0x7f6d78104ce0 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.175+0000 7f6d777fe640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b2860 0x7f6d600b4d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55cb36d9ce00 tx=0x7f6d70009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.175+0000 7f6d8482a740 1 -- 172.21.15.39:0/295237788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 msgr2=0x55cb36da30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.175+0000 7f6d8482a740 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36da30b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0ffa00 tx=0x7f6d6c105f50 comp rx=0 tx=0).stop 2024-05-04T03:24:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.175+0000 7f6d8482a740 1 -- 172.21.15.39:0/295237788 shutdown_connections 2024-05-04T03:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.175+0000 7f6d8482a740 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36da30b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.176+0000 7f6d8482a740 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d9c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.176+0000 7f6d8482a740 1 --2- 172.21.15.39:0/295237788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36d9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.176+0000 7f6d8482a740 1 -- 172.21.15.39:0/295237788 >> 172.21.15.39:0/295237788 conn(0x55cb369ee300 msgr2=0x55cb36d975c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.176+0000 7f6d8482a740 1 -- 172.21.15.39:0/295237788 shutdown_connections 2024-05-04T03:24:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.176+0000 7f6d8482a740 1 -- 172.21.15.39:0/295237788 wait complete. 2024-05-04T03:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.176+0000 7f6d8482a740 1 -- start start 2024-05-04T03:24:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.176+0000 7f6d8482a740 1 Processor -- start 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d12f0 0x55cb36da5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d94d0 0x55cb36da5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d767fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d94d0 0x55cb36da5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d767fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d94d0 0x55cb36da5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57770/0 (socket says 172.21.15.39:57770) 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d767fc640 1 -- 172.21.15.39:0/317531885 learned_addr learned my addr 172.21.15.39:0/317531885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d12f0 0x55cb36da5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cb36cfc590 con 0x55cb369d94d0 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb36cbd370 con 0x55cb369d12f0 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cb36da38c0 con 0x55cb369fa1b0 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d777fe640 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.177+0000 7f6d8482a740 10 client.0 _ll_register_callbacks cb 0x55cb36a11a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.178+0000 7f6d777fe640 1 -- 172.21.15.39:0/317531885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d12f0 msgr2=0x55cb36da5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.178+0000 7f6d777fe640 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d12f0 0x55cb36da5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.178+0000 7f6d777fe640 1 -- 172.21.15.39:0/317531885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d94d0 msgr2=0x55cb36da5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.178+0000 7f6d777fe640 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d94d0 0x55cb36da5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.178+0000 7f6d777fe640 1 -- 172.21.15.39:0/317531885 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb36ade620 con 0x55cb369fa1b0 2024-05-04T03:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.178+0000 7f6d777fe640 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dac8f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009fd0 tx=0x7f6d70004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67773265e74075dc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.179+0000 7f6d56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.179+0000 7f6d56ffd640 1 -- 172.21.15.39:0/317531885 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70004830 con 0x55cb369fa1b0 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.179+0000 7f6d56ffd640 1 -- 172.21.15.39:0/317531885 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d70017030 con 0x55cb369fa1b0 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.179+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c10f070 con 0x55cb369fa1b0 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.180+0000 7f6d56ffd640 1 -- 172.21.15.39:0/317531885 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6d70011020 con 0x55cb369fa1b0 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.180+0000 7f6d8482a740 10 client.26449 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.180+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cb36daf8f0 con 0x55cb369fa1b0 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.180+0000 7f6d56ffd640 1 -- 172.21.15.39:0/317531885 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c10f070 con 0x55cb369fa1b0 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.180+0000 7f6d56ffd640 1 -- 172.21.15.39:0/317531885 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6d700105c0 con 0x55cb369fa1b0 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.181+0000 7f6d8482a740 10 client.26449 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.181+0000 7f6d8482a740 10 client.26449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.181+0000 7f6d8482a740 20 client.26449 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.181+0000 7f6d8482a740 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55cb369b8d80 0x55cb36db3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.181+0000 7f6d8482a740 4 client.26449 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.181+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb36d0e1e0 con 0x55cb369b8d80 2024-05-04T03:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.181+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55cb369b8d80 0x55cb36db3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.182+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55cb369b8d80 0x55cb36db3060 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143af2fe03628dd3 server_cookie=a21b5743d991777b in_seq=0 out_seq=0 2024-05-04T03:24:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.182+0000 7f6d56ffd640 10 client.26449 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.182+0000 7f6d56ffd640 1 -- 172.21.15.39:0/317531885 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb36d0e1e0 con 0x55cb369b8d80 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.182+0000 7f6d56ffd640 10 client.26449 handle_command_reply: tid=0 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 client.26449 shutdown 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 msgr2=0x55cb36dac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dac8f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009fd0 tx=0x7f6d70004ab0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 shutdown_connections 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55cb369b8d80 0x55cb36db3060 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dac8f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d94d0 0x55cb36da5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 --2- 172.21.15.39:0/317531885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d12f0 0x55cb36da5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 >> 172.21.15.39:0/317531885 conn(0x55cb369ee300 msgr2=0x55cb36d96580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d56ffd640 0 client.26449 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 shutdown_connections 2024-05-04T03:24:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.183+0000 7f6d8482a740 1 -- 172.21.15.39:0/317531885 wait complete. 2024-05-04T03:24:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.184+0000 7f6d8482a740 20 client.26449 trim_cache size 0 max 16384 2024-05-04T03:24:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.216+0000 7f6d8482a740 1 Processor -- start 2024-05-04T03:24:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.216+0000 7f6d8482a740 1 -- start start 2024-05-04T03:24:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.217+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.217+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36dac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.217+0000 7f6d76ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.217+0000 7f6d76ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57776/0 (socket says 172.21.15.39:57776) 2024-05-04T03:24:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.217+0000 7f6d76ffd640 1 -- 172.21.15.39:0/3734451110 learned_addr learned my addr 172.21.15.39:0/3734451110 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d767fc640 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36dac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d8482a740 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36dacd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d8482a740 1 -- 172.21.15.39:0/3734451110 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ad300 con 0x55cb369d12f0 2024-05-04T03:24:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d8482a740 1 -- 172.21.15.39:0/3734451110 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781aee40 con 0x55cb369fa1b0 2024-05-04T03:24:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d8482a740 1 -- 172.21.15.39:0/3734451110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ad9d0 con 0x55cb369d94d0 2024-05-04T03:24:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d777fe640 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36dacd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d767fc640 1 -- 172.21.15.39:0/3734451110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 msgr2=0x55cb36dacd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d767fc640 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36dacd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d767fc640 1 -- 172.21.15.39:0/3734451110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 msgr2=0x55cb36d975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d767fc640 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.218+0000 7f6d767fc640 1 -- 172.21.15.39:0/3734451110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb36da3440 con 0x55cb369d94d0 2024-05-04T03:24:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.222+0000 7f6d767fc640 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36dac810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c02a900 tx=0x7f6d6c02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ad6147e8e859d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.222+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/3734451110 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0c8f60 con 0x55cb369d94d0 2024-05-04T03:24:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.222+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/3734451110 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c10f070 con 0x55cb369d94d0 2024-05-04T03:24:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.222+0000 7f6d5d7fa640 1 -- 172.21.15.39:0/3734451110 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0c9150 con 0x55cb369d94d0 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.223+0000 7f6d8482a740 1 -- 172.21.15.39:0/3734451110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 msgr2=0x55cb36dac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.223+0000 7f6d8482a740 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36dac810 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c02a900 tx=0x7f6d6c02a340 comp rx=0 tx=0).stop 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.224+0000 7f6d8482a740 1 -- 172.21.15.39:0/3734451110 shutdown_connections 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.224+0000 7f6d8482a740 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369fa1b0 0x55cb36dacd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.224+0000 7f6d8482a740 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369d94d0 0x55cb36dac810 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.224+0000 7f6d8482a740 1 --2- 172.21.15.39:0/3734451110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36d975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.224+0000 7f6d8482a740 1 -- 172.21.15.39:0/3734451110 >> 172.21.15.39:0/3734451110 conn(0x55cb369ee300 msgr2=0x55cb36d96ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.224+0000 7f6d8482a740 1 -- 172.21.15.39:0/3734451110 shutdown_connections 2024-05-04T03:24:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.224+0000 7f6d8482a740 1 -- 172.21.15.39:0/3734451110 wait complete. 2024-05-04T03:24:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.224+0000 7f6d8482a740 1 -- start start 2024-05-04T03:24:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.225+0000 7f6d8482a740 1 Processor -- start 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.225+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36daca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.225+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d94d0 0x55cb36dacf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.225+0000 7f6d8482a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.225+0000 7f6d8482a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ae770 con 0x55cb369d12f0 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.225+0000 7f6d8482a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ae0a0 con 0x55cb369d94d0 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.225+0000 7f6d8482a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ab2e0 con 0x55cb369fa1b0 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.225+0000 7f6d8482a740 10 client.0 _ll_register_callbacks cb 0x7f6d70003c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d767fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d94d0 0x55cb36dacf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d767fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d94d0 0x55cb36dacf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:57436/0 (socket says 172.21.15.39:57436) 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d767fc640 1 -- 172.21.15.39:0/1844838003 learned_addr learned my addr 172.21.15.39:0/1844838003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d777fe640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36daca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d767fc640 1 -- 172.21.15.39:0/1844838003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 msgr2=0x55cb36dab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d767fc640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d767fc640 1 -- 172.21.15.39:0/1844838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 msgr2=0x55cb36daca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d767fc640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36daca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d767fc640 1 -- 172.21.15.39:0/1844838003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb36dab7b0 con 0x55cb369d94d0 2024-05-04T03:24:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d777fe640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.226+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36daca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:24:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.227+0000 7f6d767fc640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d94d0 0x55cb36dacf40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0c5690 tx=0x7f6d6c0c5b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6923c837b7767f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.227+0000 7f6d56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.227+0000 7f6d56ffd640 1 -- 172.21.15.39:0/1844838003 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0c93e0 con 0x55cb369d94d0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.227+0000 7f6d56ffd640 1 -- 172.21.15.39:0/1844838003 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f6d6c1050c0 con 0x55cb369d94d0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.227+0000 7f6d8482a740 1 -- 172.21.15.39:0/1844838003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c10f070 con 0x55cb369d94d0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.227+0000 7f6d56ffd640 1 -- 172.21.15.39:0/1844838003 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0c4f80 con 0x55cb369d94d0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.227+0000 7f6d8482a740 10 client.26417 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.227+0000 7f6d8482a740 1 -- 172.21.15.39:0/1844838003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cb36dab7b0 con 0x55cb369d94d0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.228+0000 7f6d56ffd640 1 -- 172.21.15.39:0/1844838003 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d6c10f070 con 0x55cb369d94d0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.228+0000 7f6d56ffd640 1 -- 172.21.15.39:0/1844838003 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6d6c04d060 con 0x55cb369d94d0 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.228+0000 7f6d8482a740 10 client.26417 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.228+0000 7f6d8482a740 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.228+0000 7f6d8482a740 20 client.26417 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.228+0000 7f6d8482a740 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55cb3698e1c0 0x55cb36e1e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.228+0000 7f6d8482a740 4 client.26417 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.228+0000 7f6d8482a740 1 -- 172.21.15.39:0/1844838003 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb36d08050 con 0x55cb3698e1c0 2024-05-04T03:24:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.229+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55cb3698e1c0 0x55cb36e1e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.230+0000 7f6d76ffd640 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55cb3698e1c0 0x55cb36e1e320 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5f8a50c16682bb server_cookie=766aa50ccd4b1e78 in_seq=0 out_seq=0 2024-05-04T03:24:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.230+0000 7f6d56ffd640 10 client.26417 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d56ffd640 1 -- 172.21.15.39:0/1844838003 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb36d08050 con 0x55cb3698e1c0 2024-05-04T03:24:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d56ffd640 10 client.26417 handle_command_reply: tid=0 2024-05-04T03:24:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 client.26417 shutdown 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 -- 172.21.15.39:0/1844838003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d94d0 msgr2=0x55cb36dacf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d94d0 0x55cb36dacf40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0c5690 tx=0x7f6d6c0c5b50 comp rx=0 tx=0).stop 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 -- 172.21.15.39:0/1844838003 shutdown_connections 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55cb3698e1c0 0x55cb36e1e320 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb369fa1b0 0x55cb36dab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb369d94d0 0x55cb36dacf40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 --2- 172.21.15.39:0/1844838003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb369d12f0 0x55cb36daca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d56ffd640 0 client.26417 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.242+0000 7f6d8482a740 1 -- 172.21.15.39:0/1844838003 >> 172.21.15.39:0/1844838003 conn(0x55cb369ee300 msgr2=0x55cb36d9a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.243+0000 7f6d8482a740 1 -- 172.21.15.39:0/1844838003 shutdown_connections 2024-05-04T03:24:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.243+0000 7f6d8482a740 1 -- 172.21.15.39:0/1844838003 wait complete. 2024-05-04T03:24:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.243+0000 7f6d8482a740 20 client.26417 trim_cache size 0 max 16384 2024-05-04T03:24:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:24:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:24:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:24:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:24:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.244+0000 7f6d77fff640 1 -- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b2860 msgr2=0x7f6d600b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.244+0000 7f6d77fff640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b2860 0x7f6d600b4d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55cb36d9ce00 tx=0x7f6d70009500 comp rx=0 tx=0).stop 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.244+0000 7f6d77fff640 1 -- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78104ce0 msgr2=0x7f6d78076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.244+0000 7f6d77fff640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78104ce0 0x7f6d78076130 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c02a930 tx=0x7f6d6c02ad70 comp rx=0 tx=0).stop 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.244+0000 7f6d77fff640 1 -- 172.21.15.39:0/1111818437 shutdown_connections 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.244+0000 7f6d77fff640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f6d600b2860 0x7f6d600b4d70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.244+0000 7f6d77fff640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d781099e0 0x7f6d78076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.244+0000 7f6d77fff640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78104ce0 0x7f6d78076130 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.245+0000 7f6d77fff640 1 --2- 172.21.15.39:0/1111818437 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d78103aa0 0x7f6d78077a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.245+0000 7f6d77fff640 1 -- 172.21.15.39:0/1111818437 >> 172.21.15.39:0/1111818437 conn(0x7f6d780fecd0 msgr2=0x7f6d781059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.245+0000 7f6d77fff640 1 -- 172.21.15.39:0/1111818437 shutdown_connections 2024-05-04T03:24:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:45.245+0000 7f6d77fff640 1 -- 172.21.15.39:0/1111818437 wait complete. 2024-05-04T03:24:46.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:46 smithi039 ceph-mon[29715]: pgmap v805: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-05-04T03:24:46.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 655.0 s, 1191.8 tps, lat 83.931 ms stddev 31.692 2024-05-04T03:24:46.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:46 smithi066 ceph-mon[33264]: pgmap v805: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-05-04T03:24:46.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:46 smithi167 ceph-mon[33614]: pgmap v805: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.42k op/s 2024-05-04T03:24:48.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:48 smithi066 ceph-mon[33264]: pgmap v806: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:48.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:48 smithi167 ceph-mon[33614]: pgmap v806: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:48.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:48 smithi039 ceph-mon[29715]: pgmap v806: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-05-04T03:24:50.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:50 smithi066 ceph-mon[33264]: pgmap v807: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-05-04T03:24:50.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:50 smithi167 ceph-mon[33614]: pgmap v807: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-05-04T03:24:50.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:50 smithi039 ceph-mon[29715]: pgmap v807: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-05-04T03:24:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 660.0 s, 919.8 tps, lat 108.434 ms stddev 88.775 2024-05-04T03:24:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:52 smithi066 ceph-mon[33264]: pgmap v808: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.41k op/s 2024-05-04T03:24:52.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:52 smithi167 ceph-mon[33614]: pgmap v808: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.41k op/s 2024-05-04T03:24:52.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:52 smithi039 ceph-mon[29715]: pgmap v808: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.41k op/s 2024-05-04T03:24:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:54 smithi066 ceph-mon[33264]: pgmap v809: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-05-04T03:24:54.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:54 smithi167 ceph-mon[33614]: pgmap v809: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-05-04T03:24:54.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:54 smithi039 ceph-mon[29715]: pgmap v809: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-05-04T03:24:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 665.0 s, 1060.8 tps, lat 94.632 ms stddev 32.349 2024-05-04T03:24:56.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:56 smithi066 ceph-mon[33264]: pgmap v810: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:24:56.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:56 smithi167 ceph-mon[33614]: pgmap v810: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:24:56.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:56 smithi039 ceph-mon[29715]: pgmap v810: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:24:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 -- 172.21.15.39:0/1969646249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102560 msgr2=0x7f8db4102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 --2- 172.21.15.39:0/1969646249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102560 0x7f8db4102970 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009900 tx=0x7f8da402a410 comp rx=0 tx=0).stop 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 -- 172.21.15.39:0/1969646249 shutdown_connections 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 --2- 172.21.15.39:0/1969646249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8db4103800 0x7f8db4103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 --2- 172.21.15.39:0/1969646249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db4102eb0 0x7f8db41032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 --2- 172.21.15.39:0/1969646249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102560 0x7f8db4102970 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 -- 172.21.15.39:0/1969646249 >> 172.21.15.39:0/1969646249 conn(0x7f8db40fdd90 msgr2=0x7f8db4100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 -- 172.21.15.39:0/1969646249 shutdown_connections 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.402+0000 7f8db889f640 1 -- 172.21.15.39:0/1969646249 wait complete. 2024-05-04T03:24:58.405 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db889f640 1 Processor -- start 2024-05-04T03:24:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db889f640 1 -- start start 2024-05-04T03:24:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db889f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db4102560 0x7f8db41a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.406 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db889f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102eb0 0x7f8db41a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db889f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8db4103800 0x7f8db41a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db889f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8db4105210 con 0x7f8db4102560 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db889f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8db4105090 con 0x7f8db4102eb0 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db889f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8db4105390 con 0x7f8db4103800 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102eb0 0x7f8db41a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8db4103800 0x7f8db41a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8db4103800 0x7f8db41a7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:51720/0 (socket says 172.21.15.39:51720) 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.404+0000 7f8db2d9d640 1 -- 172.21.15.39:0/1686710415 learned_addr learned my addr 172.21.15.39:0/1686710415 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:58.407 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.405+0000 7f8db259c640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db4102560 0x7f8db41a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.405+0000 7f8db259c640 1 -- 172.21.15.39:0/1686710415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8db4103800 msgr2=0x7f8db41a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.405+0000 7f8db259c640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8db4103800 0x7f8db41a7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.405+0000 7f8db259c640 1 -- 172.21.15.39:0/1686710415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102eb0 msgr2=0x7f8db41a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.405+0000 7f8db259c640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102eb0 0x7f8db41a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.405+0000 7f8db259c640 1 -- 172.21.15.39:0/1686710415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db41a82e0 con 0x7f8db4102560 2024-05-04T03:24:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.405+0000 7f8db1d9b640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102eb0 0x7f8db41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.406+0000 7f8db259c640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db4102560 0x7f8db41a0850 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009230 tx=0x7f8da4004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.406+0000 7f8da37fe640 1 -- 172.21.15.39:0/1686710415 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4040070 con 0x7f8db4102560 2024-05-04T03:24:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.406+0000 7f8db889f640 1 -- 172.21.15.39:0/1686710415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db41a8570 con 0x7f8db4102560 2024-05-04T03:24:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.406+0000 7f8db889f640 1 -- 172.21.15.39:0/1686710415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db41a8a80 con 0x7f8db4102560 2024-05-04T03:24:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.406+0000 7f8da37fe640 1 -- 172.21.15.39:0/1686710415 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8da4004d50 con 0x7f8db4102560 2024-05-04T03:24:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.406+0000 7f8da37fe640 1 -- 172.21.15.39:0/1686710415 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da404a060 con 0x7f8db4102560 2024-05-04T03:24:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.410+0000 7f8da37fe640 1 -- 172.21.15.39:0/1686710415 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8da4043070 con 0x7f8db4102560 2024-05-04T03:24:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.410+0000 7f8dbf82d740 1 Processor -- start 2024-05-04T03:24:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.411+0000 7f8dbf82d740 1 -- start start 2024-05-04T03:24:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.411+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.411+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71ec2f0 0x55b7d75b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.411+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.411+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b7d751b5d0 con 0x55b7d72151b0 2024-05-04T03:24:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.411+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b7d74d4890 con 0x55b7d71ec2f0 2024-05-04T03:24:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.411+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7d7517590 con 0x55b7d71f44d0 2024-05-04T03:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.411+0000 7f8da37fe640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8d9c0b2860 0x7f8d9c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.412+0000 7f8db259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71ec2f0 0x55b7d75b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.412+0000 7f8db259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71ec2f0 0x55b7d75b6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:35774/0 (socket says 172.21.15.39:35774) 2024-05-04T03:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.412+0000 7f8db259c640 1 -- 172.21.15.39:0/2189105541 learned_addr learned my addr 172.21.15.39:0/2189105541 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.412+0000 7f8db1d9b640 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.412+0000 7f8db259c640 1 -- 172.21.15.39:0/2189105541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 msgr2=0x55b7d75bdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.412+0000 7f8da37fe640 1 -- 172.21.15.39:0/1686710415 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8da4102c30 con 0x7f8db4102560 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.412+0000 7f8db259c640 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75bdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.412+0000 7f8db259c640 1 -- 172.21.15.39:0/2189105541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 msgr2=0x55b7d75b6a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.413+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.413+0000 7f8db259c640 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.413+0000 7f8db259c640 1 -- 172.21.15.39:0/2189105541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d722d210 con 0x55b7d71ec2f0 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.413+0000 7f8db1d9b640 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.413+0000 7f8db259c640 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71ec2f0 0x55b7d75b6fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8da40c6260 tx=0x7f8da4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7c36eaec02d1dc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.413+0000 7f8da0ff9640 1 -- 172.21.15.39:0/2189105541 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4035070 con 0x55b7d71ec2f0 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.414+0000 7f8da0ff9640 1 -- 172.21.15.39:0/2189105541 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8da40c9cf0 con 0x55b7d71ec2f0 2024-05-04T03:24:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.414+0000 7f8da0ff9640 1 -- 172.21.15.39:0/2189105541 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da40d3040 con 0x55b7d71ec2f0 2024-05-04T03:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.415+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8d9c0b2860 0x7f8d9c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.415+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2189105541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71ec2f0 msgr2=0x55b7d75b6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.415+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71ec2f0 0x55b7d75b6fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8da40c6260 tx=0x7f8da4009260 comp rx=0 tx=0).stop 2024-05-04T03:24:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.416+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2189105541 shutdown_connections 2024-05-04T03:24:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.416+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.416+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71ec2f0 0x55b7d75b6fe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.416+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2189105541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.416+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2189105541 >> 172.21.15.39:0/2189105541 conn(0x55b7d7209300 msgr2=0x55b7d75b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.416+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8d9c0b2860 0x7f8d9c0b4d70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55b7d75b7c30 tx=0x7f8dac004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.417+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2189105541 shutdown_connections 2024-05-04T03:24:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.417+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2189105541 wait complete. 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.417+0000 7f8dbf82d740 1 -- start start 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.417+0000 7f8dbf82d740 1 Processor -- start 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 0x55b7d75c0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d72151b0 0x55b7d75c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b7d75be6f0 con 0x55b7d71ec2f0 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b7d74d8370 con 0x55b7d71f44d0 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7d75be870 con 0x55b7d72151b0 2024-05-04T03:24:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8dbf82d740 10 client.0 _ll_register_callbacks cb 0x55b7d722ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8db2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d72151b0 0x55b7d75c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8db2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d72151b0 0x55b7d75c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:51740/0 (socket says 172.21.15.39:51740) 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8db2d9d640 1 -- 172.21.15.39:0/2298203294 learned_addr learned my addr 172.21.15.39:0/2298203294 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8db259c640 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.418+0000 7f8db1d9b640 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 0x55b7d75c0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8db2d9d640 1 -- 172.21.15.39:0/2298203294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 msgr2=0x55b7d75c0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 0x55b7d75c0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8db2d9d640 1 -- 172.21.15.39:0/2298203294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 msgr2=0x55b7d75c02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8db2d9d640 1 -- 172.21.15.39:0/2298203294 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d72f9620 con 0x55b7d72151b0 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d72151b0 0x55b7d75c7780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8dac01ab90 tx=0x7f8dac009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287fe0f6251595f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8d99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:24:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2298203294 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac02a060 con 0x55b7d72151b0 2024-05-04T03:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2298203294 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8dac00ecc0 con 0x55b7d72151b0 2024-05-04T03:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.419+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2298203294 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da40c9cf0 con 0x55b7d72151b0 2024-05-04T03:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.420+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2298203294 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8dac025400 con 0x55b7d72151b0 2024-05-04T03:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.420+0000 7f8dbf82d740 10 client.26464 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.420+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2298203294 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b7d75ca720 con 0x55b7d72151b0 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.420+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2298203294 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dac025a30 con 0x55b7d72151b0 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.420+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2298203294 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8dac018070 con 0x55b7d72151b0 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.421+0000 7f8dbf82d740 10 client.26464 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.421+0000 7f8dbf82d740 10 client.26464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.421+0000 7f8dbf82d740 20 client.26464 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.421+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b7d71d3d80 0x55b7d75cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.421+0000 7f8dbf82d740 4 client.26464 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.421+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2298203294 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8da40c9cf0 con 0x55b7d71d3d80 2024-05-04T03:24:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.421+0000 7f8db259c640 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b7d71d3d80 0x55b7d75cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.423+0000 7f8d99ffb640 10 client.26464 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.423+0000 7f8db259c640 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b7d71d3d80 0x55b7d75cdee0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87bd8c1d3b7f6c9 server_cookie=aedabc4a79974d95 in_seq=0 out_seq=0 2024-05-04T03:24:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.424+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2298203294 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f8da40c9cf0 con 0x55b7d71d3d80 2024-05-04T03:24:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.424+0000 7f8d99ffb640 10 client.26464 handle_command_reply: tid=0 2024-05-04T03:24:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.424+0000 7f8dbf82d740 1 client.26464 shutdown 2024-05-04T03:24:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2298203294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d72151b0 msgr2=0x55b7d75c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d72151b0 0x55b7d75c7780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8dac01ab90 tx=0x7f8dac009500 comp rx=0 tx=0).stop 2024-05-04T03:24:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2298203294 shutdown_connections 2024-05-04T03:24:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b7d71d3d80 0x55b7d75cdee0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d72151b0 0x55b7d75c7780 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 0x55b7d75c0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2298203294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8d99ffb640 0 client.26464 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2298203294 >> 172.21.15.39:0/2298203294 conn(0x55b7d7209300 msgr2=0x55b7d75b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2298203294 shutdown_connections 2024-05-04T03:24:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.425+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2298203294 wait complete. 2024-05-04T03:24:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.426+0000 7f8dbf82d740 20 client.26464 trim_cache size 0 max 16384 2024-05-04T03:24:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.455+0000 7f8dbf82d740 1 Processor -- start 2024-05-04T03:24:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.456+0000 7f8dbf82d740 1 -- start start 2024-05-04T03:24:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.456+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71ec2f0 0x55b7d75bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.456+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 0x55b7d75b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.456+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.457+0000 7f8db259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71ec2f0 0x55b7d75bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.457+0000 7f8db259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71ec2f0 0x55b7d75bb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:51752/0 (socket says 172.21.15.39:51752) 2024-05-04T03:24:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.457+0000 7f8db259c640 1 -- 172.21.15.39:0/1067698846 learned_addr learned my addr 172.21.15.39:0/1067698846 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.457+0000 7f8dbf82d740 1 -- 172.21.15.39:0/1067698846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9400e420 con 0x55b7d72151b0 2024-05-04T03:24:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.457+0000 7f8db1d9b640 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 0x55b7d75b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.457+0000 7f8dbf82d740 1 -- 172.21.15.39:0/1067698846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9400ff60 con 0x55b7d71f44d0 2024-05-04T03:24:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.457+0000 7f8dbf82d740 1 -- 172.21.15.39:0/1067698846 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9400eaf0 con 0x55b7d71ec2f0 2024-05-04T03:24:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.457+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.458+0000 7f8db2d9d640 1 -- 172.21.15.39:0/1067698846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71ec2f0 msgr2=0x55b7d75bb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.458+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71ec2f0 0x55b7d75bb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.458+0000 7f8db2d9d640 1 -- 172.21.15.39:0/1067698846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 msgr2=0x55b7d75b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.458+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 0x55b7d75b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.458+0000 7f8db2d9d640 1 -- 172.21.15.39:0/1067698846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d75b2d00 con 0x55b7d72151b0 2024-05-04T03:24:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.458+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75c76a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8dac030800 tx=0x7f8dac01a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4729e68d1062 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.459+0000 7f8d9bfff640 1 -- 172.21.15.39:0/1067698846 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac035070 con 0x55b7d72151b0 2024-05-04T03:24:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.459+0000 7f8d9bfff640 1 -- 172.21.15.39:0/1067698846 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dac0303d0 con 0x55b7d72151b0 2024-05-04T03:24:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.460+0000 7f8d9bfff640 1 -- 172.21.15.39:0/1067698846 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac025a30 con 0x55b7d72151b0 2024-05-04T03:24:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.461+0000 7f8dbf82d740 1 -- 172.21.15.39:0/1067698846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 msgr2=0x55b7d75c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.461+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75c76a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8dac030800 tx=0x7f8dac01a7d0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.461+0000 7f8dbf82d740 1 -- 172.21.15.39:0/1067698846 shutdown_connections 2024-05-04T03:24:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.461+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d72151b0 0x55b7d75c76a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.461+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d71f44d0 0x55b7d75b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.461+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/1067698846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71ec2f0 0x55b7d75bb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.461+0000 7f8dbf82d740 1 -- 172.21.15.39:0/1067698846 >> 172.21.15.39:0/1067698846 conn(0x55b7d7209300 msgr2=0x55b7d75b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.462+0000 7f8dbf82d740 1 -- 172.21.15.39:0/1067698846 shutdown_connections 2024-05-04T03:24:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.462+0000 7f8dbf82d740 1 -- 172.21.15.39:0/1067698846 wait complete. 2024-05-04T03:24:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.462+0000 7f8dbf82d740 1 -- start start 2024-05-04T03:24:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.462+0000 7f8dbf82d740 1 Processor -- start 2024-05-04T03:24:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.463+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.463+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75c6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.463+0000 7f8dbf82d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d72151b0 0x55b7d75c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.463+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9400f890 con 0x55b7d71ec2f0 2024-05-04T03:24:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.463+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9400f1c0 con 0x55b7d72151b0 2024-05-04T03:24:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.463+0000 7f8dbf82d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9400c400 con 0x55b7d71f44d0 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.463+0000 7f8dbf82d740 10 client.0 _ll_register_callbacks cb 0x7f8dac009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d72151b0 0x55b7d75c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d72151b0 0x55b7d75c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:35792/0 (socket says 172.21.15.39:35792) 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db2d9d640 1 -- 172.21.15.39:0/2111066504 learned_addr learned my addr 172.21.15.39:0/2111066504 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db1d9b640 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db259c640 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db2d9d640 1 -- 172.21.15.39:0/2111066504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 msgr2=0x55b7d75c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db2d9d640 1 -- 172.21.15.39:0/2111066504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 msgr2=0x55b7d75c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db2d9d640 1 -- 172.21.15.39:0/2111066504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d75c7710 con 0x55b7d72151b0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.464+0000 7f8db259c640 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.465+0000 7f8db2d9d640 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d72151b0 0x55b7d75c7150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00efd0 tx=0x7f8dac009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf2578cb6dae47d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.465+0000 7f8d99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.465+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2111066504 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac046060 con 0x55b7d72151b0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.465+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2111066504 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8dac0255a0 con 0x55b7d72151b0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.465+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2111066504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dac0303d0 con 0x55b7d72151b0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.465+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2111066504 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8dac035040 con 0x55b7d72151b0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.465+0000 7f8dbf82d740 10 client.26432 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.465+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2111066504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b7d7636770 con 0x55b7d72151b0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2111066504 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dac0303d0 con 0x55b7d72151b0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2111066504 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8dac018050 con 0x55b7d72151b0 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8dbf82d740 10 client.26432 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8dbf82d740 10 client.26432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8dbf82d740 20 client.26432 populate_metadata read hostname 'smithi039' 2024-05-04T03:24:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b7d71a91c0 0x55b7d7639160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8dbf82d740 4 client.26432 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2111066504 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7d75291e0 con 0x55b7d71a91c0 2024-05-04T03:24:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.466+0000 7f8db259c640 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b7d71a91c0 0x55b7d7639160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.467+0000 7f8db259c640 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b7d71a91c0 0x55b7d7639160 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5be1743aedcfb36 server_cookie=7e2ad29a6227b9d2 in_seq=0 out_seq=0 2024-05-04T03:24:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.467+0000 7f8d99ffb640 10 client.26432 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.480+0000 7f8d99ffb640 1 -- 172.21.15.39:0/2111066504 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7d75291e0 con 0x55b7d71a91c0 2024-05-04T03:24:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.480+0000 7f8d99ffb640 10 client.26432 handle_command_reply: tid=0 2024-05-04T03:24:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.480+0000 7f8dbf82d740 1 client.26432 shutdown 2024-05-04T03:24:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.480+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2111066504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d72151b0 msgr2=0x55b7d75c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.480+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d72151b0 0x55b7d75c7150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00efd0 tx=0x7f8dac009630 comp rx=0 tx=0).stop 2024-05-04T03:24:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.480+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2111066504 shutdown_connections 2024-05-04T03:24:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.480+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b7d71a91c0 0x55b7d7639160 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.480+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7d72151b0 0x55b7d75c7150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.481+0000 7f8d99ffb640 0 client.26432 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.481+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7d71f44d0 0x55b7d75c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.481+0000 7f8dbf82d740 1 --2- 172.21.15.39:0/2111066504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b7d71ec2f0 0x55b7d75c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.481+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2111066504 >> 172.21.15.39:0/2111066504 conn(0x55b7d7209300 msgr2=0x55b7d75b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.481+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2111066504 shutdown_connections 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.481+0000 7f8dbf82d740 1 -- 172.21.15.39:0/2111066504 wait complete. 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.481+0000 7f8dbf82d740 20 client.26432 trim_cache size 0 max 16384 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:24:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:24:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.482+0000 7f8db889f640 1 -- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8d9c0b2860 msgr2=0x7f8d9c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.482+0000 7f8db889f640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8d9c0b2860 0x7f8d9c0b4d70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55b7d75b7c30 tx=0x7f8dac004a40 comp rx=0 tx=0).stop 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.482+0000 7f8db889f640 1 -- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db4102560 msgr2=0x7f8db41a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.482+0000 7f8db889f640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db4102560 0x7f8db41a0850 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009230 tx=0x7f8da4004950 comp rx=0 tx=0).stop 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.482+0000 7f8db889f640 1 -- 172.21.15.39:0/1686710415 shutdown_connections 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.482+0000 7f8db889f640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8d9c0b2860 0x7f8d9c0b4d70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.482+0000 7f8db889f640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8db4103800 0x7f8db41a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.483+0000 7f8db889f640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8db4102eb0 0x7f8db41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.483+0000 7f8db889f640 1 --2- 172.21.15.39:0/1686710415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db4102560 0x7f8db41a0850 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.483+0000 7f8db889f640 1 -- 172.21.15.39:0/1686710415 >> 172.21.15.39:0/1686710415 conn(0x7f8db40fdd90 msgr2=0x7f8db40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.483+0000 7f8db889f640 1 -- 172.21.15.39:0/1686710415 shutdown_connections 2024-05-04T03:24:58.487 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:24:58.483+0000 7f8db889f640 1 -- 172.21.15.39:0/1686710415 wait complete. 2024-05-04T03:24:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:24:58 smithi066 ceph-mon[33264]: pgmap v811: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-05-04T03:24:58.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:24:58 smithi167 ceph-mon[33614]: pgmap v811: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-05-04T03:24:58.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:24:58 smithi039 ceph-mon[29715]: pgmap v811: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 1.45k op/s 2024-05-04T03:25:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:00 smithi066 ceph-mon[33264]: pgmap v812: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-05-04T03:25:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:00 smithi167 ceph-mon[33614]: pgmap v812: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-05-04T03:25:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:00 smithi039 ceph-mon[29715]: pgmap v812: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.47k op/s 2024-05-04T03:25:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 670.0 s, 1158.1 tps, lat 86.442 ms stddev 22.749 2024-05-04T03:25:02.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:02 smithi066 ceph-mon[33264]: pgmap v813: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-05-04T03:25:02.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:02 smithi167 ceph-mon[33614]: pgmap v813: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-05-04T03:25:02.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:02 smithi039 ceph-mon[29715]: pgmap v813: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.40k op/s 2024-05-04T03:25:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:04 smithi066 ceph-mon[33264]: pgmap v814: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:25:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:04 smithi167 ceph-mon[33614]: pgmap v814: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:25:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:04 smithi039 ceph-mon[29715]: pgmap v814: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-05-04T03:25:05.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:05 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:25:05.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:05 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:25:05.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:05 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:25:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 675.0 s, 1101.1 tps, lat 90.694 ms stddev 97.335 2024-05-04T03:25:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:06 smithi066 ceph-mon[33264]: pgmap v815: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:25:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:06 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:25:06.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:06 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:25:06.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:06 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:25:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:06 smithi167 ceph-mon[33614]: pgmap v815: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:25:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:25:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:25:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:06 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:25:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:06 smithi039 ceph-mon[29715]: pgmap v815: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:25:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:25:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:25:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:06 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:25:08.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:08 smithi066 ceph-mon[33264]: pgmap v816: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.53k op/s 2024-05-04T03:25:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:08 smithi167 ceph-mon[33614]: pgmap v816: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.53k op/s 2024-05-04T03:25:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:08 smithi039 ceph-mon[29715]: pgmap v816: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 15 MiB/s wr, 1.53k op/s 2024-05-04T03:25:10.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:10 smithi066 ceph-mon[33264]: pgmap v817: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-05-04T03:25:10.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:10 smithi167 ceph-mon[33614]: pgmap v817: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-05-04T03:25:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:10 smithi039 ceph-mon[29715]: pgmap v817: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 1.51k op/s 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.644+0000 7f8533e96640 1 -- 172.21.15.39:0/2280713516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f852c104770 msgr2=0x7f852c106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.644+0000 7f8533e96640 1 --2- 172.21.15.39:0/2280713516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f852c104770 0x7f852c106b90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8528009900 tx=0x7f852802a410 comp rx=0 tx=0).stop 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.644+0000 7f8533e96640 1 -- 172.21.15.39:0/2280713516 shutdown_connections 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.644+0000 7f8533e96640 1 --2- 172.21.15.39:0/2280713516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f852c109a30 0x7f852c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.644+0000 7f8533e96640 1 --2- 172.21.15.39:0/2280713516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f852c1070d0 0x7f852c1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.644+0000 7f8533e96640 1 --2- 172.21.15.39:0/2280713516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f852c104770 0x7f852c106b90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.644+0000 7f8533e96640 1 -- 172.21.15.39:0/2280713516 >> 172.21.15.39:0/2280713516 conn(0x7f852c0fdd90 msgr2=0x7f852c100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.645+0000 7f8533e96640 1 -- 172.21.15.39:0/2280713516 shutdown_connections 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.645+0000 7f8533e96640 1 -- 172.21.15.39:0/2280713516 wait complete. 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8533e96640 1 Processor -- start 2024-05-04T03:25:11.648 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8533e96640 1 -- start start 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8533e96640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f852c104770 0x7f852c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8533e96640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f852c1070d0 0x7f852c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8533e96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f852c109a30 0x7f852c1a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8531c32640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f852c104770 0x7f852c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8531c32640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f852c104770 0x7f852c19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:52234/0 (socket says 172.21.15.39:52234) 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8531431640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f852c1070d0 0x7f852c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8531431640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f852c1070d0 0x7f852c19c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:51816/0 (socket says 172.21.15.39:51816) 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8531c32640 1 -- 172.21.15.39:0/3774561658 learned_addr learned my addr 172.21.15.39:0/3774561658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.646+0000 7f8532433640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f852c109a30 0x7f852c1a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f852c102ae0 con 0x7f852c109a30 2024-05-04T03:25:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f852c102960 con 0x7f852c1070d0 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f852c102c60 con 0x7f852c104770 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8531431640 1 -- 172.21.15.39:0/3774561658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f852c104770 msgr2=0x7f852c19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8531431640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f852c104770 0x7f852c19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8531431640 1 -- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f852c109a30 msgr2=0x7f852c1a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8531431640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f852c109a30 0x7f852c1a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8531431640 1 -- 172.21.15.39:0/3774561658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c1a3e10 con 0x7f852c1070d0 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.647+0000 7f8531431640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f852c1070d0 0x7f852c19c810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f851c00e9f0 tx=0x7f851c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.648+0000 7f8522ffd640 1 -- 172.21.15.39:0/3774561658 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c00cd80 con 0x7f852c1070d0 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.648+0000 7f8522ffd640 1 -- 172.21.15.39:0/3774561658 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f851c0035f0 con 0x7f852c1070d0 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.648+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c1a4100 con 0x7f852c1070d0 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.648+0000 7f8522ffd640 1 -- 172.21.15.39:0/3774561658 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c010670 con 0x7f852c1070d0 2024-05-04T03:25:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.648+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c1a4640 con 0x7f852c1070d0 2024-05-04T03:25:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.651+0000 7f853ae27740 1 Processor -- start 2024-05-04T03:25:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f853ae27740 1 -- start start 2024-05-04T03:25:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f853ae27740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdf1a1b0 0x558dce2bba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f853ae27740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f853ae27740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef94d0 0x558dce2c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f853ae27740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558dce2205d0 con 0x558dcdef94d0 2024-05-04T03:25:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f853ae27740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558dce1d9890 con 0x558dcdf1a1b0 2024-05-04T03:25:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f853ae27740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558dce21c590 con 0x558dcdef12f0 2024-05-04T03:25:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f8532433640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdf1a1b0 0x558dce2bba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f8531c32640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f8531c32640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2bbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:52250/0 (socket says 172.21.15.39:52250) 2024-05-04T03:25:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f8531c32640 1 -- 172.21.15.39:0/1225364667 learned_addr learned my addr 172.21.15.39:0/1225364667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.652+0000 7f8522ffd640 1 -- 172.21.15.39:0/3774561658 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f851c010810 con 0x7f852c1070d0 2024-05-04T03:25:11.656 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.653+0000 7f8531c32640 1 -- 172.21.15.39:0/1225364667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdf1a1b0 msgr2=0x558dce2bba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.653+0000 7f8531c32640 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdf1a1b0 0x558dce2bba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.653+0000 7f8531c32640 1 -- 172.21.15.39:0/1225364667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef94d0 msgr2=0x558dce2c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.653+0000 7f8531431640 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef94d0 0x558dce2c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.653+0000 7f8531c32640 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef94d0 0x558dce2c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.653+0000 7f8531c32640 1 -- 172.21.15.39:0/1225364667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dcdf32210 con 0x558dcdef12f0 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.654+0000 7f8522ffd640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85180b28e0 0x7f85180b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.654+0000 7f8531431640 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef94d0 0x558dce2c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.654+0000 7f8531c32640 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2bbfe0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85280040f0 tx=0x7f8528004120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b949ed8264922c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.654+0000 7f8532433640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85180b28e0 0x7f85180b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.654+0000 7f8522ffd640 1 -- 172.21.15.39:0/3774561658 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f851c0e2400 con 0x7f852c1070d0 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.654+0000 7f8517fff640 1 -- 172.21.15.39:0/1225364667 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528035030 con 0x558dcdef12f0 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.655+0000 7f8517fff640 1 -- 172.21.15.39:0/1225364667 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8528040070 con 0x558dcdef12f0 2024-05-04T03:25:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.655+0000 7f8517fff640 1 -- 172.21.15.39:0/1225364667 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852803cb90 con 0x558dcdef12f0 2024-05-04T03:25:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f8532433640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85180b28e0 0x7f85180b4df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x558dce2bcc30 tx=0x7f8524009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 -- 172.21.15.39:0/1225364667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 msgr2=0x558dce2bbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2bbfe0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85280040f0 tx=0x7f8528004120 comp rx=0 tx=0).stop 2024-05-04T03:25:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 -- 172.21.15.39:0/1225364667 shutdown_connections 2024-05-04T03:25:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef94d0 0x558dce2c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2bbfe0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 --2- 172.21.15.39:0/1225364667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdf1a1b0 0x558dce2bba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 -- 172.21.15.39:0/1225364667 >> 172.21.15.39:0/1225364667 conn(0x558dcdf0e300 msgr2=0x558dce2b73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 -- 172.21.15.39:0/1225364667 shutdown_connections 2024-05-04T03:25:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.656+0000 7f853ae27740 1 -- 172.21.15.39:0/1225364667 wait complete. 2024-05-04T03:25:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.657+0000 7f853ae27740 1 -- start start 2024-05-04T03:25:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.657+0000 7f853ae27740 1 Processor -- start 2024-05-04T03:25:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.658+0000 7f853ae27740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef12f0 0x558dce2c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f853ae27740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f853ae27740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdf1a1b0 0x558dce2ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f8531c32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef12f0 0x558dce2c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f8531c32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef12f0 0x558dce2c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:51846/0 (socket says 172.21.15.39:51846) 2024-05-04T03:25:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f8531431640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f8531431640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:52256/0 (socket says 172.21.15.39:52256) 2024-05-04T03:25:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f8531c32640 1 -- 172.21.15.39:0/1850639072 learned_addr learned my addr 172.21.15.39:0/1850639072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f8532433640 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdf1a1b0 0x558dce2ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.659+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558dce2c38d0 con 0x558dcdf1a1b0 2024-05-04T03:25:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558dce1dd370 con 0x558dcdef12f0 2024-05-04T03:25:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558dce2c3a50 con 0x558dcdef94d0 2024-05-04T03:25:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f8531431640 1 -- 172.21.15.39:0/1850639072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef12f0 msgr2=0x558dce2c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f8531431640 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef12f0 0x558dce2c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f8531431640 1 -- 172.21.15.39:0/1850639072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdf1a1b0 msgr2=0x558dce2ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f8531431640 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdf1a1b0 0x558dce2ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f8531431640 1 -- 172.21.15.39:0/1850639072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dcdffe620 con 0x558dcdef94d0 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f853ae27740 10 client.0 _ll_register_callbacks cb 0x558dcdf31a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.660+0000 7f8532433640 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdf1a1b0 0x558dce2ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f8531c32640 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef12f0 0x558dce2c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f8531431640 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c5b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f851c0a96d0 tx=0x7f851c0a91c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=715e8a7ea6348099 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f85157fa640 10 client.26482 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8528040070 con 0x558dcdef94d0 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f85157fa640 1 -- 172.21.15.39:0/1850639072 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0a94e0 con 0x558dcdef94d0 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f85157fa640 1 -- 172.21.15.39:0/1850639072 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f851c003aa0 con 0x558dcdef94d0 2024-05-04T03:25:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f85157fa640 1 -- 172.21.15.39:0/1850639072 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f851c0aa0d0 con 0x558dcdef94d0 2024-05-04T03:25:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f853ae27740 10 client.26482 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.661+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=32}) -- 0x558dce2cf990 con 0x558dcdef94d0 2024-05-04T03:25:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.662+0000 7f85157fa640 1 -- 172.21.15.39:0/1850639072 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f851c0cc0e0 con 0x558dcdef94d0 2024-05-04T03:25:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.662+0000 7f853ae27740 10 client.26482 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.662+0000 7f853ae27740 10 client.26482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.662+0000 7f853ae27740 20 client.26482 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.662+0000 7f853ae27740 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558dcded8d80 0x558dce2d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.662+0000 7f853ae27740 4 client.26482 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:25:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.662+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dce22e1e0 con 0x558dcded8d80 2024-05-04T03:25:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.663+0000 7f8531c32640 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558dcded8d80 0x558dce2d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.664+0000 7f8531c32640 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558dcded8d80 0x558dce2d3050 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77bf1c876914d73 server_cookie=c55e2c9dbe10d8f7 in_seq=0 out_seq=0 2024-05-04T03:25:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.664+0000 7f85157fa640 10 client.26482 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f85157fa640 1 -- 172.21.15.39:0/1850639072 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558dce22e1e0 con 0x558dcded8d80 2024-05-04T03:25:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f85157fa640 10 client.26482 handle_command_reply: tid=0 2024-05-04T03:25:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 client.26482 shutdown 2024-05-04T03:25:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 msgr2=0x558dce2c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c5b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f851c0a96d0 tx=0x7f851c0a91c0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 shutdown_connections 2024-05-04T03:25:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558dcded8d80 0x558dce2d3050 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdf1a1b0 0x558dce2ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c5b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f85157fa640 0 client.26482 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 --2- 172.21.15.39:0/1850639072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef12f0 0x558dce2c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.665+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 >> 172.21.15.39:0/1850639072 conn(0x558dcdf0e300 msgr2=0x558dce2ba300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.666+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 shutdown_connections 2024-05-04T03:25:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.666+0000 7f853ae27740 1 -- 172.21.15.39:0/1850639072 wait complete. 2024-05-04T03:25:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.666+0000 7f853ae27740 20 client.26482 trim_cache size 0 max 16384 2024-05-04T03:25:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.697+0000 7f853ae27740 1 Processor -- start 2024-05-04T03:25:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.697+0000 7f853ae27740 1 -- start start 2024-05-04T03:25:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f853ae27740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcded8d80 0x558dce2bfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f853ae27740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef12f0 0x558dce2cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f853ae27740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f853ae27740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1ad320 con 0x558dcdef12f0 2024-05-04T03:25:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f853ae27740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1aee60 con 0x558dcded8d80 2024-05-04T03:25:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f853ae27740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1ad9f0 con 0x558dcdef94d0 2024-05-04T03:25:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f8531c32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcded8d80 0x558dce2bfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f8531c32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcded8d80 0x558dce2bfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:51862/0 (socket says 172.21.15.39:51862) 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f8531431640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef12f0 0x558dce2cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.698+0000 7f8532433640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f8531c32640 1 -- 172.21.15.39:0/2958629476 learned_addr learned my addr 172.21.15.39:0/2958629476 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f8531431640 1 -- 172.21.15.39:0/2958629476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 msgr2=0x558dce2c0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f8531431640 1 --2- 172.21.15.39:0/2958629476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f8531431640 1 -- 172.21.15.39:0/2958629476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcded8d80 msgr2=0x558dce2bfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f8531431640 1 --2- 172.21.15.39:0/2958629476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcded8d80 0x558dce2bfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f8531431640 1 -- 172.21.15.39:0/2958629476 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dce2bc0f0 con 0x558dcdef12f0 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f8531c32640 1 --2- 172.21.15.39:0/2958629476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcded8d80 0x558dce2bfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f8531431640 1 --2- 172.21.15.39:0/2958629476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef12f0 0x558dce2cc9d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f851c0a5a50 tx=0x7f851c0cc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a6cf42924a4e01 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.699+0000 7f85177fe640 1 -- 172.21.15.39:0/2958629476 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0addd0 con 0x558dcdef12f0 2024-05-04T03:25:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.700+0000 7f85177fe640 1 -- 172.21.15.39:0/2958629476 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8528040070 con 0x558dcdef12f0 2024-05-04T03:25:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.700+0000 7f85177fe640 1 -- 172.21.15.39:0/2958629476 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0cc0e0 con 0x558dcdef12f0 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 -- 172.21.15.39:0/2958629476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef12f0 msgr2=0x558dce2cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 --2- 172.21.15.39:0/2958629476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef12f0 0x558dce2cc9d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f851c0a5a50 tx=0x7f851c0cc070 comp rx=0 tx=0).stop 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 -- 172.21.15.39:0/2958629476 shutdown_connections 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 --2- 172.21.15.39:0/2958629476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef94d0 0x558dce2c0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 --2- 172.21.15.39:0/2958629476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcdef12f0 0x558dce2cc9d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 --2- 172.21.15.39:0/2958629476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcded8d80 0x558dce2bfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 -- 172.21.15.39:0/2958629476 >> 172.21.15.39:0/2958629476 conn(0x558dcdf0e300 msgr2=0x558dce2bb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 -- 172.21.15.39:0/2958629476 shutdown_connections 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.701+0000 7f853ae27740 1 -- 172.21.15.39:0/2958629476 wait complete. 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.702+0000 7f853ae27740 1 -- start start 2024-05-04T03:25:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.702+0000 7f853ae27740 1 Processor -- start 2024-05-04T03:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.702+0000 7f853ae27740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcded8d80 0x558dce2c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.702+0000 7f853ae27740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f853ae27740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef94d0 0x558dce339390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f853ae27740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1ae790 con 0x558dcded8d80 2024-05-04T03:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f853ae27740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1ae0c0 con 0x558dcdef94d0 2024-05-04T03:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f853ae27740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1ab300 con 0x558dcdef12f0 2024-05-04T03:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcded8d80 0x558dce2c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f853ae27740 10 client.0 _ll_register_callbacks cb 0x7f851c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcded8d80 0x558dce2c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58148/0 (socket says 172.21.15.39:58148) 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 -- 172.21.15.39:0/1102984005 learned_addr learned my addr 172.21.15.39:0/1102984005 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531431640 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8532433640 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef94d0 0x558dce339390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 -- 172.21.15.39:0/1102984005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 msgr2=0x558dce2ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 -- 172.21.15.39:0/1102984005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef94d0 msgr2=0x558dce339390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef94d0 0x558dce339390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 -- 172.21.15.39:0/1102984005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dce339900 con 0x558dcded8d80 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f8531c32640 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcded8d80 0x558dce2c9b80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8528009850 tx=0x7f8528004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1594cdc7628d6f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.703+0000 7f85157fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.704+0000 7f85157fa640 1 -- 172.21.15.39:0/1102984005 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528035040 con 0x558dcded8d80 2024-05-04T03:25:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.704+0000 7f85157fa640 1 -- 172.21.15.39:0/1102984005 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f852803cd60 con 0x558dcded8d80 2024-05-04T03:25:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.704+0000 7f853ae27740 1 -- 172.21.15.39:0/1102984005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8528040070 con 0x558dcded8d80 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.705+0000 7f85157fa640 1 -- 172.21.15.39:0/1102984005 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f852803c720 con 0x558dcded8d80 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.705+0000 7f853ae27740 10 client.16866 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.705+0000 7f853ae27740 1 -- 172.21.15.39:0/1102984005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558dce33abc0 con 0x558dcded8d80 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.705+0000 7f85157fa640 1 -- 172.21.15.39:0/1102984005 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8528040070 con 0x558dcded8d80 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.706+0000 7f85157fa640 1 -- 172.21.15.39:0/1102984005 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8528003ab0 con 0x558dcded8d80 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.706+0000 7f853ae27740 10 client.16866 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.706+0000 7f853ae27740 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.706+0000 7f853ae27740 20 client.16866 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.706+0000 7f853ae27740 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558dcdf1a1b0 0x558dce33d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.706+0000 7f853ae27740 4 client.16866 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.706+0000 7f853ae27740 1 -- 172.21.15.39:0/1102984005 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dce228050 con 0x558dcdf1a1b0 2024-05-04T03:25:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.706+0000 7f8531431640 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558dcdf1a1b0 0x558dce33d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.707+0000 7f8531431640 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558dcdf1a1b0 0x558dce33d160 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9545f9f1f55c26f server_cookie=29fc04bd1946c3ff in_seq=0 out_seq=0 2024-05-04T03:25:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.707+0000 7f85157fa640 10 client.16866 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.715+0000 7f85157fa640 1 -- 172.21.15.39:0/1102984005 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dce228050 con 0x558dcdf1a1b0 2024-05-04T03:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.715+0000 7f85157fa640 10 client.16866 handle_command_reply: tid=0 2024-05-04T03:25:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.715+0000 7f853ae27740 1 client.16866 shutdown 2024-05-04T03:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f853ae27740 1 -- 172.21.15.39:0/1102984005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcded8d80 msgr2=0x558dce2c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f853ae27740 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcded8d80 0x558dce2c9b80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8528009850 tx=0x7f8528004610 comp rx=0 tx=0).stop 2024-05-04T03:25:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f853ae27740 1 -- 172.21.15.39:0/1102984005 shutdown_connections 2024-05-04T03:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f853ae27740 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558dcdf1a1b0 0x558dce33d160 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f853ae27740 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558dcdef94d0 0x558dce339390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f853ae27740 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558dcdef12f0 0x558dce2ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f85157fa640 0 client.16866 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f853ae27740 1 --2- 172.21.15.39:0/1102984005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dcded8d80 0x558dce2c9b80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.716+0000 7f853ae27740 1 -- 172.21.15.39:0/1102984005 >> 172.21.15.39:0/1102984005 conn(0x558dcdf0e300 msgr2=0x558dce2bb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.717+0000 7f853ae27740 1 -- 172.21.15.39:0/1102984005 shutdown_connections 2024-05-04T03:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.717+0000 7f853ae27740 1 -- 172.21.15.39:0/1102984005 wait complete. 2024-05-04T03:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.717+0000 7f853ae27740 20 client.16866 trim_cache size 0 max 16384 2024-05-04T03:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:25:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85180b28e0 msgr2=0x7f85180b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85180b28e0 0x7f85180b4df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x558dce2bcc30 tx=0x7f8524009500 comp rx=0 tx=0).stop 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f852c1070d0 msgr2=0x7f852c19c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f852c1070d0 0x7f852c19c810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f851c00e9f0 tx=0x7f851c00eeb0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 shutdown_connections 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f85180b28e0 0x7f85180b4df0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f852c109a30 0x7f852c1a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f852c1070d0 0x7f852c19c810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 --2- 172.21.15.39:0/3774561658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f852c104770 0x7f852c19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 >> 172.21.15.39:0/3774561658 conn(0x7f852c0fdd90 msgr2=0x7f852c109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 shutdown_connections 2024-05-04T03:25:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:11.718+0000 7f8533e96640 1 -- 172.21.15.39:0/3774561658 wait complete. 2024-05-04T03:25:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 680.0 s, 1342.4 tps, lat 74.642 ms stddev 20.307 2024-05-04T03:25:12.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:12 smithi066 ceph-mon[33264]: pgmap v818: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:25:12.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:12 smithi167 ceph-mon[33614]: pgmap v818: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:25:12.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:12 smithi039 ceph-mon[29715]: pgmap v818: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:25:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:14 smithi066 ceph-mon[33264]: pgmap v819: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-05-04T03:25:14.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:14 smithi167 ceph-mon[33614]: pgmap v819: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-05-04T03:25:14.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:14 smithi039 ceph-mon[29715]: pgmap v819: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 1.60k op/s 2024-05-04T03:25:16.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 685.0 s, 1229.0 tps, lat 80.876 ms stddev 25.547 2024-05-04T03:25:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:16 smithi066 ceph-mon[33264]: pgmap v820: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.52k op/s 2024-05-04T03:25:16.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:16 smithi167 ceph-mon[33614]: pgmap v820: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.52k op/s 2024-05-04T03:25:16.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:16 smithi039 ceph-mon[29715]: pgmap v820: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.52k op/s 2024-05-04T03:25:18.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:18 smithi066 ceph-mon[33264]: pgmap v821: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:25:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:18 smithi167 ceph-mon[33614]: pgmap v821: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:25:18.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:18 smithi039 ceph-mon[29715]: pgmap v821: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.55k op/s 2024-05-04T03:25:20.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:20 smithi066 ceph-mon[33264]: pgmap v822: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-05-04T03:25:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:20 smithi167 ceph-mon[33614]: pgmap v822: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-05-04T03:25:20.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:20 smithi039 ceph-mon[29715]: pgmap v822: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-05-04T03:25:21.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 690.0 s, 1354.6 tps, lat 74.117 ms stddev 21.542 2024-05-04T03:25:22.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:22 smithi066 ceph-mon[33264]: pgmap v823: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.41k op/s 2024-05-04T03:25:22.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:22 smithi167 ceph-mon[33614]: pgmap v823: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.41k op/s 2024-05-04T03:25:22.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:22 smithi039 ceph-mon[29715]: pgmap v823: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.41k op/s 2024-05-04T03:25:24.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:24 smithi066 ceph-mon[33264]: pgmap v824: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:25:24.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:24 smithi167 ceph-mon[33614]: pgmap v824: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:25:24.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:24 smithi039 ceph-mon[29715]: pgmap v824: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-05-04T03:25:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 -- 172.21.15.39:0/2316931326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c103d90 msgr2=0x7f517c1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 --2- 172.21.15.39:0/2316931326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c103d90 0x7f517c1041a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f516c009860 tx=0x7f516c02a340 comp rx=0 tx=0).stop 2024-05-04T03:25:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 -- 172.21.15.39:0/2316931326 shutdown_connections 2024-05-04T03:25:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 --2- 172.21.15.39:0/2316931326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c108a90 0x7f517c10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 --2- 172.21.15.39:0/2316931326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c103d90 0x7f517c1041a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 --2- 172.21.15.39:0/2316931326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517c102b50 0x7f517c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 -- 172.21.15.39:0/2316931326 >> 172.21.15.39:0/2316931326 conn(0x7f517c0fdd40 msgr2=0x7f517c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 -- 172.21.15.39:0/2316931326 shutdown_connections 2024-05-04T03:25:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.945+0000 7f51828f9640 1 -- 172.21.15.39:0/2316931326 wait complete. 2024-05-04T03:25:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.946+0000 7f51828f9640 1 Processor -- start 2024-05-04T03:25:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.946+0000 7f51828f9640 1 -- start start 2024-05-04T03:25:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f51828f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c102b50 0x7f517c1134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f51828f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c103d90 0x7f517c1139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f51828f9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517c108a90 0x7f517c111b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f517b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c103d90 0x7f517c1139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f517b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c103d90 0x7f517c1139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:44368/0 (socket says 172.21.15.39:44368) 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f517b7fe640 1 -- 172.21.15.39:0/490554704 learned_addr learned my addr 172.21.15.39:0/490554704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f517bfff640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c102b50 0x7f517c1134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f5180e96640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517c108a90 0x7f517c111b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f517c10c050 con 0x7f517c102b50 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f517c10bed0 con 0x7f517c108a90 2024-05-04T03:25:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.947+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f517c10c1d0 con 0x7f517c103d90 2024-05-04T03:25:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.948+0000 7f517bfff640 1 -- 172.21.15.39:0/490554704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c103d90 msgr2=0x7f517c1139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.948+0000 7f517bfff640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c103d90 0x7f517c1139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.948+0000 7f517bfff640 1 -- 172.21.15.39:0/490554704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517c108a90 msgr2=0x7f517c111b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.948+0000 7f517bfff640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517c108a90 0x7f517c111b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.948+0000 7f517bfff640 1 -- 172.21.15.39:0/490554704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c112270 con 0x7f517c102b50 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.949+0000 7f5180e96640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517c108a90 0x7f517c111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.949+0000 7f517bfff640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c102b50 0x7f517c1134a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f517c103750 tx=0x7f517000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.949+0000 7f51797fa640 1 -- 172.21.15.39:0/490554704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170017070 con 0x7f517c102b50 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.949+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517c112560 con 0x7f517c102b50 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.949+0000 7f51797fa640 1 -- 172.21.15.39:0/490554704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5170009c00 con 0x7f517c102b50 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.950+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517c1b1300 con 0x7f517c102b50 2024-05-04T03:25:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.950+0000 7f51797fa640 1 -- 172.21.15.39:0/490554704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170021060 con 0x7f517c102b50 2024-05-04T03:25:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.953+0000 7f51797fa640 1 -- 172.21.15.39:0/490554704 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f517001a070 con 0x7f517c102b50 2024-05-04T03:25:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.957+0000 7f51797fa640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f51640b2850 0x7f51640b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.957+0000 7f517b7fe640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f51640b2850 0x7f51640b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.957+0000 7f51797fa640 1 -- 172.21.15.39:0/490554704 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f51700e2ab0 con 0x7f517c102b50 2024-05-04T03:25:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.963+0000 7f517b7fe640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f51640b2850 0x7f51640b4d60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f516c0057c0 tx=0x7f516c005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.964+0000 7f51897aa740 1 Processor -- start 2024-05-04T03:25:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.964+0000 7f51897aa740 1 -- start start 2024-05-04T03:25:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.964+0000 7f51897aa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61571b0 0x5603b64f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.964+0000 7f51897aa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 0x5603b64f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.964+0000 7f51897aa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b64ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.964+0000 7f51897aa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5603b645d5d0 con 0x5603b61571b0 2024-05-04T03:25:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.964+0000 7f51897aa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5603b6416890 con 0x5603b612e2f0 2024-05-04T03:25:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.964+0000 7f51897aa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5603b6459590 con 0x5603b61364d0 2024-05-04T03:25:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f517bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61571b0 0x5603b64f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f517b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 0x5603b64f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f517bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61571b0 0x5603b64f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60050/0 (socket says 172.21.15.39:60050) 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f5180e96640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b64ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f517bfff640 1 -- 172.21.15.39:0/238394544 learned_addr learned my addr 172.21.15.39:0/238394544 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f5180e96640 1 -- 172.21.15.39:0/238394544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 msgr2=0x5603b64f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f5180e96640 1 --2- 172.21.15.39:0/238394544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 0x5603b64f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f5180e96640 1 -- 172.21.15.39:0/238394544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61571b0 msgr2=0x5603b64f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f5180e96640 1 --2- 172.21.15.39:0/238394544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61571b0 0x5603b64f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.965+0000 7f5180e96640 1 -- 172.21.15.39:0/238394544 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b616f210 con 0x5603b61364d0 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.966+0000 7f5180e96640 1 --2- 172.21.15.39:0/238394544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b64ffee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5174009fc0 tx=0x7f517400eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a95da90fe5f66d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.966+0000 7f51627fc640 1 -- 172.21.15.39:0/238394544 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517401d070 con 0x5603b61364d0 2024-05-04T03:25:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.966+0000 7f51627fc640 1 -- 172.21.15.39:0/238394544 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5174003e00 con 0x5603b61364d0 2024-05-04T03:25:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.966+0000 7f51627fc640 1 -- 172.21.15.39:0/238394544 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740045d0 con 0x5603b61364d0 2024-05-04T03:25:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 -- 172.21.15.39:0/238394544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 msgr2=0x5603b64ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 --2- 172.21.15.39:0/238394544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b64ffee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5174009fc0 tx=0x7f517400eb70 comp rx=0 tx=0).stop 2024-05-04T03:25:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 -- 172.21.15.39:0/238394544 shutdown_connections 2024-05-04T03:25:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 --2- 172.21.15.39:0/238394544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b64ffee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 --2- 172.21.15.39:0/238394544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 0x5603b64f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 --2- 172.21.15.39:0/238394544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61571b0 0x5603b64f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 -- 172.21.15.39:0/238394544 >> 172.21.15.39:0/238394544 conn(0x5603b614b300 msgr2=0x5603b64f43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 -- 172.21.15.39:0/238394544 shutdown_connections 2024-05-04T03:25:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.968+0000 7f51897aa740 1 -- 172.21.15.39:0/238394544 wait complete. 2024-05-04T03:25:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.969+0000 7f51897aa740 1 -- start start 2024-05-04T03:25:24.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.969+0000 7f51897aa740 1 Processor -- start 2024-05-04T03:25:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f51897aa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b612e2f0 0x5603b6502700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f51897aa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b6502c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f517bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b612e2f0 0x5603b6502700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f517bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b612e2f0 0x5603b6502700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:44396/0 (socket says 172.21.15.39:44396) 2024-05-04T03:25:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f517bfff640 1 -- 172.21.15.39:0/3018908553 learned_addr learned my addr 172.21.15.39:0/3018908553 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f51897aa740 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b61571b0 0x5603b6509ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5603b6500930 con 0x5603b61364d0 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5603b641a370 con 0x5603b61571b0 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5603b6500ab0 con 0x5603b612e2f0 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f517b7fe640 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b6502c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.970+0000 7f51897aa740 10 client.0 _ll_register_callbacks cb 0x5603b616ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f5180e96640 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b61571b0 0x5603b6509ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f517bfff640 1 -- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b61571b0 msgr2=0x5603b6509ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f517bfff640 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b61571b0 0x5603b6509ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f517bfff640 1 -- 172.21.15.39:0/3018908553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 msgr2=0x5603b6502c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f517bfff640 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b6502c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f517bfff640 1 -- 172.21.15.39:0/3018908553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b623b620 con 0x5603b612e2f0 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f517b7fe640 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b6502c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f517bfff640 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b612e2f0 0x5603b6502700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51700a73a0 tx=0x7f51700aa020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d1809cb35f75d6a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f515b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:25:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.971+0000 7f515b7fe640 1 -- 172.21.15.39:0/3018908553 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51700c4060 con 0x5603b612e2f0 2024-05-04T03:25:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.972+0000 7f515b7fe640 1 -- 172.21.15.39:0/3018908553 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f51700a63c0 con 0x5603b612e2f0 2024-05-04T03:25:24.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.972+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5174003e00 con 0x5603b612e2f0 2024-05-04T03:25:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.972+0000 7f515b7fe640 1 -- 172.21.15.39:0/3018908553 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f51700cb050 con 0x5603b612e2f0 2024-05-04T03:25:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.972+0000 7f51897aa740 10 client.26500 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.972+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603b650c930 con 0x5603b612e2f0 2024-05-04T03:25:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.973+0000 7f515b7fe640 1 -- 172.21.15.39:0/3018908553 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5174003e00 con 0x5603b612e2f0 2024-05-04T03:25:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.973+0000 7f515b7fe640 1 -- 172.21.15.39:0/3018908553 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f51700a8e00 con 0x5603b612e2f0 2024-05-04T03:25:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.973+0000 7f51897aa740 10 client.26500 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.973+0000 7f51897aa740 10 client.26500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.973+0000 7f51897aa740 20 client.26500 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.973+0000 7f51897aa740 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5603b6115d80 0x5603b650ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.973+0000 7f51897aa740 4 client.26500 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:25:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.973+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603b646b1e0 con 0x5603b6115d80 2024-05-04T03:25:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.974+0000 7f517b7fe640 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5603b6115d80 0x5603b650ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.974+0000 7f515b7fe640 10 client.26500 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.974+0000 7f517b7fe640 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5603b6115d80 0x5603b650ffe0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b14c4b5f4d051 server_cookie=f01a6ba46cbc1379 in_seq=0 out_seq=0 2024-05-04T03:25:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.976+0000 7f515b7fe640 1 -- 172.21.15.39:0/3018908553 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603b646b1e0 con 0x5603b6115d80 2024-05-04T03:25:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.976+0000 7f515b7fe640 10 client.26500 handle_command_reply: tid=0 2024-05-04T03:25:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.977+0000 7f51897aa740 1 client.26500 shutdown 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b612e2f0 msgr2=0x5603b6502700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f51897aa740 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b612e2f0 0x5603b6502700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51700a73a0 tx=0x7f51700aa020 comp rx=0 tx=0).stop 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 shutdown_connections 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f51897aa740 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5603b6115d80 0x5603b650ffe0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f51897aa740 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b61571b0 0x5603b6509ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f515b7fe640 0 client.26500 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f51897aa740 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b6502c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f51897aa740 1 --2- 172.21.15.39:0/3018908553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b612e2f0 0x5603b6502700 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.978+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 >> 172.21.15.39:0/3018908553 conn(0x5603b614b300 msgr2=0x5603b64f7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.979+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 shutdown_connections 2024-05-04T03:25:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.979+0000 7f51897aa740 1 -- 172.21.15.39:0/3018908553 wait complete. 2024-05-04T03:25:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:24.979+0000 7f51897aa740 20 client.26500 trim_cache size 0 max 16384 2024-05-04T03:25:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.007+0000 7f51897aa740 1 Processor -- start 2024-05-04T03:25:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.007+0000 7f51897aa740 1 -- start start 2024-05-04T03:25:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.007+0000 7f51897aa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b6115d80 0x5603b64fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.007+0000 7f51897aa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 0x5603b64fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.007+0000 7f51897aa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b64f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.007+0000 7f51897aa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1b9f70 con 0x5603b61364d0 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.007+0000 7f51897aa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1bbab0 con 0x5603b612e2f0 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.007+0000 7f51897aa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1ba640 con 0x5603b6115d80 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f5180e96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b64f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f5180e96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b64f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60068/0 (socket says 172.21.15.39:60068) 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f5180e96640 1 -- 172.21.15.39:0/3343300058 learned_addr learned my addr 172.21.15.39:0/3343300058 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f517b7fe640 1 --2- 172.21.15.39:0/3343300058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 0x5603b64fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f5180e96640 1 -- 172.21.15.39:0/3343300058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b6115d80 msgr2=0x5603b64fcb60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:25:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f5180e96640 1 --2- 172.21.15.39:0/3343300058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b6115d80 0x5603b64fcb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f5180e96640 1 -- 172.21.15.39:0/3343300058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 msgr2=0x5603b64fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f5180e96640 1 --2- 172.21.15.39:0/3343300058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 0x5603b64fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.008+0000 7f5180e96640 1 -- 172.21.15.39:0/3343300058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b64f4d00 con 0x5603b61364d0 2024-05-04T03:25:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.009+0000 7f5180e96640 1 --2- 172.21.15.39:0/3343300058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b64f3c20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5174009850 tx=0x7f51740097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfecf565dc158741 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.010+0000 7f5161ffb640 1 -- 172.21.15.39:0/3343300058 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517401d040 con 0x5603b61364d0 2024-05-04T03:25:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.010+0000 7f5161ffb640 1 -- 172.21.15.39:0/3343300058 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f517400cce0 con 0x5603b61364d0 2024-05-04T03:25:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.011+0000 7f5161ffb640 1 -- 172.21.15.39:0/3343300058 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174003da0 con 0x5603b61364d0 2024-05-04T03:25:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 -- 172.21.15.39:0/3343300058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 msgr2=0x5603b64f3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 --2- 172.21.15.39:0/3343300058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b64f3c20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5174009850 tx=0x7f51740097e0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 -- 172.21.15.39:0/3343300058 shutdown_connections 2024-05-04T03:25:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 --2- 172.21.15.39:0/3343300058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b61364d0 0x5603b64f3c20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 --2- 172.21.15.39:0/3343300058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b612e2f0 0x5603b64fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 --2- 172.21.15.39:0/3343300058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b6115d80 0x5603b64fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 -- 172.21.15.39:0/3343300058 >> 172.21.15.39:0/3343300058 conn(0x5603b614b300 msgr2=0x5603b65026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 -- 172.21.15.39:0/3343300058 shutdown_connections 2024-05-04T03:25:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.012+0000 7f51897aa740 1 -- 172.21.15.39:0/3343300058 wait complete. 2024-05-04T03:25:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.013+0000 7f51897aa740 1 -- start start 2024-05-04T03:25:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.013+0000 7f51897aa740 1 Processor -- start 2024-05-04T03:25:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f51897aa740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b6115d80 0x5603b6506460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f51897aa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b612e2f0 0x5603b65069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f51897aa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b6507880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f51897aa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1bb3e0 con 0x5603b612e2f0 2024-05-04T03:25:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f51897aa740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1bad10 con 0x5603b6115d80 2024-05-04T03:25:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f51897aa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1b7f50 con 0x5603b61364d0 2024-05-04T03:25:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f51897aa740 10 client.0 _ll_register_callbacks cb 0x7f51700a98a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f517bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b6115d80 0x5603b6506460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f517b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b612e2f0 0x5603b65069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f517bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b6115d80 0x5603b6506460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:58798/0 (socket says 172.21.15.39:58798) 2024-05-04T03:25:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f517bfff640 1 -- 172.21.15.39:0/4217035244 learned_addr learned my addr 172.21.15.39:0/4217035244 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.014+0000 7f5180e96640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b6507880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f517bfff640 1 -- 172.21.15.39:0/4217035244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 msgr2=0x5603b6507880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f517bfff640 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b6507880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f517bfff640 1 -- 172.21.15.39:0/4217035244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b612e2f0 msgr2=0x5603b65069a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f517bfff640 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b612e2f0 0x5603b65069a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f517bfff640 1 -- 172.21.15.39:0/4217035244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b6507e40 con 0x5603b6115d80 2024-05-04T03:25:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f517b7fe640 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b612e2f0 0x5603b65069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f5180e96640 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b6507880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f517bfff640 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b6115d80 0x5603b6506460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5170010fd0 tx=0x7f51700a65f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc53e5ea5eea43a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f515b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f515b7fe640 1 -- 172.21.15.39:0/4217035244 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51700c4050 con 0x5603b6115d80 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f515b7fe640 1 -- 172.21.15.39:0/4217035244 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f51700af700 con 0x5603b6115d80 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.015+0000 7f51897aa740 1 -- 172.21.15.39:0/4217035244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517400cce0 con 0x5603b6115d80 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f515b7fe640 1 -- 172.21.15.39:0/4217035244 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5170088070 con 0x5603b6115d80 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f51897aa740 10 client.26468 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f51897aa740 1 -- 172.21.15.39:0/4217035244 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603b65090d0 con 0x5603b6115d80 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f515b7fe640 1 -- 172.21.15.39:0/4217035244 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f517400cce0 con 0x5603b6115d80 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f515b7fe640 1 -- 172.21.15.39:0/4217035244 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f51700e0a20 con 0x5603b6115d80 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f51897aa740 10 client.26468 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f51897aa740 10 client.26468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f51897aa740 20 client.26468 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f51897aa740 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5603b61571b0 0x5603b657a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f51897aa740 4 client.26468 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:25:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.016+0000 7f51897aa740 1 -- 172.21.15.39:0/4217035244 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603b6465050 con 0x5603b61571b0 2024-05-04T03:25:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.017+0000 7f517b7fe640 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5603b61571b0 0x5603b657a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.018+0000 7f517b7fe640 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5603b61571b0 0x5603b657a620 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e3798afaf50459 server_cookie=32add5da1aaf164c in_seq=0 out_seq=0 2024-05-04T03:25:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.018+0000 7f515b7fe640 10 client.26468 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.026+0000 7f515b7fe640 1 -- 172.21.15.39:0/4217035244 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603b6465050 con 0x5603b61571b0 2024-05-04T03:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.026+0000 7f515b7fe640 10 client.26468 handle_command_reply: tid=0 2024-05-04T03:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.026+0000 7f51897aa740 1 client.26468 shutdown 2024-05-04T03:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 -- 172.21.15.39:0/4217035244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b6115d80 msgr2=0x5603b6506460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b6115d80 0x5603b6506460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5170010fd0 tx=0x7f51700a65f0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 -- 172.21.15.39:0/4217035244 shutdown_connections 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5603b61571b0 0x5603b657a620 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603b61364d0 0x5603b6507880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603b612e2f0 0x5603b65069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f515b7fe640 0 client.26468 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 --2- 172.21.15.39:0/4217035244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5603b6115d80 0x5603b6506460 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 -- 172.21.15.39:0/4217035244 >> 172.21.15.39:0/4217035244 conn(0x5603b614b300 msgr2=0x5603b65026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 -- 172.21.15.39:0/4217035244 shutdown_connections 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 1 -- 172.21.15.39:0/4217035244 wait complete. 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.027+0000 7f51897aa740 20 client.26468 trim_cache size 0 max 16384 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:25:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:25:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:25:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f51640b2850 msgr2=0x7f51640b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f51640b2850 0x7f51640b4d60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f516c0057c0 tx=0x7f516c005c90 comp rx=0 tx=0).stop 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c102b50 msgr2=0x7f517c1134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c102b50 0x7f517c1134a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f517c103750 tx=0x7f517000dde0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 shutdown_connections 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f51640b2850 0x7f51640b4d60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f517c108a90 0x7f517c111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f517c103d90 0x7f517c1139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 --2- 172.21.15.39:0/490554704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f517c102b50 0x7f517c1134a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 >> 172.21.15.39:0/490554704 conn(0x7f517c0fdd40 msgr2=0x7f517c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.028+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 shutdown_connections 2024-05-04T03:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:25.029+0000 7f51828f9640 1 -- 172.21.15.39:0/490554704 wait complete. 2024-05-04T03:25:26.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 695.0 s, 1409.8 tps, lat 70.952 ms stddev 20.744 2024-05-04T03:25:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:26 smithi167 ceph-mon[33614]: pgmap v825: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-04T03:25:26.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:26 smithi039 ceph-mon[29715]: pgmap v825: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-04T03:25:27.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:26 smithi066 ceph-mon[33264]: pgmap v825: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-05-04T03:25:28.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:28 smithi167 ceph-mon[33614]: pgmap v826: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:25:28.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:28 smithi039 ceph-mon[29715]: pgmap v826: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:25:29.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:28 smithi066 ceph-mon[33264]: pgmap v826: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 13 MiB/s wr, 1.38k op/s 2024-05-04T03:25:30.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:30 smithi167 ceph-mon[33614]: pgmap v827: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:25:30.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:30 smithi039 ceph-mon[29715]: pgmap v827: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:25:31.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:30 smithi066 ceph-mon[33264]: pgmap v827: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 1.43k op/s 2024-05-04T03:25:31.165 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-04T03:25:31.166 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-05-04T03:25:31.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:25:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:25:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 700.0 s, 1327.6 tps, lat 75.235 ms stddev 23.119 2024-05-04T03:25:32.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:32 smithi167 ceph-mon[33614]: pgmap v828: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-04T03:25:32.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:32 smithi039 ceph-mon[29715]: pgmap v828: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-04T03:25:33.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:32 smithi066 ceph-mon[33264]: pgmap v828: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-05-04T03:25:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.801+0000 7fe44639a700 1 -- 172.21.15.39:0/264489090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4401045f0 msgr2=0x7fe440104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.801+0000 7fe44639a700 1 --2- 172.21.15.39:0/264489090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4401045f0 0x7fe440104a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe43800b0d0 tx=0x7fe43800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:25:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.801+0000 7fe44639a700 1 -- 172.21.15.39:0/264489090 shutdown_connections 2024-05-04T03:25:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.801+0000 7fe44639a700 1 --2- 172.21.15.39:0/264489090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe440109870 0x7fe4401111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.801+0000 7fe44639a700 1 --2- 172.21.15.39:0/264489090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4401045f0 0x7fe440104a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.801+0000 7fe44639a700 1 --2- 172.21.15.39:0/264489090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440108f20 0x7fe440109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.801+0000 7fe44639a700 1 -- 172.21.15.39:0/264489090 >> 172.21.15.39:0/264489090 conn(0x7fe440100040 msgr2=0x7fe440102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:33.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.802+0000 7fe44639a700 1 -- 172.21.15.39:0/264489090 shutdown_connections 2024-05-04T03:25:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.804+0000 7fe44639a700 1 -- 172.21.15.39:0/264489090 wait complete. 2024-05-04T03:25:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.805+0000 7fe44639a700 1 Processor -- start 2024-05-04T03:25:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.805+0000 7fe44639a700 1 -- start start 2024-05-04T03:25:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.805+0000 7fe44639a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4401045f0 0x7fe4401a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.805+0000 7fe44639a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440108f20 0x7fe4401a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe43f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440108f20 0x7fe4401a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe43f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440108f20 0x7fe4401a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43020/0 (socket says 172.21.15.39:43020) 2024-05-04T03:25:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe43ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4401045f0 0x7fe4401a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe43ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4401045f0 0x7fe4401a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:51962/0 (socket says 172.21.15.39:51962) 2024-05-04T03:25:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe44639a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440109870 0x7fe44019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe43f7fe700 1 -- 172.21.15.39:0/3340003860 learned_addr learned my addr 172.21.15.39:0/3340003860 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401143f0 con 0x7fe440108f20 2024-05-04T03:25:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe440114270 con 0x7fe4401045f0 2024-05-04T03:25:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe440114570 con 0x7fe440109870 2024-05-04T03:25:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.806+0000 7fe444937700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440109870 0x7fe44019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.807+0000 7fe444937700 1 -- 172.21.15.39:0/3340003860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4401045f0 msgr2=0x7fe4401a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.807+0000 7fe444937700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4401045f0 0x7fe4401a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.807+0000 7fe444937700 1 -- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440108f20 msgr2=0x7fe4401a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.807+0000 7fe444937700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440108f20 0x7fe4401a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.807+0000 7fe444937700 1 -- 172.21.15.39:0/3340003860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44019b0c0 con 0x7fe440109870 2024-05-04T03:25:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.807+0000 7fe444937700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440109870 0x7fe44019aa10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe43000fe20 tx=0x7fe43000def0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.808+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3340003860 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43001d070 con 0x7fe440109870 2024-05-04T03:25:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.808+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3340003860 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe430010bb0 con 0x7fe440109870 2024-05-04T03:25:33.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.808+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3340003860 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300165a0 con 0x7fe440109870 2024-05-04T03:25:33.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.809+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44019b3b0 con 0x7fe440109870 2024-05-04T03:25:33.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.809+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4400759f0 con 0x7fe440109870 2024-05-04T03:25:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.810+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe440105aa0 con 0x7fe440109870 2024-05-04T03:25:33.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.812+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3340003860 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe430020070 con 0x7fe440109870 2024-05-04T03:25:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.814+0000 7fe43d7fa700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4280b2a50 0x7fe4280b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.814+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3340003860 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe4300e56c0 con 0x7fe440109870 2024-05-04T03:25:33.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.814+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4280b2a50 0x7fe4280b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:33.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.815+0000 7fe43ffff700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4280b2a50 0x7fe4280b4f60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe438009900 tx=0x7fe43800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:33.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:33.818+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3340003860 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe4300ad390 con 0x7fe440109870 2024-05-04T03:25:34.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.308+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4400512c0 con 0x7fe440109870 2024-05-04T03:25:34.312 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:25:34.312 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:25:34.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.309+0000 7fe43d7fa700 1 -- 172.21.15.39:0/3340003860 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7fe4300b5410 con 0x7fe440109870 2024-05-04T03:25:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4280b2a50 msgr2=0x7fe4280b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4280b2a50 0x7fe4280b4f60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe438009900 tx=0x7fe43800b040 comp rx=0 tx=0).stop 2024-05-04T03:25:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440109870 msgr2=0x7fe44019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440109870 0x7fe44019aa10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe43000fe20 tx=0x7fe43000def0 comp rx=0 tx=0).stop 2024-05-04T03:25:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 shutdown_connections 2024-05-04T03:25:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe4280b2a50 0x7fe4280b4f60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe440109870 0x7fe44019aa10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe4401045f0 0x7fe4401a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 --2- 172.21.15.39:0/3340003860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440108f20 0x7fe4401a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 >> 172.21.15.39:0/3340003860 conn(0x7fe440100040 msgr2=0x7fe440108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 shutdown_connections 2024-05-04T03:25:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:34.313+0000 7fe44639a700 1 -- 172.21.15.39:0/3340003860 wait complete. 2024-05-04T03:25:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:25:34.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:34 smithi167 ceph-mon[33614]: pgmap v829: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.50k op/s 2024-05-04T03:25:34.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:34 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3340003860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:25:34.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:34 smithi039 ceph-mon[29715]: pgmap v829: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.50k op/s 2024-05-04T03:25:34.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:34 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3340003860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:25:35.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:34 smithi066 ceph-mon[33264]: pgmap v829: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 1.50k op/s 2024-05-04T03:25:35.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:34 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3340003860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:25:35.174 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 85.35 --await-for 42.0' 2024-05-04T03:25:35.175 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 85.35 --await-for 42.0 2024-05-04T03:25:35.667 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:25:36.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 705.0 s, 1154.8 tps, lat 86.305 ms stddev 26.896 2024-05-04T03:25:36.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:36 smithi039 ceph-mon[29715]: pgmap v830: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-05-04T03:25:36.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:36 smithi167 ceph-mon[33614]: pgmap v830: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-05-04T03:25:37.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:36 smithi066 ceph-mon[33264]: pgmap v830: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-05-04T03:25:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.259+0000 7f9e06e51640 1 -- 172.21.15.39:0/626201434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00103d90 msgr2=0x7f9e001041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.259+0000 7f9e06e51640 1 --2- 172.21.15.39:0/626201434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00103d90 0x7f9e001041a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009900 tx=0x7f9dec02a410 comp rx=0 tx=0).stop 2024-05-04T03:25:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.260+0000 7f9e06e51640 1 -- 172.21.15.39:0/626201434 shutdown_connections 2024-05-04T03:25:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.260+0000 7f9e06e51640 1 --2- 172.21.15.39:0/626201434 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e00108a90 0x7f9e0010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.260+0000 7f9e06e51640 1 --2- 172.21.15.39:0/626201434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00103d90 0x7f9e001041a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.260+0000 7f9e06e51640 1 --2- 172.21.15.39:0/626201434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e00102b50 0x7f9e00103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.260+0000 7f9e06e51640 1 -- 172.21.15.39:0/626201434 >> 172.21.15.39:0/626201434 conn(0x7f9e000fdd40 msgr2=0x7f9e001001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.260+0000 7f9e06e51640 1 -- 172.21.15.39:0/626201434 shutdown_connections 2024-05-04T03:25:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.260+0000 7f9e06e51640 1 -- 172.21.15.39:0/626201434 wait complete. 2024-05-04T03:25:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9e06e51640 1 Processor -- start 2024-05-04T03:25:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9e06e51640 1 -- start start 2024-05-04T03:25:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9e06e51640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00102b50 0x7f9e0019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9e06e51640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e00103d90 0x7f9e0019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9e06e51640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e00108a90 0x7f9e001a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9e06e51640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00111870 con 0x7f9e00108a90 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e00103d90 0x7f9e0019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9e04bed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00102b50 0x7f9e0019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e00103d90 0x7f9e0019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:33580/0 (socket says 172.21.15.39:33580) 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9e04bed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00102b50 0x7f9e0019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:56730/0 (socket says 172.21.15.39:56730) 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.261+0000 7f9dfffff640 1 -- 172.21.15.39:0/1017247216 learned_addr learned my addr 172.21.15.39:0/1017247216 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001116f0 con 0x7f9e00103d90 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001119f0 con 0x7f9e00102b50 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e053ee640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e00108a90 0x7f9e001a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e04bed640 1 -- 172.21.15.39:0/1017247216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e00103d90 msgr2=0x7f9e0019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e04bed640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e00103d90 0x7f9e0019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e04bed640 1 -- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e00108a90 msgr2=0x7f9e001a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e04bed640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e00108a90 0x7f9e001a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e04bed640 1 -- 172.21.15.39:0/1017247216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e001a3fa0 con 0x7f9e00102b50 2024-05-04T03:25:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.262+0000 7f9e04bed640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00102b50 0x7f9e0019c460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9e00102420 tx=0x7f9df4009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.263+0000 7f9dfdffb640 1 -- 172.21.15.39:0/1017247216 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df400eab0 con 0x7f9e00102b50 2024-05-04T03:25:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.263+0000 7f9dfdffb640 1 -- 172.21.15.39:0/1017247216 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9df4019070 con 0x7f9e00102b50 2024-05-04T03:25:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.263+0000 7f9dfdffb640 1 -- 172.21.15.39:0/1017247216 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df4014d70 con 0x7f9e00102b50 2024-05-04T03:25:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.263+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e001a4290 con 0x7f9e00102b50 2024-05-04T03:25:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.263+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e001a46d0 con 0x7f9e00102b50 2024-05-04T03:25:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.267+0000 7f9e0dde3740 1 Processor -- start 2024-05-04T03:25:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.267+0000 7f9dfdffb640 1 -- 172.21.15.39:0/1017247216 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9df401e460 con 0x7f9e00102b50 2024-05-04T03:25:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9e0dde3740 1 -- start start 2024-05-04T03:25:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea381b0 0x55a9bede2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bede27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea174d0 0x55a9bede96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a9bed3e5d0 con 0x55a9bea0f2f0 2024-05-04T03:25:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9becf7890 con 0x55a9bea381b0 2024-05-04T03:25:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a9bed3a590 con 0x55a9bea174d0 2024-05-04T03:25:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea174d0 0x55a9bede96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea174d0 0x55a9bede96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:56736/0 (socket says 172.21.15.39:56736) 2024-05-04T03:25:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.268+0000 7f9dfffff640 1 -- 172.21.15.39:0/2146528801 learned_addr learned my addr 172.21.15.39:0/2146528801 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.269+0000 7f9e053ee640 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea381b0 0x55a9bede2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.269+0000 7f9e04bed640 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bede27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.269+0000 7f9dfffff640 1 -- 172.21.15.39:0/2146528801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea381b0 msgr2=0x55a9bede2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.269+0000 7f9dfffff640 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea381b0 0x55a9bede2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.269+0000 7f9dfffff640 1 -- 172.21.15.39:0/2146528801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 msgr2=0x55a9bede27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.269+0000 7f9dfffff640 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bede27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.269+0000 7f9dfdffb640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9df00b2970 0x7f9df00b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.269+0000 7f9dfffff640 1 -- 172.21.15.39:0/2146528801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9bea50210 con 0x55a9bea174d0 2024-05-04T03:25:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.270+0000 7f9dfdffb640 1 -- 172.21.15.39:0/1017247216 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9df40e1e70 con 0x7f9e00102b50 2024-05-04T03:25:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.271+0000 7f9dfffff640 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea174d0 0x55a9bede96f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9dec005760 tx=0x7f9dec004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba78b675ac5f917 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.271+0000 7f9e053ee640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9df00b2970 0x7f9df00b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.271+0000 7f9deaffd640 1 -- 172.21.15.39:0/2146528801 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec035030 con 0x55a9bea174d0 2024-05-04T03:25:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.271+0000 7f9deaffd640 1 -- 172.21.15.39:0/2146528801 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9dec040070 con 0x55a9bea174d0 2024-05-04T03:25:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.271+0000 7f9deaffd640 1 -- 172.21.15.39:0/2146528801 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec035030 con 0x55a9bea174d0 2024-05-04T03:25:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.272+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2146528801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea174d0 msgr2=0x55a9bede96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.272+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea174d0 0x55a9bede96f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9dec005760 tx=0x7f9dec004260 comp rx=0 tx=0).stop 2024-05-04T03:25:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.272+0000 7f9e053ee640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9df00b2970 0x7f9df00b4e80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55a9bede3440 tx=0x7f9df8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.272+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2146528801 shutdown_connections 2024-05-04T03:25:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.272+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea174d0 0x55a9bede96f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.272+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bede27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.272+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/2146528801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea381b0 0x55a9bede2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.273+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2146528801 >> 172.21.15.39:0/2146528801 conn(0x55a9bea2c300 msgr2=0x55a9bedddbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.273+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2146528801 shutdown_connections 2024-05-04T03:25:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.273+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2146528801 wait complete. 2024-05-04T03:25:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.273+0000 7f9e0dde3740 1 -- start start 2024-05-04T03:25:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.274+0000 7f9e0dde3740 1 Processor -- start 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.274+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bedd6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.274+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bedd73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea381b0 0x55a9beddc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a9bedd4fd0 con 0x55a9bea0f2f0 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a9becfb370 con 0x55a9bea174d0 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a9bedd5150 con 0x55a9bea381b0 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e053ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea381b0 0x55a9beddc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e053ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea381b0 0x55a9beddc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:56742/0 (socket says 172.21.15.39:56742) 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e053ee640 1 -- 172.21.15.39:0/3446829966 learned_addr learned my addr 172.21.15.39:0/3446829966 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e0dde3740 10 client.0 _ll_register_callbacks cb 0x55a9bea4fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e04bed640 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bedd6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9dfffff640 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bedd73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e053ee640 1 -- 172.21.15.39:0/3446829966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 msgr2=0x55a9bedd73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e053ee640 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bedd73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e053ee640 1 -- 172.21.15.39:0/3446829966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 msgr2=0x55a9bedd6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e053ee640 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bedd6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.275+0000 7f9e053ee640 1 -- 172.21.15.39:0/3446829966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9beb1c620 con 0x55a9bea381b0 2024-05-04T03:25:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.276+0000 7f9e053ee640 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea381b0 0x55a9beddc2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9df8012910 tx=0x7f9df8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0ef3ce8f15546f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.276+0000 7f9ddffff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:25:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.276+0000 7f9ddffff640 1 -- 172.21.15.39:0/3446829966 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8010df0 con 0x55a9bea381b0 2024-05-04T03:25:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.276+0000 7f9ddffff640 1 -- 172.21.15.39:0/3446829966 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9df801b460 con 0x55a9bea381b0 2024-05-04T03:25:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.276+0000 7f9e0dde3740 1 -- 172.21.15.39:0/3446829966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dec040070 con 0x55a9bea381b0 2024-05-04T03:25:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.276+0000 7f9ddffff640 1 -- 172.21.15.39:0/3446829966 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9df801b660 con 0x55a9bea381b0 2024-05-04T03:25:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.277+0000 7f9e0dde3740 10 client.26521 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.277+0000 7f9e0dde3740 1 -- 172.21.15.39:0/3446829966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a9bedfeb00 con 0x55a9bea381b0 2024-05-04T03:25:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.278+0000 7f9ddffff640 1 -- 172.21.15.39:0/3446829966 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9dec040070 con 0x55a9bea381b0 2024-05-04T03:25:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.280+0000 7f9ddffff640 1 -- 172.21.15.39:0/3446829966 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9df8024070 con 0x55a9bea381b0 2024-05-04T03:25:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.280+0000 7f9e0dde3740 10 client.26521 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.280+0000 7f9e0dde3740 10 client.26521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.280+0000 7f9e0dde3740 20 client.26521 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.280+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a9be9f6d80 0x55a9bee02240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.280+0000 7f9e0dde3740 4 client.26521 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:25:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.281+0000 7f9e0dde3740 1 -- 172.21.15.39:0/3446829966 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9bed4c1e0 con 0x55a9be9f6d80 2024-05-04T03:25:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.281+0000 7f9e04bed640 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a9be9f6d80 0x55a9bee02240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.281+0000 7f9e04bed640 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a9be9f6d80 0x55a9bee02240 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e2cf0e3b8499b1 server_cookie=66e183fba3bb8af9 in_seq=0 out_seq=0 2024-05-04T03:25:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.281+0000 7f9ddffff640 10 client.26521 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.282+0000 7f9ddffff640 1 -- 172.21.15.39:0/3446829966 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a9bed4c1e0 con 0x55a9be9f6d80 2024-05-04T03:25:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.282+0000 7f9ddffff640 10 client.26521 handle_command_reply: tid=0 2024-05-04T03:25:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 client.26521 shutdown 2024-05-04T03:25:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 -- 172.21.15.39:0/3446829966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea381b0 msgr2=0x55a9beddc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea381b0 0x55a9beddc2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9df8012910 tx=0x7f9df8004a40 comp rx=0 tx=0).stop 2024-05-04T03:25:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 -- 172.21.15.39:0/3446829966 shutdown_connections 2024-05-04T03:25:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a9be9f6d80 0x55a9bee02240 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9ddffff640 0 client.26521 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9bea381b0 0x55a9beddc2f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.286 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bedd73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/3446829966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bedd6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.283+0000 7f9e0dde3740 1 -- 172.21.15.39:0/3446829966 >> 172.21.15.39:0/3446829966 conn(0x55a9bea2c300 msgr2=0x55a9bede0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.284+0000 7f9e0dde3740 1 -- 172.21.15.39:0/3446829966 shutdown_connections 2024-05-04T03:25:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.284+0000 7f9e0dde3740 1 -- 172.21.15.39:0/3446829966 wait complete. 2024-05-04T03:25:38.287 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.284+0000 7f9e0dde3740 20 client.26521 trim_cache size 0 max 16384 2024-05-04T03:25:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e0dde3740 1 Processor -- start 2024-05-04T03:25:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e0dde3740 1 -- start start 2024-05-04T03:25:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9beddc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9beddd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bedddab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e04bed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9beddc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e04bed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9beddc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:56752/0 (socket says 172.21.15.39:56752) 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001a9060 con 0x55a9bea0f2f0 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001aaba0 con 0x55a9bea174d0 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.316+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001a9730 con 0x55a9be9f6d80 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9beddd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9beddd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57378/0 (socket says 172.21.15.39:57378) 2024-05-04T03:25:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9dfffff640 1 -- 172.21.15.39:0/2122796468 learned_addr learned my addr 172.21.15.39:0/2122796468 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9dfffff640 1 -- 172.21.15.39:0/2122796468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 msgr2=0x55a9beddc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9dfffff640 1 --2- 172.21.15.39:0/2122796468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9beddc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9dfffff640 1 -- 172.21.15.39:0/2122796468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 msgr2=0x55a9bedddab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:25:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9dfffff640 1 --2- 172.21.15.39:0/2122796468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bedddab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9dfffff640 1 -- 172.21.15.39:0/2122796468 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9bede3470 con 0x55a9bea0f2f0 2024-05-04T03:25:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.317+0000 7f9e04bed640 1 --2- 172.21.15.39:0/2122796468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9beddc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.318+0000 7f9dfffff640 1 --2- 172.21.15.39:0/2122796468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9beddd570 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55a9bedde990 tx=0x7f9dec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a310c394b7068a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.318+0000 7f9dea7fc640 1 -- 172.21.15.39:0/2122796468 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec052070 con 0x55a9bea0f2f0 2024-05-04T03:25:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.319+0000 7f9dea7fc640 1 -- 172.21.15.39:0/2122796468 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9dec040040 con 0x55a9bea0f2f0 2024-05-04T03:25:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.319+0000 7f9dea7fc640 1 -- 172.21.15.39:0/2122796468 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec035050 con 0x55a9bea0f2f0 2024-05-04T03:25:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.320+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2122796468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 msgr2=0x55a9beddd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.320+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/2122796468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9beddd570 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55a9bedde990 tx=0x7f9dec000bc0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.320+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2122796468 shutdown_connections 2024-05-04T03:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.320+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/2122796468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bedddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.320+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/2122796468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9beddd570 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.320+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/2122796468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9beddc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.320+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2122796468 >> 172.21.15.39:0/2122796468 conn(0x55a9bea2c300 msgr2=0x55a9beddc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.320+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2122796468 shutdown_connections 2024-05-04T03:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.321+0000 7f9e0dde3740 1 -- 172.21.15.39:0/2122796468 wait complete. 2024-05-04T03:25:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.321+0000 7f9e0dde3740 1 -- start start 2024-05-04T03:25:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.321+0000 7f9e0dde3740 1 Processor -- start 2024-05-04T03:25:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9bee10f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bee11480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e0dde3740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bee123b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001aa4d0 con 0x55a9bea0f2f0 2024-05-04T03:25:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001a9e00 con 0x55a9bea174d0 2024-05-04T03:25:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e0dde3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e001a7040 con 0x55a9be9f6d80 2024-05-04T03:25:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e0dde3740 10 client.0 _ll_register_callbacks cb 0x7f9df8003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e04bed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9bee10f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e04bed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9bee10f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:56768/0 (socket says 172.21.15.39:56768) 2024-05-04T03:25:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e04bed640 1 -- 172.21.15.39:0/4060467621 learned_addr learned my addr 172.21.15.39:0/4060467621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9e053ee640 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bee123b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.322+0000 7f9dfffff640 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bee11480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.323+0000 7f9e04bed640 1 -- 172.21.15.39:0/4060467621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 msgr2=0x55a9bee123b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.323+0000 7f9e04bed640 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bee123b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.323+0000 7f9e04bed640 1 -- 172.21.15.39:0/4060467621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 msgr2=0x55a9bee11480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.323+0000 7f9e04bed640 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bee11480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.323+0000 7f9e04bed640 1 -- 172.21.15.39:0/4060467621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9bee12970 con 0x55a9be9f6d80 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.323+0000 7f9e04bed640 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9bee10f40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9df4000c00 tx=0x7f9df400b2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c3503968835584 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.323+0000 7f9ddffff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.324+0000 7f9ddffff640 1 -- 172.21.15.39:0/4060467621 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df40ad7f0 con 0x55a9be9f6d80 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.324+0000 7f9ddffff640 1 -- 172.21.15.39:0/4060467621 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9df400ec50 con 0x55a9be9f6d80 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.324+0000 7f9e0dde3740 1 -- 172.21.15.39:0/4060467621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dec040040 con 0x55a9be9f6d80 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.324+0000 7f9ddffff640 1 -- 172.21.15.39:0/4060467621 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9df40b22a0 con 0x55a9be9f6d80 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.324+0000 7f9e0dde3740 10 client.26527 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.324+0000 7f9e0dde3740 1 -- 172.21.15.39:0/4060467621 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a9bee13c70 con 0x55a9be9f6d80 2024-05-04T03:25:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.325+0000 7f9ddffff640 1 -- 172.21.15.39:0/4060467621 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9dec040040 con 0x55a9be9f6d80 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.325+0000 7f9ddffff640 1 -- 172.21.15.39:0/4060467621 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9df40144b0 con 0x55a9be9f6d80 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.325+0000 7f9e0dde3740 10 client.26527 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.325+0000 7f9e0dde3740 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.326+0000 7f9e0dde3740 20 client.26527 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.326+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a9bea381b0 0x55a9bee6c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.326+0000 7f9e0dde3740 4 client.26527 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.326+0000 7f9e0dde3740 1 -- 172.21.15.39:0/4060467621 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9bee02830 con 0x55a9bea381b0 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.326+0000 7f9dfffff640 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a9bea381b0 0x55a9bee6c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.327+0000 7f9ddffff640 10 client.26527 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.327+0000 7f9dfffff640 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a9bea381b0 0x55a9bee6c8f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d61a3f27520e893 server_cookie=264d5ceda8eb0fbd in_seq=0 out_seq=0 2024-05-04T03:25:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.336+0000 7f9ddffff640 1 -- 172.21.15.39:0/4060467621 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9bee02830 con 0x55a9bea381b0 2024-05-04T03:25:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.336+0000 7f9ddffff640 10 client.26527 handle_command_reply: tid=0 2024-05-04T03:25:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.336+0000 7f9e0dde3740 1 client.26527 shutdown 2024-05-04T03:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.336+0000 7f9e0dde3740 1 -- 172.21.15.39:0/4060467621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 msgr2=0x55a9bee10f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.336+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9bee10f40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9df4000c00 tx=0x7f9df400b2d0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 1 -- 172.21.15.39:0/4060467621 shutdown_connections 2024-05-04T03:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a9bea381b0 0x55a9bee6c8f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a9bea174d0 0x55a9bee123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9ddffff640 0 client.26527 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a9bea0f2f0 0x55a9bee11480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 1 --2- 172.21.15.39:0/4060467621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a9be9f6d80 0x55a9bee10f40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 1 -- 172.21.15.39:0/4060467621 >> 172.21.15.39:0/4060467621 conn(0x55a9bea2c300 msgr2=0x55a9beddd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 1 -- 172.21.15.39:0/4060467621 shutdown_connections 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 1 -- 172.21.15.39:0/4060467621 wait complete. 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.337+0000 7f9e0dde3740 20 client.26527 trim_cache size 0 max 16384 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9df00b2970 msgr2=0x7f9df00b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9df00b2970 0x7f9df00b4e80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55a9bede3440 tx=0x7f9df8009500 comp rx=0 tx=0).stop 2024-05-04T03:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00102b50 msgr2=0x7f9e0019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00102b50 0x7f9e0019c460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9e00102420 tx=0x7f9df4009c60 comp rx=0 tx=0).stop 2024-05-04T03:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 shutdown_connections 2024-05-04T03:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9df00b2970 0x7f9df00b4e80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e00108a90 0x7f9e001a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e00103d90 0x7f9e0019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 --2- 172.21.15.39:0/1017247216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e00102b50 0x7f9e0019c460 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.339+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 >> 172.21.15.39:0/1017247216 conn(0x7f9e000fdd40 msgr2=0x7f9e00104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.340+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 shutdown_connections 2024-05-04T03:25:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:38.340+0000 7f9e06e51640 1 -- 172.21.15.39:0/1017247216 wait complete. 2024-05-04T03:25:38.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 -- 172.21.15.39:0/1269189537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330c11d930 msgr2=0x7f330c11fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 --2- 172.21.15.39:0/1269189537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330c11d930 0x7f330c11fd50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f330400b0a0 tx=0x7f330400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 -- 172.21.15.39:0/1269189537 shutdown_connections 2024-05-04T03:25:38.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 --2- 172.21.15.39:0/1269189537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f330c10a930 0x7f330c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 --2- 172.21.15.39:0/1269189537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330c11d930 0x7f330c11fd50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 --2- 172.21.15.39:0/1269189537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f330c10b280 0x7f330c11d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 -- 172.21.15.39:0/1269189537 >> 172.21.15.39:0/1269189537 conn(0x7f330c06f4d0 msgr2=0x7f330c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 -- 172.21.15.39:0/1269189537 shutdown_connections 2024-05-04T03:25:38.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.696+0000 7f33129eb700 1 -- 172.21.15.39:0/1269189537 wait complete. 2024-05-04T03:25:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.697+0000 7f33129eb700 1 Processor -- start 2024-05-04T03:25:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.697+0000 7f33129eb700 1 -- start start 2024-05-04T03:25:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f33129eb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f330c10a930 0x7f330c1af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f33129eb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f330c10b280 0x7f330c1afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f33129eb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330c11d930 0x7f330c1a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f33129eb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f330c123120 con 0x7f330c10b280 2024-05-04T03:25:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f33129eb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f330c122fa0 con 0x7f330c10a930 2024-05-04T03:25:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f33129eb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f330c1232a0 con 0x7f330c11d930 2024-05-04T03:25:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f330bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f330c10a930 0x7f330c1af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f330b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f330c10b280 0x7f330c1afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f330bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f330c10a930 0x7f330c1af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:33622/0 (socket says 172.21.15.39:33622) 2024-05-04T03:25:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f330bfff700 1 -- 172.21.15.39:0/1545740485 learned_addr learned my addr 172.21.15.39:0/1545740485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.698+0000 7f3310f88700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330c11d930 0x7f330c1a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.699+0000 7f330b7fe700 1 -- 172.21.15.39:0/1545740485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330c11d930 msgr2=0x7f330c1a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.699+0000 7f330b7fe700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330c11d930 0x7f330c1a9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.699+0000 7f330b7fe700 1 -- 172.21.15.39:0/1545740485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f330c10a930 msgr2=0x7f330c1af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.699+0000 7f330b7fe700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f330c10a930 0x7f330c1af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.699+0000 7f330b7fe700 1 -- 172.21.15.39:0/1545740485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c1a9fd0 con 0x7f330c10b280 2024-05-04T03:25:38.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.699+0000 7f330b7fe700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f330c10b280 0x7f330c1afc40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f330000d950 tx=0x7f330000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.699+0000 7f33097fa700 1 -- 172.21.15.39:0/1545740485 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300016070 con 0x7f330c10b280 2024-05-04T03:25:38.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.700+0000 7f33097fa700 1 -- 172.21.15.39:0/1545740485 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f330000fd20 con 0x7f330c10b280 2024-05-04T03:25:38.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.700+0000 7f33097fa700 1 -- 172.21.15.39:0/1545740485 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330000e3b0 con 0x7f330c10b280 2024-05-04T03:25:38.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.700+0000 7f33129eb700 1 -- 172.21.15.39:0/1545740485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c1aa260 con 0x7f330c10b280 2024-05-04T03:25:38.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.700+0000 7f33129eb700 1 -- 172.21.15.39:0/1545740485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c1aa750 con 0x7f330c10b280 2024-05-04T03:25:38.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.704+0000 7f33097fa700 1 -- 172.21.15.39:0/1545740485 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3300019070 con 0x7f330c10b280 2024-05-04T03:25:38.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.705+0000 7f33097fa700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f32f40b2a50 0x7f32f40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.705+0000 7f33213ffb80 1 Processor -- start 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.705+0000 7f33213ffb80 1 -- start start 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f33097fa700 1 -- 172.21.15.39:0/1545740485 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f33000e1b10 con 0x7f330c10b280 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f330bfff700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f32f40b2a50 0x7f32f40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cb3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cbe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94cc0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f33213ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560f94bd69f0 con 0x560f94cbe970 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f33213ffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560f94b75950 con 0x560f94cb3fb0 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f33213ffb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560f94bd29b0 con 0x560f94cb3660 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f330bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cb3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f330bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cb3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:56796/0 (socket says 172.21.15.39:56796) 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f330bfff700 1 -- 172.21.15.39:0/573025168 learned_addr learned my addr 172.21.15.39:0/573025168 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.706+0000 7f330b7fe700 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cbe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f3310f88700 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94cc0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f330bfff700 1 -- 172.21.15.39:0/573025168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 msgr2=0x560f94cbe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f330bfff700 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cbe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f330bfff700 1 -- 172.21.15.39:0/573025168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 msgr2=0x560f94cc0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f330bfff700 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94cc0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f330bfff700 1 -- 172.21.15.39:0/573025168 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f94b23a60 con 0x560f94cb3660 2024-05-04T03:25:38.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f330b7fe700 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cbe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:38.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f330bfff700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f32f40b2a50 0x7f32f40b4f60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f32fc012b30 tx=0x7f32fc012620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f330bfff700 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cb3a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00b7e0 tx=0x7f32fc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a479326f8ff43f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.707+0000 7f32f27fc700 1 -- 172.21.15.39:0/573025168 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc024070 con 0x560f94cb3660 2024-05-04T03:25:38.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.708+0000 7f32f27fc700 1 -- 172.21.15.39:0/573025168 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32fc0044c0 con 0x560f94cb3660 2024-05-04T03:25:38.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.708+0000 7f32f27fc700 1 -- 172.21.15.39:0/573025168 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc024070 con 0x560f94cb3660 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.709+0000 7f33213ffb80 1 -- 172.21.15.39:0/573025168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 msgr2=0x560f94cb3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.709+0000 7f33213ffb80 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cb3a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00b7e0 tx=0x7f32fc01c000 comp rx=0 tx=0).stop 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.709+0000 7f33213ffb80 1 -- 172.21.15.39:0/573025168 shutdown_connections 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.709+0000 7f33213ffb80 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94cc0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.709+0000 7f33213ffb80 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cb3a70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.709+0000 7f33213ffb80 1 --2- 172.21.15.39:0/573025168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cbe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.709+0000 7f33213ffb80 1 -- 172.21.15.39:0/573025168 >> 172.21.15.39:0/573025168 conn(0x560f94b36fd0 msgr2=0x560f94caf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.710+0000 7f33213ffb80 1 -- 172.21.15.39:0/573025168 shutdown_connections 2024-05-04T03:25:38.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.710+0000 7f33213ffb80 1 -- 172.21.15.39:0/573025168 wait complete. 2024-05-04T03:25:38.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.711+0000 7f33213ffb80 1 -- start start 2024-05-04T03:25:38.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.711+0000 7f33213ffb80 1 Processor -- start 2024-05-04T03:25:38.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.712+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3660 0x560f94cbb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.712+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3fb0 0x560f94cbbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.713+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cbc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.713+0000 7f33213ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560f94cc16f0 con 0x560f94cb3fb0 2024-05-04T03:25:38.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.713+0000 7f330bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3660 0x560f94cbb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.713+0000 7f330bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3660 0x560f94cbb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:33640/0 (socket says 172.21.15.39:33640) 2024-05-04T03:25:38.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.713+0000 7f330bfff700 1 -- 172.21.15.39:0/2536806117 learned_addr learned my addr 172.21.15.39:0/2536806117 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.713+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560f94b3cdc0 con 0x560f94cb3660 2024-05-04T03:25:38.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.713+0000 7f330b7fe700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3fb0 0x560f94cbbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.713+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560f94cc1870 con 0x560f94cbe970 2024-05-04T03:25:38.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.714+0000 7f33213ffb80 10 client.0 _ll_register_callbacks cb 0x560f94b6eab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:38.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f3310f88700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cbc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f330b7fe700 1 -- 172.21.15.39:0/2536806117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 msgr2=0x560f94cbc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f330b7fe700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cbc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f330b7fe700 1 -- 172.21.15.39:0/2536806117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3660 msgr2=0x560f94cbb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f330b7fe700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3660 0x560f94cbb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f330b7fe700 1 -- 172.21.15.39:0/2536806117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f94b72820 con 0x560f94cb3fb0 2024-05-04T03:25:38.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f3310f88700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cbc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:25:38.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f330b7fe700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3fb0 0x560f94cbbee0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f33000df5c0 tx=0x7f33000a9060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdad70be14d6e252 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f32eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:25:38.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.715+0000 7f330bfff700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3660 0x560f94cbb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:38.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.716+0000 7f32eb7fe700 1 -- 172.21.15.39:0/2536806117 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300020070 con 0x560f94cb3fb0 2024-05-04T03:25:38.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.716+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32fc0044c0 con 0x560f94cb3fb0 2024-05-04T03:25:38.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.716+0000 7f32eb7fe700 1 -- 172.21.15.39:0/2536806117 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f33000ad170 con 0x560f94cb3fb0 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.716+0000 7f32eb7fe700 1 -- 172.21.15.39:0/2536806117 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f33000a95d0 con 0x560f94cb3fb0 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.716+0000 7f33213ffb80 10 client.16920 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.716+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560f94ccc440 con 0x560f94cb3fb0 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.717+0000 7f32eb7fe700 1 -- 172.21.15.39:0/2536806117 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33000b1ab0 con 0x560f94cb3fb0 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.717+0000 7f32eb7fe700 1 -- 172.21.15.39:0/2536806117 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f33000dfa80 con 0x560f94cb3fb0 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.717+0000 7f33213ffb80 10 client.16920 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.717+0000 7f33213ffb80 10 client.16920 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.717+0000 7f33213ffb80 20 client.16920 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:38.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.717+0000 7f33213ffb80 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560f94cce9d0 0x560f94cd0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.717+0000 7f33213ffb80 4 client.16920 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.717+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f32fc0044c0 con 0x560f94cce9d0 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.718+0000 7f330bfff700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560f94cce9d0 0x560f94cd0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.719+0000 7f330bfff700 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560f94cce9d0 0x560f94cd0e10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6935241a1ad771ad server_cookie=ed72802a3ab24fd6 in_seq=0 out_seq=0 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.719+0000 7f32eb7fe700 10 client.16920 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.720+0000 7f32eb7fe700 1 -- 172.21.15.39:0/2536806117 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f32fc0044c0 con 0x560f94cce9d0 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.720+0000 7f32eb7fe700 10 client.16920 handle_command_reply: tid=0 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.720+0000 7f33213ffb80 1 client.16920 shutdown 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.720+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3fb0 msgr2=0x560f94cbbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.720+0000 7f33213ffb80 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3fb0 0x560f94cbbee0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f33000df5c0 tx=0x7f33000a9060 comp rx=0 tx=0).stop 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 shutdown_connections 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f33213ffb80 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560f94cce9d0 0x560f94cd0e10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f33213ffb80 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cbc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f33213ffb80 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3660 0x560f94cbb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f33213ffb80 1 --2- 172.21.15.39:0/2536806117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3fb0 0x560f94cbbee0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f32eb7fe700 0 client.16920 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:38.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 >> 172.21.15.39:0/2536806117 conn(0x560f94b36fd0 msgr2=0x560f94cb5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 shutdown_connections 2024-05-04T03:25:38.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.721+0000 7f33213ffb80 1 -- 172.21.15.39:0/2536806117 wait complete. 2024-05-04T03:25:38.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.722+0000 7f33213ffb80 20 client.16920 trim_cache size 0 max 16384 2024-05-04T03:25:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.758+0000 7f33213ffb80 1 Processor -- start 2024-05-04T03:25:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.759+0000 7f33213ffb80 1 -- start start 2024-05-04T03:25:38.761 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:38 smithi039 ceph-mon[29715]: pgmap v831: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:25:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.759+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3660 0x560f94cbc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.759+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cd65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.759+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cd6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.759+0000 7f33213ffb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f330c1bd940 con 0x560f94cb3660 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.759+0000 7f330b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cd65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.759+0000 7f330b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cd65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:33644/0 (socket says 172.21.15.39:33644) 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.759+0000 7f330b7fe700 1 -- 172.21.15.39:0/199455926 learned_addr learned my addr 172.21.15.39:0/199455926 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f3310f88700 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cd6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f33213ffb80 1 -- 172.21.15.39:0/199455926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f330c1bf480 con 0x560f94cb3fb0 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f33213ffb80 1 -- 172.21.15.39:0/199455926 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f330c1be010 con 0x560f94cbe970 2024-05-04T03:25:38.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f330bfff700 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3660 0x560f94cbc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f330b7fe700 1 -- 172.21.15.39:0/199455926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 msgr2=0x560f94cd6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f330b7fe700 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cd6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f330b7fe700 1 -- 172.21.15.39:0/199455926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3660 msgr2=0x560f94cbc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f330b7fe700 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3660 0x560f94cbc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.760+0000 7f330b7fe700 1 -- 172.21.15.39:0/199455926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f94cb9f10 con 0x560f94cb3fb0 2024-05-04T03:25:38.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.761+0000 7f330b7fe700 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cd65d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f33000dfef0 tx=0x7f330000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59cd1ec3ec481f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.761+0000 7f32f1ffb700 1 -- 172.21.15.39:0/199455926 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300020030 con 0x560f94cb3fb0 2024-05-04T03:25:38.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.761+0000 7f32f1ffb700 1 -- 172.21.15.39:0/199455926 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33000ad170 con 0x560f94cb3fb0 2024-05-04T03:25:38.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.761+0000 7f32f1ffb700 1 -- 172.21.15.39:0/199455926 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33000b1d10 con 0x560f94cb3fb0 2024-05-04T03:25:38.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 -- 172.21.15.39:0/199455926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 msgr2=0x560f94cd65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:38.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cd65d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f33000dfef0 tx=0x7f330000bf90 comp rx=0 tx=0).stop 2024-05-04T03:25:38.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 -- 172.21.15.39:0/199455926 shutdown_connections 2024-05-04T03:25:38.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cb3660 0x560f94cbc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cbe970 0x560f94cd6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 --2- 172.21.15.39:0/199455926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cd65d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 -- 172.21.15.39:0/199455926 >> 172.21.15.39:0/199455926 conn(0x560f94b36fd0 msgr2=0x560f94cbd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:38.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 -- 172.21.15.39:0/199455926 shutdown_connections 2024-05-04T03:25:38.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.763+0000 7f33213ffb80 1 -- 172.21.15.39:0/199455926 wait complete. 2024-05-04T03:25:38.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.764+0000 7f33213ffb80 1 -- start start 2024-05-04T03:25:38.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.764+0000 7f33213ffb80 1 Processor -- start 2024-05-04T03:25:38.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cc3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f33213ffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cc3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f330bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cc3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f330bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cc3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:56834/0 (socket says 172.21.15.39:56834) 2024-05-04T03:25:38.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f330bfff700 1 -- 172.21.15.39:0/3688792923 learned_addr learned my addr 172.21.15.39:0/3688792923 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:38.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f33213ffb80 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94d31140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f330c1bedb0 con 0x560f94cbe970 2024-05-04T03:25:38.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f330c1be6e0 con 0x560f94cb3fb0 2024-05-04T03:25:38.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f330c1bb8e0 con 0x560f94cb3660 2024-05-04T03:25:38.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.765+0000 7f33213ffb80 10 client.0 _ll_register_callbacks cb 0x560f94b6eab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:38.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.766+0000 7f330b7fe700 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cc3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.766+0000 7f3310f88700 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94d31140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.766+0000 7f330bfff700 1 -- 172.21.15.39:0/3688792923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 msgr2=0x560f94cc3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.766+0000 7f330bfff700 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cc3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.766+0000 7f330bfff700 1 -- 172.21.15.39:0/3688792923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 msgr2=0x560f94d31140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.766+0000 7f330bfff700 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94d31140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.766+0000 7f330bfff700 1 -- 172.21.15.39:0/3688792923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f94d31780 con 0x560f94cb3660 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.766+0000 7f3310f88700 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94d31140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.767+0000 7f330bfff700 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cc3250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00bb90 tx=0x7f32fc00e6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec0ec4ed4b0df7c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.767+0000 7f32eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:25:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.767+0000 7f32eb7fe700 1 -- 172.21.15.39:0/3688792923 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc024040 con 0x560f94cb3660 2024-05-04T03:25:38.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.767+0000 7f32eb7fe700 1 -- 172.21.15.39:0/3688792923 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f32fc005310 con 0x560f94cb3660 2024-05-04T03:25:38.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.767+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33000ad170 con 0x560f94cb3660 2024-05-04T03:25:38.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.767+0000 7f32eb7fe700 1 -- 172.21.15.39:0/3688792923 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32fc005510 con 0x560f94cb3660 2024-05-04T03:25:38.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.767+0000 7f33213ffb80 10 client.26542 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.767+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560f94d39bc0 con 0x560f94cb3660 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f32eb7fe700 1 -- 172.21.15.39:0/3688792923 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32fc031810 con 0x560f94cb3660 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f32eb7fe700 1 -- 172.21.15.39:0/3688792923 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f32fc01f070 con 0x560f94cb3660 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f33213ffb80 10 client.26542 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f33213ffb80 10 client.26542 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f33213ffb80 20 client.26542 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f33213ffb80 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560f94d3a1c0 0x560f94d3c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f33213ffb80 4 client.26542 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.35, "await_for": 42.0}] 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.35, "await_for": 42.0}) -- 0x7f33000ad170 con 0x560f94d3a1c0 2024-05-04T03:25:38.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.768+0000 7f330b7fe700 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560f94d3a1c0 0x560f94d3c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.769+0000 7f330b7fe700 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560f94d3a1c0 0x560f94d3c5e0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c909d77426209ff server_cookie=935f9af485f04ae3 in_seq=0 out_seq=0 2024-05-04T03:25:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:38.769+0000 7f32eb7fe700 10 client.26542 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:38.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:38 smithi167 ceph-mon[33614]: pgmap v831: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:25:39.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:38 smithi066 ceph-mon[33264]: pgmap v831: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.45k op/s 2024-05-04T03:25:39.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f32eb7fe700 1 -- 172.21.15.39:0/3688792923 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f33000ad170 con 0x560f94d3a1c0 2024-05-04T03:25:39.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f32eb7fe700 10 client.26542 handle_command_reply: tid=0 2024-05-04T03:25:39.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 client.26542 shutdown 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 msgr2=0x560f94cc3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cc3250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00bb90 tx=0x7f32fc00e6e0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 shutdown_connections 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560f94d3a1c0 0x560f94d3c5e0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f94cbe970 0x560f94d31140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560f94cb3660 0x560f94cc3250 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 --2- 172.21.15.39:0/3688792923 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560f94cb3fb0 0x560f94cc3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f32eb7fe700 0 client.26542 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 >> 172.21.15.39:0/3688792923 conn(0x560f94b36fd0 msgr2=0x560f94cae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 shutdown_connections 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 1 -- 172.21.15.39:0/3688792923 wait complete. 2024-05-04T03:25:39.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.143+0000 7f33213ffb80 20 client.26542 trim_cache size 0 max 16384 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 38, 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "b55298cb": { 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 38, 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:25:39.147 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 85.3, 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:25:39.148 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:25:39.149 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:25:39.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 -- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f32f40b2a50 msgr2=0x7f32f40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:39.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f32f40b2a50 0x7f32f40b4f60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f32fc012b30 tx=0x7f32fc012620 comp rx=0 tx=0).stop 2024-05-04T03:25:39.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 -- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f330c10b280 msgr2=0x7f330c1afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:39.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f330c10b280 0x7f330c1afc40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f330000d950 tx=0x7f330000dc60 comp rx=0 tx=0).stop 2024-05-04T03:25:39.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 -- 172.21.15.39:0/1545740485 shutdown_connections 2024-05-04T03:25:39.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f32f40b2a50 0x7f32f40b4f60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f330c11d930 0x7f330c1a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f330c10a930 0x7f330c1af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 --2- 172.21.15.39:0/1545740485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f330c10b280 0x7f330c1afc40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:39.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 -- 172.21.15.39:0/1545740485 >> 172.21.15.39:0/1545740485 conn(0x7f330c06f4d0 msgr2=0x7f330c11b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:39.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 -- 172.21.15.39:0/1545740485 shutdown_connections 2024-05-04T03:25:39.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:25:39.144+0000 7f33129eb700 1 -- 172.21.15.39:0/1545740485 wait complete. 2024-05-04T03:25:39.801 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b55298cb', quiesce took 8.6 seconds. Will release after: 25.35 2024-05-04T03:25:40.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:40 smithi167 ceph-mon[33614]: pgmap v832: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:25:40.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:40 smithi039 ceph-mon[29715]: pgmap v832: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:25:41.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:40 smithi066 ceph-mon[33264]: pgmap v832: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 12 MiB/s wr, 1.38k op/s 2024-05-04T03:25:41.780 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 710.0 s, 476.5 tps, lat 85.949 ms stddev 27.517 2024-05-04T03:25:42.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:42 smithi167 ceph-mon[33614]: pgmap v833: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-05-04T03:25:42.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:42 smithi039 ceph-mon[29715]: pgmap v833: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-05-04T03:25:43.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:42 smithi066 ceph-mon[33264]: pgmap v833: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-05-04T03:25:44.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:44 smithi167 ceph-mon[33614]: pgmap v834: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 1.13k op/s 2024-05-04T03:25:44.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:44 smithi039 ceph-mon[29715]: pgmap v834: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 1.13k op/s 2024-05-04T03:25:45.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:44 smithi066 ceph-mon[33264]: pgmap v834: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.9 MiB/s wr, 1.13k op/s 2024-05-04T03:25:46.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 715.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:25:46.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:46 smithi167 ceph-mon[33614]: pgmap v835: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 822 op/s 2024-05-04T03:25:46.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:46 smithi039 ceph-mon[29715]: pgmap v835: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 822 op/s 2024-05-04T03:25:47.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:46 smithi066 ceph-mon[33264]: pgmap v835: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.2 MiB/s wr, 822 op/s 2024-05-04T03:25:48.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:48 smithi167 ceph-mon[33614]: pgmap v836: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 622 op/s 2024-05-04T03:25:48.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:48 smithi039 ceph-mon[29715]: pgmap v836: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 622 op/s 2024-05-04T03:25:49.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:48 smithi066 ceph-mon[33264]: pgmap v836: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 622 op/s 2024-05-04T03:25:49.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:49 smithi167 ceph-mon[33614]: pgmap v837: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-05-04T03:25:49.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:49 smithi039 ceph-mon[29715]: pgmap v837: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-05-04T03:25:50.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:49 smithi066 ceph-mon[33264]: pgmap v837: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-05-04T03:25:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 -- 172.21.15.39:0/1670595741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 msgr2=0x7f912010be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 --2- 172.21.15.39:0/1670595741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 0x7f912010be30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9118009ca0 tx=0x7f911802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 -- 172.21.15.39:0/1670595741 shutdown_connections 2024-05-04T03:25:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 --2- 172.21.15.39:0/1670595741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 0x7f912010be30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 --2- 172.21.15.39:0/1670595741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91201018d0 0x7f9120101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 --2- 172.21.15.39:0/1670595741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9120105220 0x7f9120105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 -- 172.21.15.39:0/1670595741 >> 172.21.15.39:0/1670595741 conn(0x7f91200fd4f0 msgr2=0x7f91200ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 -- 172.21.15.39:0/1670595741 shutdown_connections 2024-05-04T03:25:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 -- 172.21.15.39:0/1670595741 wait complete. 2024-05-04T03:25:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.460+0000 7f9125b3f640 1 Processor -- start 2024-05-04T03:25:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f9125b3f640 1 -- start start 2024-05-04T03:25:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f9125b3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91201018d0 0x7f9120104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f9125b3f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 0x7f9120102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f9125b3f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9120105220 0x7f9120102fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f9125b3f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f912010f6b0 con 0x7f91201023d0 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f9125b3f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f912010f530 con 0x7f9120105220 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f9125b3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f912010f830 con 0x7f91201018d0 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f911effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 0x7f9120102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f911f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91201018d0 0x7f9120104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f911effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 0x7f9120102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33180/0 (socket says 172.21.15.39:33180) 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f911effd640 1 -- 172.21.15.39:0/3649801560 learned_addr learned my addr 172.21.15.39:0/3649801560 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.461+0000 7f911ffff640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9120105220 0x7f9120102fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911effd640 1 -- 172.21.15.39:0/3649801560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91201018d0 msgr2=0x7f9120104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911effd640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91201018d0 0x7f9120104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911effd640 1 -- 172.21.15.39:0/3649801560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9120105220 msgr2=0x7f9120102fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.465 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911effd640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9120105220 0x7f9120102fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911effd640 1 -- 172.21.15.39:0/3649801560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912019e0d0 con 0x7f91201023d0 2024-05-04T03:25:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911effd640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 0x7f9120102a70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f911400d950 tx=0x7f911400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911cff9640 1 -- 172.21.15.39:0/3649801560 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114017070 con 0x7f91201023d0 2024-05-04T03:25:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911cff9640 1 -- 172.21.15.39:0/3649801560 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9114009c30 con 0x7f91201023d0 2024-05-04T03:25:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f9125b3f640 1 -- 172.21.15.39:0/3649801560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91201035b0 con 0x7f91201023d0 2024-05-04T03:25:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f911cff9640 1 -- 172.21.15.39:0/3649801560 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911400f620 con 0x7f91201023d0 2024-05-04T03:25:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.462+0000 7f9125b3f640 1 -- 172.21.15.39:0/3649801560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91201a8d20 con 0x7f91201023d0 2024-05-04T03:25:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f912c9f9740 1 Processor -- start 2024-05-04T03:25:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f912c9f9740 1 -- start start 2024-05-04T03:25:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f911cff9640 1 -- 172.21.15.39:0/3649801560 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9114010460 con 0x7f91201023d0 2024-05-04T03:25:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a1ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a1bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a22ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f912c9f9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563f449805d0 con 0x563f446594d0 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f912c9f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f44939890 con 0x563f4467a1b0 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f912c9f9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f4497c590 con 0x563f446512f0 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f911effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a22ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f911effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a22ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33196/0 (socket says 172.21.15.39:33196) 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f911effd640 1 -- 172.21.15.39:0/2509082934 learned_addr learned my addr 172.21.15.39:0/2509082934 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f911f7fe640 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a1bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.465+0000 7f911ffff640 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a1ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911effd640 1 -- 172.21.15.39:0/2509082934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 msgr2=0x563f44a1bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911effd640 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a1bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911effd640 1 -- 172.21.15.39:0/2509082934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 msgr2=0x563f44a1ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911effd640 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a1ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911effd640 1 -- 172.21.15.39:0/2509082934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f44692210 con 0x563f446594d0 2024-05-04T03:25:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911cff9640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91080b27e0 0x7f91080b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911effd640 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a22ee0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f91140e5ea0 tx=0x7f91140e5f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2a3b5eac9722f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911ffff640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91080b27e0 0x7f91080b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f9105ffb640 1 -- 172.21.15.39:0/2509082934 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91140a5320 con 0x563f446594d0 2024-05-04T03:25:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f911cff9640 1 -- 172.21.15.39:0/3649801560 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f91140e1d80 con 0x7f91201023d0 2024-05-04T03:25:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f9105ffb640 1 -- 172.21.15.39:0/2509082934 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91140a8f70 con 0x563f446594d0 2024-05-04T03:25:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.466+0000 7f9105ffb640 1 -- 172.21.15.39:0/2509082934 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91140b5dd0 con 0x563f446594d0 2024-05-04T03:25:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 -- 172.21.15.39:0/2509082934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 msgr2=0x563f44a22ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f911ffff640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91080b27e0 0x7f91080b4cf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9118004af0 tx=0x7f9118031000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a22ee0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f91140e5ea0 tx=0x7f91140e5f90 comp rx=0 tx=0).stop 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 -- 172.21.15.39:0/2509082934 shutdown_connections 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a22ee0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a1bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2509082934 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a1ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 -- 172.21.15.39:0/2509082934 >> 172.21.15.39:0/2509082934 conn(0x563f4466e300 msgr2=0x563f44a173a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 -- 172.21.15.39:0/2509082934 shutdown_connections 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.467+0000 7f912c9f9740 1 -- 172.21.15.39:0/2509082934 wait complete. 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 1 -- start start 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 1 Processor -- start 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a25680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a25bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a2cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563f44a23930 con 0x563f446594d0 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563f4493d370 con 0x563f4467a1b0 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f44a23ab0 con 0x563f446512f0 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f912c9f9740 10 client.0 _ll_register_callbacks cb 0x563f44691a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f911effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a25bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f911effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a25bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33200/0 (socket says 172.21.15.39:33200) 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f911effd640 1 -- 172.21.15.39:0/912786773 learned_addr learned my addr 172.21.15.39:0/912786773 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f911f7fe640 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a25680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.468+0000 7f911ffff640 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a2cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f911effd640 1 -- 172.21.15.39:0/912786773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 msgr2=0x563f44a25680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f911effd640 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a25680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f911effd640 1 -- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 msgr2=0x563f44a2cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f911effd640 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a2cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f911effd640 1 -- 172.21.15.39:0/912786773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4475e620 con 0x563f446594d0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f911ffff640 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a2cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f911effd640 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a25bc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f91140df8e0 tx=0x7f91140e56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385594c413df93b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f90feffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f90feffd640 1 -- 172.21.15.39:0/912786773 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114020070 con 0x563f446594d0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f90feffd640 1 -- 172.21.15.39:0/912786773 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f91140a5a50 con 0x563f446594d0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f912c9f9740 1 -- 172.21.15.39:0/912786773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91140a8f70 con 0x563f446594d0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f90feffd640 1 -- 172.21.15.39:0/912786773 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91140a8bd0 con 0x563f446594d0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f912c9f9740 10 client.16938 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f912c9f9740 1 -- 172.21.15.39:0/912786773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f44a2fa50 con 0x563f446594d0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.469+0000 7f90feffd640 1 -- 172.21.15.39:0/912786773 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91140a8f70 con 0x563f446594d0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f90feffd640 1 -- 172.21.15.39:0/912786773 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f91140b1d80 con 0x563f446594d0 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f912c9f9740 10 client.16938 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f912c9f9740 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f912c9f9740 20 client.16938 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f912c9f9740 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f44638d80 0x563f44a32db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f912c9f9740 4 client.16938 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f912c9f9740 1 -- 172.21.15.39:0/912786773 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f4498e1e0 con 0x563f44638d80 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f911f7fe640 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f44638d80 0x563f44a32db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f90feffd640 10 client.16938 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.470+0000 7f911f7fe640 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f44638d80 0x563f44a32db0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27aa80e1b77266f server_cookie=1ecc2a23e094fba6 in_seq=0 out_seq=0 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f90feffd640 1 -- 172.21.15.39:0/912786773 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f4498e1e0 con 0x563f44638d80 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f90feffd640 10 client.16938 handle_command_reply: tid=0 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 client.16938 shutdown 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 -- 172.21.15.39:0/912786773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 msgr2=0x563f44a25bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a25bc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f91140df8e0 tx=0x7f91140e56b0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 -- 172.21.15.39:0/912786773 shutdown_connections 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f44638d80 0x563f44a32db0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a2cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446594d0 0x563f44a25bc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 --2- 172.21.15.39:0/912786773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a25680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f90feffd640 0 client.16938 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 -- 172.21.15.39:0/912786773 >> 172.21.15.39:0/912786773 conn(0x563f4466e300 msgr2=0x563f44a1a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 -- 172.21.15.39:0/912786773 shutdown_connections 2024-05-04T03:25:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.471+0000 7f912c9f9740 1 -- 172.21.15.39:0/912786773 wait complete. 2024-05-04T03:25:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.472+0000 7f912c9f9740 20 client.16938 trim_cache size 0 max 16384 2024-05-04T03:25:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f912c9f9740 1 Processor -- start 2024-05-04T03:25:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f912c9f9740 1 -- start start 2024-05-04T03:25:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a16dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f446594d0 0x563f44a2ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f4467a1b0 0x563f44a17310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f912c9f9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91201b18e0 con 0x563f4467a1b0 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f912c9f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91201b3420 con 0x563f446594d0 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f912c9f9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91201b1fb0 con 0x563f446512f0 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f911ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f4467a1b0 0x563f44a17310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f911ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f4467a1b0 0x563f44a17310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33206/0 (socket says 172.21.15.39:33206) 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f911ffff640 1 -- 172.21.15.39:0/3019512888 learned_addr learned my addr 172.21.15.39:0/3019512888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.487+0000 7f911f7fe640 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a16dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f911effd640 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f446594d0 0x563f44a2ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f911ffff640 1 -- 172.21.15.39:0/3019512888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 msgr2=0x563f44a16dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f911ffff640 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a16dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f911ffff640 1 -- 172.21.15.39:0/3019512888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f446594d0 msgr2=0x563f44a2ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f911ffff640 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f446594d0 0x563f44a2ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f911ffff640 1 -- 172.21.15.39:0/3019512888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f44a12780 con 0x563f4467a1b0 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f911f7fe640 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a16dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f911ffff640 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f4467a1b0 0x563f44a17310 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9118009ca0 tx=0x7f9118041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb0526d19263d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f91057fa640 1 -- 172.21.15.39:0/3019512888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118032da0 con 0x563f4467a1b0 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f91057fa640 1 -- 172.21.15.39:0/3019512888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9118036050 con 0x563f4467a1b0 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.488+0000 7f91057fa640 1 -- 172.21.15.39:0/3019512888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118004180 con 0x563f4467a1b0 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 -- 172.21.15.39:0/3019512888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f4467a1b0 msgr2=0x563f44a17310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f4467a1b0 0x563f44a17310 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9118009ca0 tx=0x7f9118041000 comp rx=0 tx=0).stop 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 -- 172.21.15.39:0/3019512888 shutdown_connections 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f4467a1b0 0x563f44a17310 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f446594d0 0x563f44a2ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 --2- 172.21.15.39:0/3019512888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446512f0 0x563f44a16dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 -- 172.21.15.39:0/3019512888 >> 172.21.15.39:0/3019512888 conn(0x563f4466e300 msgr2=0x563f44a1fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 -- 172.21.15.39:0/3019512888 shutdown_connections 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 -- 172.21.15.39:0/3019512888 wait complete. 2024-05-04T03:25:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 -- start start 2024-05-04T03:25:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.489+0000 7f912c9f9740 1 Processor -- start 2024-05-04T03:25:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446512f0 0x563f44a2a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446594d0 0x563f44a2a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f912c9f9740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a2b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f912c9f9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91201b2d50 con 0x563f446512f0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f912c9f9740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91201b2680 con 0x563f4467a1b0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f912c9f9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91201af8a0 con 0x563f446594d0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f912c9f9740 10 client.0 _ll_register_callbacks cb 0x7f9118004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446512f0 0x563f44a2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446512f0 0x563f44a2a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33218/0 (socket says 172.21.15.39:33218) 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 -- 172.21.15.39:0/2213670757 learned_addr learned my addr 172.21.15.39:0/2213670757 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911effd640 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446594d0 0x563f44a2a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 -- 172.21.15.39:0/2213670757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446594d0 msgr2=0x563f44a2a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446594d0 0x563f44a2a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 -- 172.21.15.39:0/2213670757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 msgr2=0x563f44a2b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a2b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 -- 172.21.15.39:0/2213670757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f44a2b9c0 con 0x563f446512f0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f911f7fe640 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446512f0 0x563f44a2a000 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9110005fd0 tx=0x7f9110011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7d76a16dfbb062 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f90feffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:25:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f90feffd640 1 -- 172.21.15.39:0/2213670757 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110020630 con 0x563f446512f0 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f90feffd640 1 -- 172.21.15.39:0/2213670757 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f91100207d0 con 0x563f446512f0 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.490+0000 7f912c9f9740 1 -- 172.21.15.39:0/2213670757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9118036050 con 0x563f446512f0 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f90feffd640 1 -- 172.21.15.39:0/2213670757 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9110030050 con 0x563f446512f0 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f912c9f9740 10 client.16944 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f912c9f9740 1 -- 172.21.15.39:0/2213670757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f44a2cd20 con 0x563f446512f0 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f90feffd640 1 -- 172.21.15.39:0/2213670757 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9118036050 con 0x563f446512f0 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f90feffd640 1 -- 172.21.15.39:0/2213670757 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9110010430 con 0x563f446512f0 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f912c9f9740 10 client.16944 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f912c9f9740 10 client.16944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f912c9f9740 20 client.16944 populate_metadata read hostname 'smithi039' 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f4460e1c0 0x563f44a9d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f912c9f9740 4 client.16944 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f912c9f9740 1 -- 172.21.15.39:0/2213670757 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f44988050 con 0x563f4460e1c0 2024-05-04T03:25:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.491+0000 7f911effd640 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f4460e1c0 0x563f44a9d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.492+0000 7f90feffd640 10 client.16944 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.492+0000 7f911effd640 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f4460e1c0 0x563f44a9d7d0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4811bddf694b91a1 server_cookie=2ebd1005c1da2554 in_seq=0 out_seq=0 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f90feffd640 1 -- 172.21.15.39:0/2213670757 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f44988050 con 0x563f4460e1c0 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f90feffd640 10 client.16944 handle_command_reply: tid=0 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 client.16944 shutdown 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 -- 172.21.15.39:0/2213670757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446512f0 msgr2=0x563f44a2a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446512f0 0x563f44a2a000 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9110005fd0 tx=0x7f9110011a30 comp rx=0 tx=0).stop 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 -- 172.21.15.39:0/2213670757 shutdown_connections 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563f4460e1c0 0x563f44a9d7d0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f4467a1b0 0x563f44a2b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f446594d0 0x563f44a2a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 --2- 172.21.15.39:0/2213670757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f446512f0 0x563f44a2a000 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f90feffd640 0 client.16944 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 -- 172.21.15.39:0/2213670757 >> 172.21.15.39:0/2213670757 conn(0x563f4466e300 msgr2=0x563f44a1a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 -- 172.21.15.39:0/2213670757 shutdown_connections 2024-05-04T03:25:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.501+0000 7f912c9f9740 1 -- 172.21.15.39:0/2213670757 wait complete. 2024-05-04T03:25:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f912c9f9740 20 client.16944 trim_cache size 0 max 16384 2024-05-04T03:25:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:25:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:25:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:25:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 -- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91080b27e0 msgr2=0x7f91080b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91080b27e0 0x7f91080b4cf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9118004af0 tx=0x7f9118031000 comp rx=0 tx=0).stop 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 -- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 msgr2=0x7f9120102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 0x7f9120102a70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f911400d950 tx=0x7f911400de10 comp rx=0 tx=0).stop 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 -- 172.21.15.39:0/3649801560 shutdown_connections 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f91080b27e0 0x7f91080b4cf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9120105220 0x7f9120102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91201023d0 0x7f9120102a70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 --2- 172.21.15.39:0/3649801560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91201018d0 0x7f9120104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 -- 172.21.15.39:0/3649801560 >> 172.21.15.39:0/3649801560 conn(0x7f91200fd4f0 msgr2=0x7f9120107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 -- 172.21.15.39:0/3649801560 shutdown_connections 2024-05-04T03:25:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:25:51.502+0000 7f9125b3f640 1 -- 172.21.15.39:0/3649801560 wait complete. 2024-05-04T03:25:51.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 720.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:25:52.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:51 smithi066 ceph-mon[33264]: pgmap v838: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-05-04T03:25:52.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:51 smithi167 ceph-mon[33614]: pgmap v838: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-05-04T03:25:52.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:51 smithi039 ceph-mon[29715]: pgmap v838: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-05-04T03:25:54.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:54 smithi167 ceph-mon[33614]: pgmap v839: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-04T03:25:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:54 smithi039 ceph-mon[29715]: pgmap v839: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-04T03:25:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:54 smithi066 ceph-mon[33264]: pgmap v839: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-05-04T03:25:56.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:56 smithi167 ceph-mon[33614]: pgmap v840: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 196 KiB/s wr, 33 op/s 2024-05-04T03:25:56.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:56 smithi039 ceph-mon[29715]: pgmap v840: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 196 KiB/s wr, 33 op/s 2024-05-04T03:25:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 725.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:25:56.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:56 smithi066 ceph-mon[33264]: pgmap v840: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 196 KiB/s wr, 33 op/s 2024-05-04T03:25:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:25:58 smithi167 ceph-mon[33614]: pgmap v841: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 215 KiB/s wr, 39 op/s 2024-05-04T03:25:58.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:25:58 smithi039 ceph-mon[29715]: pgmap v841: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 215 KiB/s wr, 39 op/s 2024-05-04T03:25:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:25:58 smithi066 ceph-mon[33264]: pgmap v841: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 215 KiB/s wr, 39 op/s 2024-05-04T03:26:00.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:00 smithi167 ceph-mon[33614]: pgmap v842: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 222 KiB/s wr, 38 op/s 2024-05-04T03:26:00.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:00 smithi039 ceph-mon[29715]: pgmap v842: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 222 KiB/s wr, 38 op/s 2024-05-04T03:26:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:00 smithi066 ceph-mon[33264]: pgmap v842: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 222 KiB/s wr, 38 op/s 2024-05-04T03:26:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 730.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:26:02.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:02 smithi167 ceph-mon[33614]: pgmap v843: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 209 KiB/s wr, 37 op/s 2024-05-04T03:26:02.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:02 smithi039 ceph-mon[29715]: pgmap v843: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 209 KiB/s wr, 37 op/s 2024-05-04T03:26:02.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:02 smithi066 ceph-mon[33264]: pgmap v843: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 209 KiB/s wr, 37 op/s 2024-05-04T03:26:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 -- 172.21.15.39:0/1354435831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c104eb0 msgr2=0x7fce3c1052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 --2- 172.21.15.39:0/1354435831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c104eb0 0x7fce3c1052c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fce30009900 tx=0x7fce3002a410 comp rx=0 tx=0).stop 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 -- 172.21.15.39:0/1354435831 shutdown_connections 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 --2- 172.21.15.39:0/1354435831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce3c109bb0 0x7fce3c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 --2- 172.21.15.39:0/1354435831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c104eb0 0x7fce3c1052c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 --2- 172.21.15.39:0/1354435831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce3c103c70 0x7fce3c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 -- 172.21.15.39:0/1354435831 >> 172.21.15.39:0/1354435831 conn(0x7fce3c0fee40 msgr2=0x7fce3c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 -- 172.21.15.39:0/1354435831 shutdown_connections 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.615+0000 7fce421f3640 1 -- 172.21.15.39:0/1354435831 wait complete. 2024-05-04T03:26:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce421f3640 1 Processor -- start 2024-05-04T03:26:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce421f3640 1 -- start start 2024-05-04T03:26:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce421f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c103c70 0x7fce3c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce421f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce3c104eb0 0x7fce3c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce421f3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce3c109bb0 0x7fce3c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce421f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10f620 con 0x7fce3c103c70 2024-05-04T03:26:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce421f3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10f4a0 con 0x7fce3c109bb0 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce421f3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10f7a0 con 0x7fce3c104eb0 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce3b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c103c70 0x7fce3c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce3b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c103c70 0x7fce3c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40956/0 (socket says 172.21.15.39:40956) 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce3b7fe640 1 -- 172.21.15.39:0/1973965589 learned_addr learned my addr 172.21.15.39:0/1973965589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce3bfff640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce3c109bb0 0x7fce3c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.616+0000 7fce3affd640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce3c104eb0 0x7fce3c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce3b7fe640 1 -- 172.21.15.39:0/1973965589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce3c104eb0 msgr2=0x7fce3c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce3c104eb0 0x7fce3c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce3b7fe640 1 -- 172.21.15.39:0/1973965589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce3c109bb0 msgr2=0x7fce3c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce3c109bb0 0x7fce3c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce3b7fe640 1 -- 172.21.15.39:0/1973965589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c1a1dc0 con 0x7fce3c103c70 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c103c70 0x7fce3c19a280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fce2c005fd0 tx=0x7fce2c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce38ff9640 1 -- 172.21.15.39:0/1973965589 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c017070 con 0x7fce3c103c70 2024-05-04T03:26:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce38ff9640 1 -- 172.21.15.39:0/1973965589 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce2c009c30 con 0x7fce3c103c70 2024-05-04T03:26:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce421f3640 1 -- 172.21.15.39:0/1973965589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c1a20b0 con 0x7fce3c103c70 2024-05-04T03:26:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce38ff9640 1 -- 172.21.15.39:0/1973965589 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c003c90 con 0x7fce3c103c70 2024-05-04T03:26:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.617+0000 7fce421f3640 1 -- 172.21.15.39:0/1973965589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c1a2600 con 0x7fce3c103c70 2024-05-04T03:26:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce4917f740 1 Processor -- start 2024-05-04T03:26:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce4917f740 1 -- start start 2024-05-04T03:26:04.627 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce38ff9640 1 -- 172.21.15.39:0/1973965589 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fce2c003e30 con 0x7fce3c103c70 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce4917f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ad8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce4917f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb070e2f0 0x557fb0ad8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce4917f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0adfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce4917f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557fb0a3d5d0 con 0x557fb070e2f0 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce4917f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557fb09f6890 con 0x557fb07164d0 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce4917f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557fb0a39590 con 0x557fb07371b0 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce3b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb070e2f0 0x557fb0ad8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce3affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0adfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce3b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb070e2f0 0x557fb0ad8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40958/0 (socket says 172.21.15.39:40958) 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce3b7fe640 1 -- 172.21.15.39:0/3264336555 learned_addr learned my addr 172.21.15.39:0/3264336555 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.620+0000 7fce3bfff640 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ad8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce3b7fe640 1 -- 172.21.15.39:0/3264336555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 msgr2=0x557fb0ad8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ad8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce3b7fe640 1 -- 172.21.15.39:0/3264336555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 msgr2=0x557fb0adfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0adfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce3b7fe640 1 -- 172.21.15.39:0/3264336555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fb074f210 con 0x557fb070e2f0 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce3affd640 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0adfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce38ff9640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce240b27e0 0x7fce240b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce3bfff640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce240b27e0 0x7fce240b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb070e2f0 0x557fb0ad8fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0ea0e0 tx=0x7fce2c0ea5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc4173b52938011 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce38ff9640 1 -- 172.21.15.39:0/1973965589 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fce2c0e1ef0 con 0x7fce3c103c70 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce21ffb640 1 -- 172.21.15.39:0/3264336555 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0a53e0 con 0x557fb070e2f0 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce21ffb640 1 -- 172.21.15.39:0/3264336555 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce2c0035f0 con 0x557fb070e2f0 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.621+0000 7fce21ffb640 1 -- 172.21.15.39:0/3264336555 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0a8c90 con 0x557fb070e2f0 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 -- 172.21.15.39:0/3264336555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb070e2f0 msgr2=0x557fb0ad8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce3bfff640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce240b27e0 0x7fce240b4cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x557fb0ad9c30 tx=0x7fce3400a4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb070e2f0 0x557fb0ad8fe0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0ea0e0 tx=0x7fce2c0ea5a0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 -- 172.21.15.39:0/3264336555 shutdown_connections 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0adfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb070e2f0 0x557fb0ad8fe0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 --2- 172.21.15.39:0/3264336555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ad8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 -- 172.21.15.39:0/3264336555 >> 172.21.15.39:0/3264336555 conn(0x557fb072b300 msgr2=0x557fb0ad43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 -- 172.21.15.39:0/3264336555 shutdown_connections 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.622+0000 7fce4917f740 1 -- 172.21.15.39:0/3264336555 wait complete. 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 1 -- start start 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 1 Processor -- start 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb070e2f0 0x557fb0ae2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0ae2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07371b0 0x557fb0ae9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557fb0ae0930 con 0x557fb07371b0 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557fb09fa370 con 0x557fb07164d0 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557fb0ae0ab0 con 0x557fb070e2f0 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce4917f740 10 client.0 _ll_register_callbacks cb 0x557fb074ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce3bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07371b0 0x557fb0ae9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce3bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07371b0 0x557fb0ae9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40966/0 (socket says 172.21.15.39:40966) 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce3bfff640 1 -- 172.21.15.39:0/224546375 learned_addr learned my addr 172.21.15.39:0/224546375 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb070e2f0 0x557fb0ae2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.623+0000 7fce3affd640 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0ae2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce3bfff640 1 -- 172.21.15.39:0/224546375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb070e2f0 msgr2=0x557fb0ae2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce3bfff640 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb070e2f0 0x557fb0ae2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce3bfff640 1 -- 172.21.15.39:0/224546375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 msgr2=0x557fb0ae2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce3bfff640 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0ae2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce3bfff640 1 -- 172.21.15.39:0/224546375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fb081b620 con 0x557fb07371b0 2024-05-04T03:26:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce3bfff640 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07371b0 0x557fb0ae9ae0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fce3401cc00 tx=0x7fce34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b294cfc5c7382d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce1affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce1affd640 1 -- 172.21.15.39:0/224546375 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3402c060 con 0x557fb07371b0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce1affd640 1 -- 172.21.15.39:0/224546375 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fce34016030 con 0x557fb07371b0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce4917f740 1 -- 172.21.15.39:0/224546375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557fb0a4b1e0 con 0x557fb07371b0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce1affd640 1 -- 172.21.15.39:0/224546375 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fce3401cdb0 con 0x557fb07371b0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce4917f740 10 client.16956 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce4917f740 1 -- 172.21.15.39:0/224546375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557fb0aecae0 con 0x557fb07371b0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.624+0000 7fce1affd640 1 -- 172.21.15.39:0/224546375 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x557fb0a4b1e0 con 0x557fb07371b0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.625+0000 7fce1affd640 1 -- 172.21.15.39:0/224546375 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fce340278a0 con 0x557fb07371b0 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.625+0000 7fce4917f740 10 client.16956 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.625+0000 7fce4917f740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.625+0000 7fce4917f740 20 client.16956 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.625+0000 7fce4917f740 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557fb06f5d80 0x557fb0af01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.625+0000 7fce4917f740 4 client.16956 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:26:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.625+0000 7fce4917f740 1 -- 172.21.15.39:0/224546375 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fb0a45050 con 0x557fb06f5d80 2024-05-04T03:26:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.625+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557fb06f5d80 0x557fb0af01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.626+0000 7fce1affd640 10 client.16956 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:04.633 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.626+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557fb06f5d80 0x557fb0af01b0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9374963948953beb server_cookie=ecfd50eaa1a52883 in_seq=0 out_seq=0 2024-05-04T03:26:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.626+0000 7fce1affd640 1 -- 172.21.15.39:0/224546375 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557fb0a45050 con 0x557fb06f5d80 2024-05-04T03:26:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.626+0000 7fce1affd640 10 client.16956 handle_command_reply: tid=0 2024-05-04T03:26:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 client.16956 shutdown 2024-05-04T03:26:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 -- 172.21.15.39:0/224546375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07371b0 msgr2=0x557fb0ae9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07371b0 0x557fb0ae9ae0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fce3401cc00 tx=0x7fce34004ab0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 -- 172.21.15.39:0/224546375 shutdown_connections 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557fb06f5d80 0x557fb0af01b0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07371b0 0x557fb0ae9ae0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb07164d0 0x557fb0ae2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 --2- 172.21.15.39:0/224546375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb070e2f0 0x557fb0ae2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce1affd640 0 client.16956 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 -- 172.21.15.39:0/224546375 >> 172.21.15.39:0/224546375 conn(0x557fb072b300 msgr2=0x557fb0ad7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 -- 172.21.15.39:0/224546375 shutdown_connections 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 1 -- 172.21.15.39:0/224546375 wait complete. 2024-05-04T03:26:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.627+0000 7fce4917f740 20 client.16956 trim_cache size 0 max 16384 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce4917f740 1 Processor -- start 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce4917f740 1 -- start start 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce4917f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 0x557fb0ad4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce4917f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce4917f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ae9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce4917f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c1ab3c0 con 0x557fb07164d0 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce4917f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c1acf00 con 0x557fb070e2f0 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce4917f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c1aba90 con 0x557fb07371b0 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40974/0 (socket says 172.21.15.39:40974) 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 -- 172.21.15.39:0/903930971 learned_addr learned my addr 172.21.15.39:0/903930971 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 -- 172.21.15.39:0/903930971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 msgr2=0x557fb0ae9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 --2- 172.21.15.39:0/903930971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ae9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 -- 172.21.15.39:0/903930971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 msgr2=0x557fb0ad4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/903930971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 0x557fb0ad4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 --2- 172.21.15.39:0/903930971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 0x557fb0ad4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 -- 172.21.15.39:0/903930971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fb0ad4d00 con 0x557fb07164d0 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.643+0000 7fce3affd640 1 --2- 172.21.15.39:0/903930971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae9a00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fce30005ab0 tx=0x7fce300317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4394752700de1e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce217fa640 1 -- 172.21.15.39:0/903930971 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30035040 con 0x557fb07164d0 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce217fa640 1 -- 172.21.15.39:0/903930971 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce300044c0 con 0x557fb07164d0 2024-05-04T03:26:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce217fa640 1 -- 172.21.15.39:0/903930971 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30003cc0 con 0x557fb07164d0 2024-05-04T03:26:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce4917f740 1 -- 172.21.15.39:0/903930971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 msgr2=0x557fb0ae9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce4917f740 1 --2- 172.21.15.39:0/903930971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae9a00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fce30005ab0 tx=0x7fce300317f0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce4917f740 1 -- 172.21.15.39:0/903930971 shutdown_connections 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce4917f740 1 --2- 172.21.15.39:0/903930971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ae9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce4917f740 1 --2- 172.21.15.39:0/903930971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae9a00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce4917f740 1 --2- 172.21.15.39:0/903930971 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 0x557fb0ad4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce4917f740 1 -- 172.21.15.39:0/903930971 >> 172.21.15.39:0/903930971 conn(0x557fb072b300 msgr2=0x557fb0ad3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.644+0000 7fce4917f740 1 -- 172.21.15.39:0/903930971 shutdown_connections 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 -- 172.21.15.39:0/903930971 wait complete. 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 -- start start 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 Processor -- start 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 0x557fb0ae7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ae8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c1ac830 con 0x557fb07164d0 2024-05-04T03:26:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c1ac160 con 0x557fb070e2f0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c1a93a0 con 0x557fb07371b0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce4917f740 10 client.0 _ll_register_callbacks cb 0x7fce3400a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce3affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.645+0000 7fce3affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40986/0 (socket says 172.21.15.39:40986) 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce3affd640 1 -- 172.21.15.39:0/3831274774 learned_addr learned my addr 172.21.15.39:0/3831274774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 0x557fb0ae7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce3affd640 1 -- 172.21.15.39:0/3831274774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 msgr2=0x557fb0ae8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce3affd640 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ae8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce3affd640 1 -- 172.21.15.39:0/3831274774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 msgr2=0x557fb0ae7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce3affd640 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 0x557fb0ae7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce3affd640 1 -- 172.21.15.39:0/3831274774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fb0ae91a0 con 0x557fb07164d0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce3affd640 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae7cb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fce30005760 tx=0x7fce300042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81828a1cbc6ad220 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce1affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce1affd640 1 -- 172.21.15.39:0/3831274774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30035030 con 0x557fb07164d0 2024-05-04T03:26:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce1affd640 1 -- 172.21.15.39:0/3831274774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fce300039c0 con 0x557fb07164d0 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce4917f740 1 -- 172.21.15.39:0/3831274774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557fb09c05f0 con 0x557fb07164d0 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce1affd640 1 -- 172.21.15.39:0/3831274774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fce30003bc0 con 0x557fb07164d0 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce4917f740 10 client.16962 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.646+0000 7fce4917f740 1 -- 172.21.15.39:0/3831274774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557fb0aea460 con 0x557fb07164d0 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce1affd640 1 -- 172.21.15.39:0/3831274774 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x557fb09c05f0 con 0x557fb07164d0 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce1affd640 1 -- 172.21.15.39:0/3831274774 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fce3003c440 con 0x557fb07164d0 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce4917f740 10 client.16962 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce4917f740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce4917f740 20 client.16962 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce4917f740 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557fb06cb1c0 0x557fb0b5ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce4917f740 4 client.16962 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce4917f740 1 -- 172.21.15.39:0/3831274774 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557fb0a0dc80 con 0x557fb06cb1c0 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.647+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557fb06cb1c0 0x557fb0b5ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.648+0000 7fce1affd640 10 client.16962 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.648+0000 7fce3b7fe640 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557fb06cb1c0 0x557fb0b5ac90 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550f6a2879c8c70e server_cookie=23bcea540085603d in_seq=0 out_seq=0 2024-05-04T03:26:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce1affd640 1 -- 172.21.15.39:0/3831274774 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557fb0a0dc80 con 0x557fb06cb1c0 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce1affd640 10 client.16962 handle_command_reply: tid=0 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 client.16962 shutdown 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 -- 172.21.15.39:0/3831274774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 msgr2=0x557fb0ae7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae7cb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fce30005760 tx=0x7fce300042e0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 -- 172.21.15.39:0/3831274774 shutdown_connections 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x557fb06cb1c0 0x557fb0b5ac90 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fb07371b0 0x557fb0ae8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557fb07164d0 0x557fb0ae7cb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce1affd640 0 client.16962 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 --2- 172.21.15.39:0/3831274774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557fb070e2f0 0x557fb0ae7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 -- 172.21.15.39:0/3831274774 >> 172.21.15.39:0/3831274774 conn(0x557fb072b300 msgr2=0x557fb0af49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 -- 172.21.15.39:0/3831274774 shutdown_connections 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 1 -- 172.21.15.39:0/3831274774 wait complete. 2024-05-04T03:26:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.657+0000 7fce4917f740 20 client.16962 trim_cache size 0 max 16384 2024-05-04T03:26:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:26:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:26:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:26:04.661 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 -- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce240b27e0 msgr2=0x7fce240b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce240b27e0 0x7fce240b4cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x557fb0ad9c30 tx=0x7fce3400a4d0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 -- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c103c70 msgr2=0x7fce3c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c103c70 0x7fce3c19a280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fce2c005fd0 tx=0x7fce2c00dde0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 -- 172.21.15.39:0/1973965589 shutdown_connections 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce240b27e0 0x7fce240b4cf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce3c109bb0 0x7fce3c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce3c104eb0 0x7fce3c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 --2- 172.21.15.39:0/1973965589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce3c103c70 0x7fce3c19a280 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 -- 172.21.15.39:0/1973965589 >> 172.21.15.39:0/1973965589 conn(0x7fce3c0fee40 msgr2=0x7fce3c105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 -- 172.21.15.39:0/1973965589 shutdown_connections 2024-05-04T03:26:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:04.658+0000 7fce421f3640 1 -- 172.21.15.39:0/1973965589 wait complete. 2024-05-04T03:26:04.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:04 smithi167 ceph-mon[33614]: pgmap v844: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-05-04T03:26:04.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:04 smithi039 ceph-mon[29715]: pgmap v844: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-05-04T03:26:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:04 smithi066 ceph-mon[33264]: pgmap v844: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-05-04T03:26:05.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:26:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:06.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:06 smithi167 ceph-mon[33614]: pgmap v845: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 212 KiB/s wr, 35 op/s 2024-05-04T03:26:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:06 smithi039 ceph-mon[29715]: pgmap v845: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 212 KiB/s wr, 35 op/s 2024-05-04T03:26:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 735.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:26:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:06 smithi066 ceph-mon[33264]: pgmap v845: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 212 KiB/s wr, 35 op/s 2024-05-04T03:26:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2186249107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bcc10e8d0 msgr2=0x7f3bcc110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2186249107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bcc10e8d0 0x7f3bcc110db0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b0a0 tx=0x7f3bc400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2186249107 shutdown_connections 2024-05-04T03:26:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2186249107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bcc10e8d0 0x7f3bcc110db0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2186249107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc105460 0x7f3bcc105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2186249107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc105db0 0x7f3bcc10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2186249107 >> 172.21.15.39:0/2186249107 conn(0x7f3bcc0fedc0 msgr2=0x7f3bcc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2186249107 shutdown_connections 2024-05-04T03:26:07.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.239+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2186249107 wait complete. 2024-05-04T03:26:07.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.240+0000 7f3bd3bdd700 1 Processor -- start 2024-05-04T03:26:07.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.240+0000 7f3bd3bdd700 1 -- start start 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd3bdd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bcc105460 0x7f3bcc1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd3bdd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc105db0 0x7f3bcc1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd3bdd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10e8d0 0x7f3bcc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd3bdd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc114310 con 0x7f3bcc105460 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd3bdd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc114190 con 0x7f3bcc105db0 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd3bdd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc114490 con 0x7f3bcc10e8d0 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1178700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc105db0 0x7f3bcc1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1178700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc105db0 0x7f3bcc1a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:33712/0 (socket says 172.21.15.39:33712) 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1178700 1 -- 172.21.15.39:0/2919948003 learned_addr learned my addr 172.21.15.39:0/2919948003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1979700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bcc105460 0x7f3bcc1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd217a700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10e8d0 0x7f3bcc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1178700 1 -- 172.21.15.39:0/2919948003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10e8d0 msgr2=0x7f3bcc19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1178700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10e8d0 0x7f3bcc19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1178700 1 -- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bcc105460 msgr2=0x7f3bcc1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1178700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bcc105460 0x7f3bcc1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.241+0000 7f3bd1178700 1 -- 172.21.15.39:0/2919948003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc19b1b0 con 0x7f3bcc105db0 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.242+0000 7f3bd1178700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc105db0 0x7f3bcc1a0df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00e9c0 tx=0x7f3bbc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.242+0000 7f3bc2ffd700 1 -- 172.21.15.39:0/2919948003 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00cbd0 con 0x7f3bcc105db0 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.242+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc19b440 con 0x7f3bcc105db0 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.242+0000 7f3bc2ffd700 1 -- 172.21.15.39:0/2919948003 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bbc00ee60 con 0x7f3bcc105db0 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.242+0000 7f3bc2ffd700 1 -- 172.21.15.39:0/2919948003 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0054b0 con 0x7f3bcc105db0 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.242+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a7740 con 0x7f3bcc105db0 2024-05-04T03:26:07.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.243+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc0512c0 con 0x7f3bcc105db0 2024-05-04T03:26:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.245+0000 7f3bc2ffd700 1 -- 172.21.15.39:0/2919948003 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3bbc004750 con 0x7f3bcc105db0 2024-05-04T03:26:07.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.246+0000 7f3bc2ffd700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3bb80b2880 0x7f3bb80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:07.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.246+0000 7f3bd1979700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3bb80b2880 0x7f3bb80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:07.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.246+0000 7f3bc2ffd700 1 -- 172.21.15.39:0/2919948003 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3bbc0e2440 con 0x7f3bcc105db0 2024-05-04T03:26:07.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.247+0000 7f3bd1979700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3bb80b2880 0x7f3bb80b4d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc19bc10 tx=0x7f3bc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:07.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.251+0000 7f3bc2ffd700 1 -- 172.21.15.39:0/2919948003 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3bbc0aa170 con 0x7f3bcc105db0 2024-05-04T03:26:07.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.535+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bcc02e5f0 con 0x7f3bcc105db0 2024-05-04T03:26:07.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.536+0000 7f3bc2ffd700 1 -- 172.21.15.39:0/2919948003 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f3bbc0a9a70 con 0x7f3bcc105db0 2024-05-04T03:26:07.538 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:26:07.539 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:26:07.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3bb80b2880 msgr2=0x7f3bb80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:07.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3bb80b2880 0x7f3bb80b4d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc19bc10 tx=0x7f3bc8009260 comp rx=0 tx=0).stop 2024-05-04T03:26:07.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc105db0 msgr2=0x7f3bcc1a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:07.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc105db0 0x7f3bcc1a0df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00e9c0 tx=0x7f3bbc00ecd0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 shutdown_connections 2024-05-04T03:26:07.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3bb80b2880 0x7f3bb80b4d90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bcc105460 0x7f3bcc1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bcc10e8d0 0x7f3bcc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 --2- 172.21.15.39:0/2919948003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bcc105db0 0x7f3bcc1a0df0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:07.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 >> 172.21.15.39:0/2919948003 conn(0x7f3bcc0fedc0 msgr2=0x7f3bcc0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:07.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 shutdown_connections 2024-05-04T03:26:07.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:07.539+0000 7f3bd3bdd700 1 -- 172.21.15.39:0/2919948003 wait complete. 2024-05-04T03:26:07.542 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:26:07.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:26:07.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:26:07.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:26:07.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:26:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:26:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:26:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:26:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:26:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:26:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:26:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:26:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:26:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:26:08.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id b55298cb --release --await' 2024-05-04T03:26:08.231 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id b55298cb --release --await 2024-05-04T03:26:08.491 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:08 smithi039 ceph-mon[29715]: pgmap v846: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 197 KiB/s wr, 37 op/s 2024-05-04T03:26:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:08 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-04T03:26:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:08 smithi039 ceph-mon[29715]: Unable to set osd_memory_target on smithi039 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-04T03:26:08.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:08 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2919948003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:08.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:08 smithi066 ceph-mon[33264]: pgmap v846: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 197 KiB/s wr, 37 op/s 2024-05-04T03:26:08.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:08 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-04T03:26:08.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:08 smithi066 ceph-mon[33264]: Unable to set osd_memory_target on smithi039 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-04T03:26:08.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:08 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2919948003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:08 smithi167 ceph-mon[33614]: pgmap v846: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 197 KiB/s wr, 37 op/s 2024-05-04T03:26:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:08 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi039 to 176.2M 2024-05-04T03:26:08.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:08 smithi167 ceph-mon[33614]: Unable to set osd_memory_target on smithi039 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-04T03:26:08.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:08 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2919948003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4185545393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b41045f0 msgr2=0x7f19b4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4185545393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b41045f0 0x7f19b4104a00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f19a80099c0 tx=0x7f19a8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4185545393 shutdown_connections 2024-05-04T03:26:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4185545393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b41045f0 0x7f19b4104a00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4185545393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19b4109870 0x7f19b41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4185545393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b4108f20 0x7f19b4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4185545393 >> 172.21.15.39:0/4185545393 conn(0x7f19b40ffec0 msgr2=0x7f19b4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4185545393 shutdown_connections 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.408+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4185545393 wait complete. 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bc7f3700 1 Processor -- start 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bc7f3700 1 -- start start 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bc7f3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b41045f0 0x7f19b419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bc7f3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b4108f20 0x7f19b419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bc7f3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19b4109870 0x7f19b419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bc7f3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4114350 con 0x7f19b4108f20 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bc7f3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41141d0 con 0x7f19b41045f0 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bc7f3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41144d0 con 0x7f19b4109870 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19b9d8e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b4108f20 0x7f19b419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19b9d8e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b4108f20 0x7f19b419b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47630/0 (socket says 172.21.15.39:47630) 2024-05-04T03:26:10.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19b9d8e700 1 -- 172.21.15.39:0/4236740440 learned_addr learned my addr 172.21.15.39:0/4236740440 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19ba58f700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b41045f0 0x7f19b419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bad90700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19b4109870 0x7f19b419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19b9d8e700 1 -- 172.21.15.39:0/4236740440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19b4109870 msgr2=0x7f19b419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19b4109870 0x7f19b419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19b9d8e700 1 -- 172.21.15.39:0/4236740440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b41045f0 msgr2=0x7f19b419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b41045f0 0x7f19b419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19b9d8e700 1 -- 172.21.15.39:0/4236740440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b419fbf0 con 0x7f19b4108f20 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19bad90700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19b4109870 0x7f19b419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.409+0000 7f19ba58f700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b41045f0 0x7f19b419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.410+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b4108f20 0x7f19b419b2c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c8f0 tx=0x7f19b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.410+0000 7f19a77fe700 1 -- 172.21.15.39:0/4236740440 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0007b00 con 0x7f19b4108f20 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.410+0000 7f19a77fe700 1 -- 172.21.15.39:0/4236740440 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19b0007ca0 con 0x7f19b4108f20 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.410+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4236740440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b419fee0 con 0x7f19b4108f20 2024-05-04T03:26:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.410+0000 7f19a77fe700 1 -- 172.21.15.39:0/4236740440 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0018660 con 0x7f19b4108f20 2024-05-04T03:26:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.410+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4236740440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b41a7800 con 0x7f19b4108f20 2024-05-04T03:26:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.413+0000 7f19a77fe700 1 -- 172.21.15.39:0/4236740440 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f19b000f450 con 0x7f19b4108f20 2024-05-04T03:26:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.414+0000 7f19a77fe700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19a00b2880 0x7f19a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.414+0000 7f19ba58f700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19a00b2880 0x7f19a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.414+0000 7f19a77fe700 1 -- 172.21.15.39:0/4236740440 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f19b00e2210 con 0x7f19b4108f20 2024-05-04T03:26:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.414+0000 7f19cb207b80 1 Processor -- start 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.414+0000 7f19ba58f700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19a00b2880 0x7f19a00b4d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f19a8000c00 tx=0x7f19a8005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.414+0000 7f19cb207b80 1 -- start start 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fc5d0 0x55c5995fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 0x55c5996073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c599609d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19cb207b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c59951f960 con 0x55c5996078e0 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19cb207b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c5994be8c0 con 0x55c5995fcf20 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19cb207b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c59951b920 con 0x55c5995fc5d0 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c599609d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c599609d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47644/0 (socket says 172.21.15.39:47644) 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 -- 172.21.15.39:0/1020123240 learned_addr learned my addr 172.21.15.39:0/1020123240 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 0x55c5996073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19ba58f700 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fc5d0 0x55c5995fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 -- 172.21.15.39:0/1020123240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fc5d0 msgr2=0x55c5995fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fc5d0 0x55c5995fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 -- 172.21.15.39:0/1020123240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 msgr2=0x55c5996073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 0x55c5996073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 -- 172.21.15.39:0/1020123240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59946c9d0 con 0x55c5996078e0 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19ba58f700 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fc5d0 0x55c5995fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.415+0000 7f19bad90700 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c599609d00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009cd0 tx=0x7f19ac007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff405de1b8cfd6f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.416+0000 7f19a4ff9700 1 -- 172.21.15.39:0/1020123240 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac019070 con 0x55c5996078e0 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.416+0000 7f19a4ff9700 1 -- 172.21.15.39:0/1020123240 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19ac014890 con 0x55c5996078e0 2024-05-04T03:26:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.416+0000 7f19a4ff9700 1 -- 172.21.15.39:0/1020123240 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0137c0 con 0x55c5996078e0 2024-05-04T03:26:10.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 -- 172.21.15.39:0/1020123240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 msgr2=0x55c599609d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c599609d00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009cd0 tx=0x7f19ac007500 comp rx=0 tx=0).stop 2024-05-04T03:26:10.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 -- 172.21.15.39:0/1020123240 shutdown_connections 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c599609d00 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fc5d0 0x55c5995fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1020123240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 0x55c5996073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 -- 172.21.15.39:0/1020123240 >> 172.21.15.39:0/1020123240 conn(0x55c59947ff40 msgr2=0x55c5995f8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 -- 172.21.15.39:0/1020123240 shutdown_connections 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 -- 172.21.15.39:0/1020123240 wait complete. 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 -- start start 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.417+0000 7f19cb207b80 1 Processor -- start 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c5996048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c599604e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5996078e0 0x55c599605360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19cb207b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c59960a670 con 0x55c5995fc5d0 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19cb207b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c599485d30 con 0x55c5996078e0 2024-05-04T03:26:10.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19cb207b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c59960a7f0 con 0x55c5995fcf20 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19cb207b80 10 client.0 _ll_register_callbacks cb 0x55c5994b7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c5996048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c5996048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47656/0 (socket says 172.21.15.39:47656) 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 -- 172.21.15.39:0/1943922120 learned_addr learned my addr 172.21.15.39:0/1943922120 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19bad90700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5996078e0 0x55c599605360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c599604e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 -- 172.21.15.39:0/1943922120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 msgr2=0x55c599604e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c599604e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 -- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5996078e0 msgr2=0x55c599605360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5996078e0 0x55c599605360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 -- 172.21.15.39:0/1943922120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5994bb790 con 0x55c5995fc5d0 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19bad90700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5996078e0 0x55c599605360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c599604e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:10.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19ba58f700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c5996048e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f19a80099c0 tx=0x7f19a8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8598f225e41699 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f199dffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f199dffb700 1 -- 172.21.15.39:0/1943922120 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8033060 con 0x55c5995fc5d0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f19cb207b80 1 -- 172.21.15.39:0/1943922120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ac014890 con 0x55c5995fc5d0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.418+0000 7f199dffb700 1 -- 172.21.15.39:0/1943922120 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f19a801e030 con 0x55c5995fc5d0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f199dffb700 1 -- 172.21.15.39:0/1943922120 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19a802e410 con 0x55c5995fc5d0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19cb207b80 10 client.16980 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19cb207b80 1 -- 172.21.15.39:0/1943922120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c599615320 con 0x55c5995fc5d0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f199dffb700 1 -- 172.21.15.39:0/1943922120 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19ac014890 con 0x55c5995fc5d0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f199dffb700 1 -- 172.21.15.39:0/1943922120 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f19a8024530 con 0x55c5995fc5d0 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19cb207b80 10 client.16980 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19cb207b80 10 client.16980 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19cb207b80 20 client.16980 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c5996178b0 0x55c599619cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19cb207b80 4 client.16980 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19cb207b80 1 -- 172.21.15.39:0/1943922120 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c59952d340 con 0x55c5996178b0 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.419+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c5996178b0 0x55c599619cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.420+0000 7f199dffb700 10 client.16980 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.420+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c5996178b0 0x55c599619cf0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6229214d6c6a07 server_cookie=f657b61f179050c8 in_seq=0 out_seq=0 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f199dffb700 1 -- 172.21.15.39:0/1943922120 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c59952d340 con 0x55c5996178b0 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f199dffb700 10 client.16980 handle_command_reply: tid=0 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 client.16980 shutdown 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 -- 172.21.15.39:0/1943922120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 msgr2=0x55c5996048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c5996048e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f19a80099c0 tx=0x7f19a8009310 comp rx=0 tx=0).stop 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 -- 172.21.15.39:0/1943922120 shutdown_connections 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c5996178b0 0x55c599619cf0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c5996048e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5996078e0 0x55c599605360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 --2- 172.21.15.39:0/1943922120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c599604e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 -- 172.21.15.39:0/1943922120 >> 172.21.15.39:0/1943922120 conn(0x55c59947ff40 msgr2=0x55c5995fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:10.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f199dffb700 0 client.16980 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 -- 172.21.15.39:0/1943922120 shutdown_connections 2024-05-04T03:26:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 1 -- 172.21.15.39:0/1943922120 wait complete. 2024-05-04T03:26:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.421+0000 7f19cb207b80 20 client.16980 trim_cache size 0 max 16384 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.441+0000 7f19cb207b80 1 Processor -- start 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.441+0000 7f19cb207b80 1 -- start start 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fc5d0 0x55c599605230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c59961f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c59961f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19cb207b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41aebd0 con 0x55c5996078e0 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19cb207b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41b0710 con 0x55c5995fc5d0 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19cb207b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41af2a0 con 0x55c5995fcf20 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c59961f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c59961f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47662/0 (socket says 172.21.15.39:47662) 2024-05-04T03:26:10.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 -- 172.21.15.39:0/2729494914 learned_addr learned my addr 172.21.15.39:0/2729494914 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19ba58f700 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fc5d0 0x55c599605230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c59961f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 -- 172.21.15.39:0/2729494914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 msgr2=0x55c59961f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c59961f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 -- 172.21.15.39:0/2729494914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fc5d0 msgr2=0x55c599605230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fc5d0 0x55c599605230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 -- 172.21.15.39:0/2729494914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5995f1a60 con 0x55c5996078e0 2024-05-04T03:26:10.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c59961f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19ba58f700 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fc5d0 0x55c599605230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.442+0000 7f19bad90700 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c59961f9f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000f60 tx=0x7f19ac009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2412f33e9ff3f3de server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.443+0000 7f199ffff700 1 -- 172.21.15.39:0/2729494914 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac021070 con 0x55c5996078e0 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.443+0000 7f199ffff700 1 -- 172.21.15.39:0/2729494914 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19ac004e10 con 0x55c5996078e0 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.443+0000 7f199ffff700 1 -- 172.21.15.39:0/2729494914 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac019040 con 0x55c5996078e0 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.443+0000 7f19cb207b80 1 -- 172.21.15.39:0/2729494914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 msgr2=0x55c59961f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.443+0000 7f19cb207b80 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c59961f9f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000f60 tx=0x7f19ac009860 comp rx=0 tx=0).stop 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 -- 172.21.15.39:0/2729494914 shutdown_connections 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5996078e0 0x55c59961f9f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fc5d0 0x55c599605230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 --2- 172.21.15.39:0/2729494914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5995fcf20 0x55c59961f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 -- 172.21.15.39:0/2729494914 >> 172.21.15.39:0/2729494914 conn(0x55c59947ff40 msgr2=0x55c5996126f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 -- 172.21.15.39:0/2729494914 shutdown_connections 2024-05-04T03:26:10.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 -- 172.21.15.39:0/2729494914 wait complete. 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 -- start start 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 Processor -- start 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c59960dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 0x55c59960e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5996078e0 0x55c59967b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41b0040 con 0x55c5995fc5d0 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41af970 con 0x55c5995fcf20 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41acb70 con 0x55c5996078e0 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.444+0000 7f19cb207b80 10 client.0 _ll_register_callbacks cb 0x55c5994b7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c59960dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c59960dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47670/0 (socket says 172.21.15.39:47670) 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 -- 172.21.15.39:0/3295687409 learned_addr learned my addr 172.21.15.39:0/3295687409 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19bad90700 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5996078e0 0x55c59967b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 0x55c59960e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 -- 172.21.15.39:0/3295687409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5996078e0 msgr2=0x55c59967b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5996078e0 0x55c59967b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 -- 172.21.15.39:0/3295687409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 msgr2=0x55c59960e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 0x55c59960e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 -- 172.21.15.39:0/3295687409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c59967b5b0 con 0x55c5995fc5d0 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19bad90700 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5996078e0 0x55c59967b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19ba58f700 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c59960dea0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009990 tx=0x7f19a8024540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91da13a7c11b07b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f199dffb700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f199dffb700 1 -- 172.21.15.39:0/3295687409 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a803d020 con 0x55c5995fc5d0 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f199dffb700 1 -- 172.21.15.39:0/3295687409 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f19a801e040 con 0x55c5995fc5d0 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19cb207b80 1 -- 172.21.15.39:0/3295687409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ac004e10 con 0x55c5995fc5d0 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f199dffb700 1 -- 172.21.15.39:0/3295687409 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19a8033040 con 0x55c5995fc5d0 2024-05-04T03:26:10.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19cb207b80 10 client.16986 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.445+0000 7f19cb207b80 1 -- 172.21.15.39:0/3295687409 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c5996827d0 con 0x55c5995fc5d0 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f199dffb700 1 -- 172.21.15.39:0/3295687409 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19ac004e10 con 0x55c5995fc5d0 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f199dffb700 1 -- 172.21.15.39:0/3295687409 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f19a8005020 con 0x55c5995fc5d0 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f19cb207b80 10 client.16986 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f19cb207b80 10 client.16986 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f19cb207b80 20 client.16986 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f19cb207b80 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c599682db0 0x55c5996851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f19cb207b80 4 client.16986 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b55298cb", "release": true, "await": true}] 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f19cb207b80 1 -- 172.21.15.39:0/3295687409 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b55298cb", "release": true, "await": true}) -- 0x55c5995273e0 con 0x55c599682db0 2024-05-04T03:26:10.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.446+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c599682db0 0x55c5996851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.447+0000 7f199dffb700 10 client.16986 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:10.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.447+0000 7f19b9d8e700 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c599682db0 0x55c5996851d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cdc5d60fb7cf67a server_cookie=dcf4b194159dc590 in_seq=0 out_seq=0 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.452+0000 7f199dffb700 1 -- 172.21.15.39:0/3295687409 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c5995273e0 con 0x55c599682db0 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.452+0000 7f199dffb700 10 client.16986 handle_command_reply: tid=0 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.452+0000 7f19cb207b80 1 client.16986 shutdown 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 -- 172.21.15.39:0/3295687409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 msgr2=0x55c59960dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c59960dea0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f19a8009990 tx=0x7f19a8024540 comp rx=0 tx=0).stop 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 -- 172.21.15.39:0/3295687409 shutdown_connections 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c599682db0 0x55c5996851d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c5995fc5d0 0x55c59960dea0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c5996078e0 0x55c59967b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 --2- 172.21.15.39:0/3295687409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5995fcf20 0x55c59960e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 -- 172.21.15.39:0/3295687409 >> 172.21.15.39:0/3295687409 conn(0x55c59947ff40 msgr2=0x55c599604f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f199dffb700 0 client.16986 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 -- 172.21.15.39:0/3295687409 shutdown_connections 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 1 -- 172.21.15.39:0/3295687409 wait complete. 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.453+0000 7f19cb207b80 20 client.16986 trim_cache size 0 max 16384 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 40, 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:26:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "b55298cb": { 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 40, 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 85.3, 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:26:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:26:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:10.459 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:26:10.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19a00b2880 msgr2=0x7f19a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19a00b2880 0x7f19a00b4d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f19a8000c00 tx=0x7f19a8005940 comp rx=0 tx=0).stop 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b4108f20 msgr2=0x7f19b419b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b4108f20 0x7f19b419b2c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c8f0 tx=0x7f19b000ccb0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4236740440 shutdown_connections 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f19a00b2880 0x7f19a00b4d90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19b4109870 0x7f19b419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b41045f0 0x7f19b419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 --2- 172.21.15.39:0/4236740440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19b4108f20 0x7f19b419b2c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4236740440 >> 172.21.15.39:0/4236740440 conn(0x7f19b40ffec0 msgr2=0x7f19b41022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4236740440 shutdown_connections 2024-05-04T03:26:10.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:10.454+0000 7f19bc7f3700 1 -- 172.21.15.39:0/4236740440 wait complete. 2024-05-04T03:26:10.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:10 smithi039 ceph-mon[29715]: pgmap v847: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 193 KiB/s wr, 35 op/s 2024-05-04T03:26:10.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:10 smithi066 ceph-mon[33264]: pgmap v847: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 193 KiB/s wr, 35 op/s 2024-05-04T03:26:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:10 smithi167 ceph-mon[33614]: pgmap v847: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 193 KiB/s wr, 35 op/s 2024-05-04T03:26:11.252 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b55298cb', seconds elapsed: 40.1 2024-05-04T03:26:11.252 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-05-04T03:26:11.253 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:26:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 740.0 s, 265.1 tps, lat 2472.261 ms stddev 8385.017 2024-05-04T03:26:12.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:12 smithi039 ceph-mon[29715]: pgmap v848: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 174 KiB/s wr, 34 op/s 2024-05-04T03:26:12.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:12 smithi066 ceph-mon[33264]: pgmap v848: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 174 KiB/s wr, 34 op/s 2024-05-04T03:26:12.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:12 smithi167 ceph-mon[33614]: pgmap v848: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 174 KiB/s wr, 34 op/s 2024-05-04T03:26:13.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 -- 172.21.15.39:0/3771695024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 msgr2=0x7f20381107e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 --2- 172.21.15.39:0/3771695024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 0x7f20381107e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f202c009990 tx=0x7f202c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 -- 172.21.15.39:0/3771695024 shutdown_connections 2024-05-04T03:26:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 --2- 172.21.15.39:0/3771695024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038110d20 0x7f2038113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 --2- 172.21.15.39:0/3771695024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2038102660 0x7f2038102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 --2- 172.21.15.39:0/3771695024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 0x7f20381107e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 -- 172.21.15.39:0/3771695024 >> 172.21.15.39:0/3771695024 conn(0x7f20380fe0d0 msgr2=0x7f20381004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 -- 172.21.15.39:0/3771695024 shutdown_connections 2024-05-04T03:26:13.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.789+0000 7f203c944700 1 -- 172.21.15.39:0/3771695024 wait complete. 2024-05-04T03:26:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.790+0000 7f203c944700 1 Processor -- start 2024-05-04T03:26:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.790+0000 7f203c944700 1 -- start start 2024-05-04T03:26:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203c944700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038102660 0x7f203819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:13.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203c944700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 0x7f203819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038102660 0x7f203819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038102660 0x7f203819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47692/0 (socket says 172.21.15.39:47692) 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f2035d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 0x7f203819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f2035d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 0x7f203819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:33800/0 (socket says 172.21.15.39:33800) 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203c944700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2038110d20 0x7f2038198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203c944700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2038108600 con 0x7f2038102660 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203c944700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2038108480 con 0x7f203810e3c0 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203c944700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2038108780 con 0x7f2038110d20 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.791+0000 7f203659c700 1 -- 172.21.15.39:0/3358895529 learned_addr learned my addr 172.21.15.39:0/3358895529 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:13.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.792+0000 7f2036d9d700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2038110d20 0x7f2038198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.792+0000 7f203659c700 1 -- 172.21.15.39:0/3358895529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2038110d20 msgr2=0x7f2038198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.792+0000 7f203659c700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2038110d20 0x7f2038198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.792+0000 7f203659c700 1 -- 172.21.15.39:0/3358895529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 msgr2=0x7f203819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:13.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.792+0000 7f203659c700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 0x7f203819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.792+0000 7f203659c700 1 -- 172.21.15.39:0/3358895529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2038199360 con 0x7f2038102660 2024-05-04T03:26:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.792+0000 7f2035d9b700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 0x7f203819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:26:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.792+0000 7f203659c700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038102660 0x7f203819e800 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2028015f90 tx=0x7f2028013330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.793+0000 7f20277fe700 1 -- 172.21.15.39:0/3358895529 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028013db0 con 0x7f2038102660 2024-05-04T03:26:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.793+0000 7f20277fe700 1 -- 172.21.15.39:0/3358895529 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2028004cc0 con 0x7f2038102660 2024-05-04T03:26:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.793+0000 7f20277fe700 1 -- 172.21.15.39:0/3358895529 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028013db0 con 0x7f2038102660 2024-05-04T03:26:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.793+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2038199650 con 0x7f2038102660 2024-05-04T03:26:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.793+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20381a5600 con 0x7f2038102660 2024-05-04T03:26:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.795+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2038103850 con 0x7f2038102660 2024-05-04T03:26:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.798+0000 7f20277fe700 1 -- 172.21.15.39:0/3358895529 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2028024060 con 0x7f2038102660 2024-05-04T03:26:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.799+0000 7f20277fe700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20200b2cf0 0x7f20200b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.799+0000 7f20277fe700 1 -- 172.21.15.39:0/3358895529 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f2028022070 con 0x7f2038102660 2024-05-04T03:26:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.799+0000 7f2035d9b700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20200b2cf0 0x7f20200b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.800+0000 7f2035d9b700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20200b2cf0 0x7f20200b5200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f202c00b650 tx=0x7f202c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:13.803+0000 7f20277fe700 1 -- 172.21.15.39:0/3358895529 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f202802b080 con 0x7f2038102660 2024-05-04T03:26:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.281+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2038199f30 con 0x7f2038102660 2024-05-04T03:26:14.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:26:14.287 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:26:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.283+0000 7f20277fe700 1 -- 172.21.15.39:0/3358895529 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f202802b080 con 0x7f2038102660 2024-05-04T03:26:14.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.287+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20200b2cf0 msgr2=0x7f20200b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:14.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.287+0000 7f203c944700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20200b2cf0 0x7f20200b5200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f202c00b650 tx=0x7f202c005fb0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.287+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038102660 msgr2=0x7f203819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:14.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.287+0000 7f203c944700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038102660 0x7f203819e800 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2028015f90 tx=0x7f2028013330 comp rx=0 tx=0).stop 2024-05-04T03:26:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.288+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 shutdown_connections 2024-05-04T03:26:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.288+0000 7f203c944700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20200b2cf0 0x7f20200b5200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.288+0000 7f203c944700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038102660 0x7f203819e800 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.288+0000 7f203c944700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2038110d20 0x7f2038198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.288+0000 7f203c944700 1 --2- 172.21.15.39:0/3358895529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f203810e3c0 0x7f203819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.288+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 >> 172.21.15.39:0/3358895529 conn(0x7f20380fe0d0 msgr2=0x7f2038105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:14.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.288+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 shutdown_connections 2024-05-04T03:26:14.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:14.288+0000 7f203c944700 1 -- 172.21.15.39:0/3358895529 wait complete. 2024-05-04T03:26:14.292 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:26:14.641 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:14 smithi039 ceph-mon[29715]: pgmap v849: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 825 KiB/s wr, 106 op/s 2024-05-04T03:26:14.641 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:14 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3358895529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:14.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:14 smithi066 ceph-mon[33264]: pgmap v849: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 825 KiB/s wr, 106 op/s 2024-05-04T03:26:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:14 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3358895529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:14.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:14 smithi167 ceph-mon[33614]: pgmap v849: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 825 KiB/s wr, 106 op/s 2024-05-04T03:26:14.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:14 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3358895529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:15.049 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 85.35 --await-for 42.0' 2024-05-04T03:26:15.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 85.35 --await-for 42.0 2024-05-04T03:26:15.468 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:16.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 745.0 s, 1218.6 tps, lat 85.268 ms stddev 44.402 2024-05-04T03:26:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:16 smithi066 ceph-mon[33264]: pgmap v850: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.1 MiB/s wr, 237 op/s 2024-05-04T03:26:16.896 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:16 smithi039 ceph-mon[29715]: pgmap v850: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.1 MiB/s wr, 237 op/s 2024-05-04T03:26:16.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:16 smithi167 ceph-mon[33614]: pgmap v850: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.1 MiB/s wr, 237 op/s 2024-05-04T03:26:17.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.453+0000 7fb3c1fda700 1 -- 172.21.15.39:0/3385020456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3bc1089f0 msgr2=0x7fb3bc108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.453+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/3385020456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3bc1089f0 0x7fb3bc108e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009990 tx=0x7fb3b0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.453+0000 7fb3c1fda700 1 -- 172.21.15.39:0/3385020456 shutdown_connections 2024-05-04T03:26:17.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.453+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/3385020456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3bc103310 0x7fb3bc10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.453+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/3385020456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3bc1089f0 0x7fb3bc108e00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.453+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/3385020456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3bc1029c0 0x7fb3bc102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.453+0000 7fb3c1fda700 1 -- 172.21.15.39:0/3385020456 >> 172.21.15.39:0/3385020456 conn(0x7fb3bc0fe6f0 msgr2=0x7fb3bc100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.453+0000 7fb3c1fda700 1 -- 172.21.15.39:0/3385020456 shutdown_connections 2024-05-04T03:26:17.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.454+0000 7fb3c1fda700 1 -- 172.21.15.39:0/3385020456 wait complete. 2024-05-04T03:26:17.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.455+0000 7fb3c1fda700 1 Processor -- start 2024-05-04T03:26:17.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.455+0000 7fb3c1fda700 1 -- start start 2024-05-04T03:26:17.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.455+0000 7fb3c1fda700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3bc1029c0 0x7fb3bc1063d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.455+0000 7fb3c1fda700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3bc103310 0x7fb3bc104830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.455+0000 7fb3c1fda700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3bc1089f0 0x7fb3bc104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.455+0000 7fb3c1fda700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1133f0 con 0x7fb3bc1029c0 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.455+0000 7fb3c1fda700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc113270 con 0x7fb3bc1089f0 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.455+0000 7fb3c1fda700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc113570 con 0x7fb3bc103310 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3baffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3bc103310 0x7fb3bc104830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3bc1029c0 0x7fb3bc1063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3baffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3bc103310 0x7fb3bc104830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:51874/0 (socket says 172.21.15.39:51874) 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3baffd700 1 -- 172.21.15.39:0/2202979628 learned_addr learned my addr 172.21.15.39:0/2202979628 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3bbfff700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3bc1089f0 0x7fb3bc104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3baffd700 1 -- 172.21.15.39:0/2202979628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3bc1089f0 msgr2=0x7fb3bc104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3baffd700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3bc1089f0 0x7fb3bc104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3baffd700 1 -- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3bc1029c0 msgr2=0x7fb3bc1063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3baffd700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3bc1029c0 0x7fb3bc1063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.456+0000 7fb3baffd700 1 -- 172.21.15.39:0/2202979628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc105640 con 0x7fb3bc103310 2024-05-04T03:26:17.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.457+0000 7fb3baffd700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3bc103310 0x7fb3bc104830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00b550 tx=0x7fb3ac00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.457+0000 7fb3b8ff9700 1 -- 172.21.15.39:0/2202979628 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac015020 con 0x7fb3bc103310 2024-05-04T03:26:17.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.457+0000 7fb3b8ff9700 1 -- 172.21.15.39:0/2202979628 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3ac011cc0 con 0x7fb3bc103310 2024-05-04T03:26:17.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.457+0000 7fb3c1fda700 1 -- 172.21.15.39:0/2202979628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc1ada60 con 0x7fb3bc103310 2024-05-04T03:26:17.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.457+0000 7fb3c1fda700 1 -- 172.21.15.39:0/2202979628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc1adf20 con 0x7fb3bc103310 2024-05-04T03:26:17.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.457+0000 7fb3b8ff9700 1 -- 172.21.15.39:0/2202979628 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac015020 con 0x7fb3bc103310 2024-05-04T03:26:17.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.461+0000 7fb3b8ff9700 1 -- 172.21.15.39:0/2202979628 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb3ac018020 con 0x7fb3bc103310 2024-05-04T03:26:17.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.463+0000 7fb3b8ff9700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb3a40b2cc0 0x7fb3a40b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.463+0000 7fb3b8ff9700 1 -- 172.21.15.39:0/2202979628 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb3ac011630 con 0x7fb3bc103310 2024-05-04T03:26:17.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.464+0000 7fb3bb7fe700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb3a40b2cc0 0x7fb3a40b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.464+0000 7fb3d09eeb80 1 Processor -- start 2024-05-04T03:26:17.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.464+0000 7fb3d09eeb80 1 -- start start 2024-05-04T03:26:17.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.464+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdbfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdbffb0 0x564d2fdca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3bb7fe700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb3a40b2cc0 0x7fb3a40b51d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000b5c0 tx=0x7fb3b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdca970 0x564d2fdccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3d09eeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564d2fce29f0 con 0x564d2fdbf660 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3d09eeb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564d2fc81950 con 0x564d2fdca970 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3d09eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564d2fcde9b0 con 0x564d2fdbffb0 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdbfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdbfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38602/0 (socket says 172.21.15.39:38602) 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3bb7fe700 1 -- 172.21.15.39:0/1400557058 learned_addr learned my addr 172.21.15.39:0/1400557058 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.465+0000 7fb3bbfff700 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdca970 0x564d2fdccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.466+0000 7fb3baffd700 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdbffb0 0x564d2fdca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.466+0000 7fb3bbfff700 1 -- 172.21.15.39:0/1400557058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdbffb0 msgr2=0x564d2fdca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.466+0000 7fb3bbfff700 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdbffb0 0x564d2fdca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.466+0000 7fb3bbfff700 1 -- 172.21.15.39:0/1400557058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 msgr2=0x564d2fdbfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.466+0000 7fb3bbfff700 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdbfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.466+0000 7fb3bbfff700 1 -- 172.21.15.39:0/1400557058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2fc2fa60 con 0x564d2fdca970 2024-05-04T03:26:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.466+0000 7fb3baffd700 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdbffb0 0x564d2fdca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.467+0000 7fb3bbfff700 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdca970 0x564d2fdccd90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009f90 tx=0x7fb3b4009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34179ff025acc41b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.467+0000 7fb3a1ffb700 1 -- 172.21.15.39:0/1400557058 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4014620 con 0x564d2fdca970 2024-05-04T03:26:17.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.467+0000 7fb3a1ffb700 1 -- 172.21.15.39:0/1400557058 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3b4019070 con 0x564d2fdca970 2024-05-04T03:26:17.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.467+0000 7fb3a1ffb700 1 -- 172.21.15.39:0/1400557058 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4014620 con 0x564d2fdca970 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.468+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/1400557058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdca970 msgr2=0x564d2fdccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.468+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdca970 0x564d2fdccd90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009f90 tx=0x7fb3b4009630 comp rx=0 tx=0).stop 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.468+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/1400557058 shutdown_connections 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.469+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdbfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.469+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdca970 0x564d2fdccd90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.469+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/1400557058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdbffb0 0x564d2fdca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.469+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/1400557058 >> 172.21.15.39:0/1400557058 conn(0x564d2fc42fd0 msgr2=0x564d2fdbb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.469+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/1400557058 shutdown_connections 2024-05-04T03:26:17.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.469+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/1400557058 wait complete. 2024-05-04T03:26:17.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.470+0000 7fb3d09eeb80 1 -- start start 2024-05-04T03:26:17.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.470+0000 7fb3d09eeb80 1 Processor -- start 2024-05-04T03:26:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.470+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbf660 0x564d2fdc7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbffb0 0x564d2fdc7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fdc82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3d09eeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564d2fdcd700 con 0x564d2fdbffb0 2024-05-04T03:26:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbf660 0x564d2fdc7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbf660 0x564d2fdc7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:57880/0 (socket says 172.21.15.39:57880) 2024-05-04T03:26:17.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3bb7fe700 1 -- 172.21.15.39:0/611155021 learned_addr learned my addr 172.21.15.39:0/611155021 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564d2fc48dc0 con 0x564d2fdbf660 2024-05-04T03:26:17.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564d2fdcd880 con 0x564d2fdca970 2024-05-04T03:26:17.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3baffd700 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbffb0 0x564d2fdc7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.471+0000 7fb3d09eeb80 10 client.0 _ll_register_callbacks cb 0x564d2fc7aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.472+0000 7fb3bb7fe700 1 -- 172.21.15.39:0/611155021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 msgr2=0x564d2fdc82c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.472+0000 7fb3bb7fe700 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fdc82c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.472+0000 7fb3bb7fe700 1 -- 172.21.15.39:0/611155021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbffb0 msgr2=0x564d2fdc7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.472+0000 7fb3bb7fe700 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbffb0 0x564d2fdc7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.472+0000 7fb3bb7fe700 1 -- 172.21.15.39:0/611155021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2fc7e820 con 0x564d2fdbf660 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.472+0000 7fb3bb7fe700 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbf660 0x564d2fdc7840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0000c00 tx=0x7fb3b001a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec4c2e7f49f6d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.472+0000 7fb39affd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.472+0000 7fb39affd700 1 -- 172.21.15.39:0/611155021 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0034060 con 0x564d2fdbf660 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.473+0000 7fb39affd700 1 -- 172.21.15.39:0/611155021 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb3b000f460 con 0x564d2fdbf660 2024-05-04T03:26:17.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.473+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b4019070 con 0x564d2fdbf660 2024-05-04T03:26:17.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.473+0000 7fb39affd700 1 -- 172.21.15.39:0/611155021 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb3b000f660 con 0x564d2fdbf660 2024-05-04T03:26:17.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.473+0000 7fb3d09eeb80 10 client.26570 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:17.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.473+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d2fdd81d0 con 0x564d2fdbf660 2024-05-04T03:26:17.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.474+0000 7fb39affd700 1 -- 172.21.15.39:0/611155021 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3b4019070 con 0x564d2fdbf660 2024-05-04T03:26:17.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.474+0000 7fb39affd700 1 -- 172.21.15.39:0/611155021 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b0022070 con 0x564d2fdbf660 2024-05-04T03:26:17.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.474+0000 7fb3d09eeb80 10 client.26570 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:17.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.474+0000 7fb3d09eeb80 10 client.26570 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:26:17.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.474+0000 7fb3d09eeb80 20 client.26570 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:17.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.474+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d2fdda7b0 0x564d2fddcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.474+0000 7fb3d09eeb80 4 client.26570 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:26:17.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.474+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d2fcf03d0 con 0x564d2fdda7b0 2024-05-04T03:26:17.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.475+0000 7fb3baffd700 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d2fdda7b0 0x564d2fddcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.475+0000 7fb3baffd700 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d2fdda7b0 0x564d2fddcc90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb11c17209296d7 server_cookie=9e6cc28272fb6f45 in_seq=0 out_seq=0 2024-05-04T03:26:17.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.475+0000 7fb39affd700 10 client.26570 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:17.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.476+0000 7fb39affd700 1 -- 172.21.15.39:0/611155021 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d2fcf03d0 con 0x564d2fdda7b0 2024-05-04T03:26:17.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.476+0000 7fb39affd700 10 client.26570 handle_command_reply: tid=0 2024-05-04T03:26:17.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.476+0000 7fb3d09eeb80 1 client.26570 shutdown 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbf660 msgr2=0x564d2fdc7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbf660 0x564d2fdc7840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0000c00 tx=0x7fb3b001a050 comp rx=0 tx=0).stop 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 shutdown_connections 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d2fdda7b0 0x564d2fddcc90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fdc82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbf660 0x564d2fdc7840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/611155021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbffb0 0x564d2fdc7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 >> 172.21.15.39:0/611155021 conn(0x564d2fc42fd0 msgr2=0x564d2fdc1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb39affd700 0 client.26570 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 shutdown_connections 2024-05-04T03:26:17.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.477+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/611155021 wait complete. 2024-05-04T03:26:17.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.478+0000 7fb3d09eeb80 20 client.26570 trim_cache size 0 max 16384 2024-05-04T03:26:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.512+0000 7fb3d09eeb80 1 Processor -- start 2024-05-04T03:26:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.513+0000 7fb3d09eeb80 1 -- start start 2024-05-04T03:26:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.513+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdc8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fde2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fde2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3d09eeb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800d320 con 0x564d2fdbf660 2024-05-04T03:26:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3d09eeb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800ee60 con 0x564d2fdbffb0 2024-05-04T03:26:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3d09eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800d9f0 con 0x564d2fdca970 2024-05-04T03:26:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdc8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3baffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fde2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdc8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38622/0 (socket says 172.21.15.39:38622) 2024-05-04T03:26:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3bb7fe700 1 -- 172.21.15.39:0/3717182771 learned_addr learned my addr 172.21.15.39:0/3717182771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3baffd700 1 -- 172.21.15.39:0/3717182771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 msgr2=0x564d2fde2990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:26:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3baffd700 1 --2- 172.21.15.39:0/3717182771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fde2990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3baffd700 1 -- 172.21.15.39:0/3717182771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 msgr2=0x564d2fdc8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3baffd700 1 --2- 172.21.15.39:0/3717182771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdc8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.514+0000 7fb3baffd700 1 -- 172.21.15.39:0/3717182771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2fdca330 con 0x564d2fdbffb0 2024-05-04T03:26:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.515+0000 7fb3baffd700 1 --2- 172.21.15.39:0/3717182771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fde2450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ded40 tx=0x7fb3ac090040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df13a0fa07102d39 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.515+0000 7fb3a17fa700 1 -- 172.21.15.39:0/3717182771 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac015040 con 0x564d2fdbffb0 2024-05-04T03:26:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.515+0000 7fb3a17fa700 1 -- 172.21.15.39:0/3717182771 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3ac0a8f10 con 0x564d2fdbffb0 2024-05-04T03:26:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.515+0000 7fb3a17fa700 1 -- 172.21.15.39:0/3717182771 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0cb0b0 con 0x564d2fdbffb0 2024-05-04T03:26:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.516+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/3717182771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 msgr2=0x564d2fde2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.516+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/3717182771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fde2450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ded40 tx=0x7fb3ac090040 comp rx=0 tx=0).stop 2024-05-04T03:26:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.516+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/3717182771 shutdown_connections 2024-05-04T03:26:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.517+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/3717182771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdc8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.517+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/3717182771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fde2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.517+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/3717182771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fde2450 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.517+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/3717182771 >> 172.21.15.39:0/3717182771 conn(0x564d2fc42fd0 msgr2=0x564d2fdc95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.517+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/3717182771 shutdown_connections 2024-05-04T03:26:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.517+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/3717182771 wait complete. 2024-05-04T03:26:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.517+0000 7fb3d09eeb80 1 -- start start 2024-05-04T03:26:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.518+0000 7fb3d09eeb80 1 Processor -- start 2024-05-04T03:26:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdd3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3d09eeb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fdd13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdd3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3bb7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdd3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38636/0 (socket says 172.21.15.39:38636) 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3bb7fe700 1 -- 172.21.15.39:0/584153249 learned_addr learned my addr 172.21.15.39:0/584153249 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fdd1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3baffd700 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fdd13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800e790 con 0x564d2fdbf660 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800e0c0 con 0x564d2fdbffb0 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a800b2c0 con 0x564d2fdca970 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.519+0000 7fb3d09eeb80 10 client.0 _ll_register_callbacks cb 0x564d2fc7aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3bbfff700 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fdd1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3baffd700 1 -- 172.21.15.39:0/584153249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 msgr2=0x564d2fdd1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3baffd700 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fdd1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3baffd700 1 -- 172.21.15.39:0/584153249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 msgr2=0x564d2fdd3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3baffd700 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdd3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3baffd700 1 -- 172.21.15.39:0/584153249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2fdd1fe0 con 0x564d2fdbffb0 2024-05-04T03:26:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3bbfff700 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fdd1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3baffd700 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fdd13f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00bbb0 tx=0x7fb3ac0cb270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=372093b49b00cb85 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb39affd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:26:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb39affd700 1 -- 172.21.15.39:0/584153249 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac015070 con 0x564d2fdbffb0 2024-05-04T03:26:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb39affd700 1 -- 172.21.15.39:0/584153249 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb3ac0a8c80 con 0x564d2fdbffb0 2024-05-04T03:26:17.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.520+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3ac0a8f10 con 0x564d2fdbffb0 2024-05-04T03:26:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.521+0000 7fb39affd700 1 -- 172.21.15.39:0/584153249 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb3ac009280 con 0x564d2fdbffb0 2024-05-04T03:26:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.521+0000 7fb3d09eeb80 10 client.26576 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.521+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d2fe45e20 con 0x564d2fdbffb0 2024-05-04T03:26:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.521+0000 7fb39affd700 1 -- 172.21.15.39:0/584153249 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3ac0b1670 con 0x564d2fdbffb0 2024-05-04T03:26:17.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.522+0000 7fb39affd700 1 -- 172.21.15.39:0/584153249 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb3ac0a9bd0 con 0x564d2fdbffb0 2024-05-04T03:26:17.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.522+0000 7fb3d09eeb80 10 client.26576 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:17.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.522+0000 7fb3d09eeb80 10 client.26576 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:26:17.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.522+0000 7fb3d09eeb80 20 client.26576 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:17.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.522+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d2fe464c0 0x564d2fe488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.522+0000 7fb3d09eeb80 4 client.26576 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.35, "await_for": 42.0}] 2024-05-04T03:26:17.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.522+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.35, "await_for": 42.0}) -- 0x7fb3ac0a8f10 con 0x564d2fe464c0 2024-05-04T03:26:17.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.523+0000 7fb3bb7fe700 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d2fe464c0 0x564d2fe488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.524+0000 7fb3bb7fe700 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d2fe464c0 0x564d2fe488e0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27f68d3cb7c9429 server_cookie=6261d2e4ff33d25b in_seq=0 out_seq=0 2024-05-04T03:26:17.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.524+0000 7fb39affd700 10 client.26576 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.759+0000 7f87ba4d1640 1 -- 172.21.15.39:0/959929380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b41030b0 msgr2=0x7f87b41034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.759+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/959929380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b41030b0 0x7f87b41034c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009900 tx=0x7f87a402a410 comp rx=0 tx=0).stop 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.759+0000 7f87ba4d1640 1 -- 172.21.15.39:0/959929380 shutdown_connections 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.760+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/959929380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b41045d0 0x7f87b410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.760+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/959929380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b4103a90 0x7f87b4103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.760+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/959929380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b41030b0 0x7f87b41034c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.760+0000 7f87ba4d1640 1 -- 172.21.15.39:0/959929380 >> 172.21.15.39:0/959929380 conn(0x7f87b40fd4b0 msgr2=0x7f87b40ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.760+0000 7f87ba4d1640 1 -- 172.21.15.39:0/959929380 shutdown_connections 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.760+0000 7f87ba4d1640 1 -- 172.21.15.39:0/959929380 wait complete. 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.760+0000 7f87ba4d1640 1 Processor -- start 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.760+0000 7f87ba4d1640 1 -- start start 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87ba4d1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b41030b0 0x7f87b419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87ba4d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b4103a90 0x7f87b419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87ba4d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b41045d0 0x7f87b41a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87ba4d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410e4f0 con 0x7f87b4103a90 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87ba4d1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410e370 con 0x7f87b41030b0 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87ba4d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410e670 con 0x7f87b41045d0 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b4103a90 0x7f87b419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b4103a90 0x7f87b419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38654/0 (socket says 172.21.15.39:38654) 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 -- 172.21.15.39:0/2036174942 learned_addr learned my addr 172.21.15.39:0/2036174942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b41045d0 0x7f87b41a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b3fff640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b41030b0 0x7f87b419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 -- 172.21.15.39:0/2036174942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b41045d0 msgr2=0x7f87b41a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b41045d0 0x7f87b41a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 -- 172.21.15.39:0/2036174942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b41030b0 msgr2=0x7f87b419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b41030b0 0x7f87b419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 -- 172.21.15.39:0/2036174942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b41a1e10 con 0x7f87b4103a90 2024-05-04T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b41045d0 0x7f87b41a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b37fe640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b4103a90 0x7f87b419a810 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f879c00d950 tx=0x7f879c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.761+0000 7f87b17fa640 1 -- 172.21.15.39:0/2036174942 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c017030 con 0x7f87b4103a90 2024-05-04T03:26:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.762+0000 7f87ba4d1640 1 -- 172.21.15.39:0/2036174942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b41a2100 con 0x7f87b4103a90 2024-05-04T03:26:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.762+0000 7f87b17fa640 1 -- 172.21.15.39:0/2036174942 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f879c004020 con 0x7f87b4103a90 2024-05-04T03:26:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.762+0000 7f87b17fa640 1 -- 172.21.15.39:0/2036174942 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c0055d0 con 0x7f87b4103a90 2024-05-04T03:26:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.762+0000 7f87ba4d1640 1 -- 172.21.15.39:0/2036174942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b41a2650 con 0x7f87b4103a90 2024-05-04T03:26:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.764+0000 7f87c1382740 1 Processor -- start 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.764+0000 7f87c1382740 1 -- start start 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.764+0000 7f87b17fa640 1 -- 172.21.15.39:0/2036174942 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f879c009a30 con 0x7f87b4103a90 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87c1382740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e07a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87c1382740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e07fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87c1382740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e0eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87c1382740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564835d6c5d0 con 0x564835a454d0 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87c1382740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564835d25890 con 0x564835a3d2f0 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87c1382740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564835d68590 con 0x564835a661b0 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b8a6e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e07a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b8a6e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e07a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:51950/0 (socket says 172.21.15.39:51950) 2024-05-04T03:26:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e0eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b8a6e640 1 -- 172.21.15.39:0/847509531 learned_addr learned my addr 172.21.15.39:0/847509531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b3fff640 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e07fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b8a6e640 1 -- 172.21.15.39:0/847509531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 msgr2=0x564835e07fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e07fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b8a6e640 1 -- 172.21.15.39:0/847509531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 msgr2=0x564835e0eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e0eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b8a6e640 1 -- 172.21.15.39:0/847509531 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564835a7e210 con 0x564835a661b0 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b3fff640 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e07fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.765+0000 7f87b17fa640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f87a00b27e0 0x7f87a00b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.766+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e07a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f87ac011790 tx=0x7f87ac011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891df2d1f601cc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.766+0000 7f87b3fff640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f87a00b27e0 0x7f87a00b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.766+0000 7f87b17fa640 1 -- 172.21.15.39:0/2036174942 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f879c0e1c10 con 0x7f87b4103a90 2024-05-04T03:26:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.766+0000 7f87aa7fc640 1 -- 172.21.15.39:0/847509531 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac003c30 con 0x564835a661b0 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.766+0000 7f87aa7fc640 1 -- 172.21.15.39:0/847509531 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87ac00fe30 con 0x564835a661b0 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.766+0000 7f87aa7fc640 1 -- 172.21.15.39:0/847509531 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac0047e0 con 0x564835a661b0 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87b3fff640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f87a00b27e0 0x7f87a00b4cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87a4005760 tx=0x7f87a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 -- 172.21.15.39:0/847509531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 msgr2=0x564835e07a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e07a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f87ac011790 tx=0x7f87ac011c50 comp rx=0 tx=0).stop 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 -- 172.21.15.39:0/847509531 shutdown_connections 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e0eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e07fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 --2- 172.21.15.39:0/847509531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e07a60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 -- 172.21.15.39:0/847509531 >> 172.21.15.39:0/847509531 conn(0x564835a5a300 msgr2=0x564835e033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 -- 172.21.15.39:0/847509531 shutdown_connections 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.767+0000 7f87c1382740 1 -- 172.21.15.39:0/847509531 wait complete. 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 1 -- start start 2024-05-04T03:26:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 1 Processor -- start 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e11700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e11c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e18ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564835e0f930 con 0x564835a454d0 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x564835d29370 con 0x564835a3d2f0 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564835e0fab0 con 0x564835a661b0 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87c1382740 10 client.0 _ll_register_callbacks cb 0x564835a7da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e11c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87b8a6e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e18ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87b8a6e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e18ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:51952/0 (socket says 172.21.15.39:51952) 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e11c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38678/0 (socket says 172.21.15.39:38678) 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87b8a6e640 1 -- 172.21.15.39:0/1553502253 learned_addr learned my addr 172.21.15.39:0/1553502253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.768+0000 7f87b3fff640 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e11700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87b8a6e640 1 -- 172.21.15.39:0/1553502253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 msgr2=0x564835e11700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e11700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87b8a6e640 1 -- 172.21.15.39:0/1553502253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 msgr2=0x564835e11c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e11c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87b8a6e640 1 -- 172.21.15.39:0/1553502253 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564835b4a620 con 0x564835a661b0 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e18ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87ac004560 tx=0x7f87ac004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=767059b93a659ddb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87937fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87937fe640 1 -- 172.21.15.39:0/1553502253 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac028020 con 0x564835a661b0 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87937fe640 1 -- 172.21.15.39:0/1553502253 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f87ac011040 con 0x564835a661b0 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87c1382740 1 -- 172.21.15.39:0/1553502253 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ac00fe30 con 0x564835a661b0 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87937fe640 1 -- 172.21.15.39:0/1553502253 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f87ac01f040 con 0x564835a661b0 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87c1382740 10 client.26608 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.769+0000 7f87c1382740 1 -- 172.21.15.39:0/1553502253 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564835e1bad0 con 0x564835a661b0 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87937fe640 1 -- 172.21.15.39:0/1553502253 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87ac00fe30 con 0x564835a661b0 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87937fe640 1 -- 172.21.15.39:0/1553502253 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f87ac018070 con 0x564835a661b0 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87c1382740 10 client.26608 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87c1382740 10 client.26608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87c1382740 20 client.26608 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87c1382740 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564835a24d80 0x564835e1efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87c1382740 4 client.26608 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87c1382740 1 -- 172.21.15.39:0/1553502253 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564835d7a1e0 con 0x564835a24d80 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.770+0000 7f87b3fff640 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564835a24d80 0x564835e1efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.771+0000 7f87937fe640 10 client.26608 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.771+0000 7f87b3fff640 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564835a24d80 0x564835e1efb0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0dada535cff3d5c server_cookie=41ab9071c6831f44 in_seq=0 out_seq=0 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.771+0000 7f87937fe640 1 -- 172.21.15.39:0/1553502253 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564835d7a1e0 con 0x564835a24d80 2024-05-04T03:26:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.771+0000 7f87937fe640 10 client.26608 handle_command_reply: tid=0 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.771+0000 7f87c1382740 1 client.26608 shutdown 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 -- 172.21.15.39:0/1553502253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 msgr2=0x564835e18ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e18ba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87ac004560 tx=0x7f87ac004590 comp rx=0 tx=0).stop 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 -- 172.21.15.39:0/1553502253 shutdown_connections 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564835a24d80 0x564835e1efb0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a661b0 0x564835e18ba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e11c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87937fe640 0 client.26608 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 --2- 172.21.15.39:0/1553502253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e11700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 -- 172.21.15.39:0/1553502253 >> 172.21.15.39:0/1553502253 conn(0x564835a5a300 msgr2=0x564835e023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 -- 172.21.15.39:0/1553502253 shutdown_connections 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 1 -- 172.21.15.39:0/1553502253 wait complete. 2024-05-04T03:26:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.772+0000 7f87c1382740 20 client.26608 trim_cache size 0 max 16384 2024-05-04T03:26:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87c1382740 1 Processor -- start 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87c1382740 1 -- start start 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87c1382740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 0x564835e11e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87c1382740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87c1382740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e19000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87c1382740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ab3e0 con 0x564835a454d0 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87c1382740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41acf20 con 0x564835a3d2f0 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87c1382740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41abab0 con 0x564835a24d80 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87b8a6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e19000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87b8a6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e19000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38690/0 (socket says 172.21.15.39:38690) 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87b8a6e640 1 -- 172.21.15.39:0/1879355481 learned_addr learned my addr 172.21.15.39:0/1879355481 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.787+0000 7f87b3fff640 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 0x564835e11e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87b37fe640 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87b8a6e640 1 -- 172.21.15.39:0/1879355481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 msgr2=0x564835e11e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 0x564835e11e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87b8a6e640 1 -- 172.21.15.39:0/1879355481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 msgr2=0x564835e18ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e18ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87b8a6e640 1 -- 172.21.15.39:0/1879355481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564835e12770 con 0x564835a454d0 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87b37fe640 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e18ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e19000 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f87ac01a900 tx=0x7f87ac00f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78871e327b8455f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87a9ffb640 1 -- 172.21.15.39:0/1879355481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac028040 con 0x564835a454d0 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87a9ffb640 1 -- 172.21.15.39:0/1879355481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87ac00fe30 con 0x564835a454d0 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.788+0000 7f87a9ffb640 1 -- 172.21.15.39:0/1879355481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac003a30 con 0x564835a454d0 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 -- 172.21.15.39:0/1879355481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 msgr2=0x564835e19000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e19000 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f87ac01a900 tx=0x7f87ac00f8c0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 -- 172.21.15.39:0/1879355481 shutdown_connections 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e19000 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e18ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 --2- 172.21.15.39:0/1879355481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 0x564835e11e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 -- 172.21.15.39:0/1879355481 >> 172.21.15.39:0/1879355481 conn(0x564835a5a300 msgr2=0x564835e11700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 -- 172.21.15.39:0/1879355481 shutdown_connections 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 -- 172.21.15.39:0/1879355481 wait complete. 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.789+0000 7f87c1382740 1 -- start start 2024-05-04T03:26:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87c1382740 1 Processor -- start 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87c1382740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 0x564835e16590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87c1382740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87c1382740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e17a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87c1382740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ac850 con 0x564835a454d0 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87c1382740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ac180 con 0x564835a3d2f0 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87c1382740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a93c0 con 0x564835a24d80 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87c1382740 10 client.0 _ll_register_callbacks cb 0x7f879c0e55c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b8a6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e17a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b8a6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e17a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38700/0 (socket says 172.21.15.39:38700) 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b8a6e640 1 -- 172.21.15.39:0/4213320049 learned_addr learned my addr 172.21.15.39:0/4213320049 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b3fff640 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 0x564835e16590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b8a6e640 1 -- 172.21.15.39:0/4213320049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 msgr2=0x564835e16590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 0x564835e16590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b8a6e640 1 -- 172.21.15.39:0/4213320049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 msgr2=0x564835e16ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e16ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b8a6e640 1 -- 172.21.15.39:0/4213320049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564835e17fc0 con 0x564835a454d0 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.790+0000 7f87b37fe640 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87b8a6e640 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e17a00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f87ac002230 tx=0x7f87ac01ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab8f92e40fceb22 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87937fe640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87937fe640 1 -- 172.21.15.39:0/4213320049 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac03d030 con 0x564835a454d0 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87937fe640 1 -- 172.21.15.39:0/4213320049 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f87ac002030 con 0x564835a454d0 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 1 -- 172.21.15.39:0/4213320049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ac00fe30 con 0x564835a454d0 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87937fe640 1 -- 172.21.15.39:0/4213320049 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f87ac028040 con 0x564835a454d0 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 10 client.17025 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 1 -- 172.21.15.39:0/4213320049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564835e19250 con 0x564835a454d0 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87937fe640 1 -- 172.21.15.39:0/4213320049 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87ac00fe30 con 0x564835a454d0 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87937fe640 1 -- 172.21.15.39:0/4213320049 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f87ac018070 con 0x564835a454d0 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 10 client.17025 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 20 client.17025 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564835a661b0 0x564835e89620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 4 client.17025 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.791+0000 7f87c1382740 1 -- 172.21.15.39:0/4213320049 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564835d74050 con 0x564835a661b0 2024-05-04T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.792+0000 7f87b3fff640 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564835a661b0 0x564835e89620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.792+0000 7f87937fe640 10 client.17025 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.792+0000 7f87b3fff640 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564835a661b0 0x564835e89620 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfbd3b78c224edd8 server_cookie=f3aa742da8bca9ba in_seq=0 out_seq=0 2024-05-04T03:26:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.799+0000 7f87937fe640 1 -- 172.21.15.39:0/4213320049 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564835d74050 con 0x564835a661b0 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.799+0000 7f87937fe640 10 client.17025 handle_command_reply: tid=0 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.799+0000 7f87c1382740 1 client.17025 shutdown 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 -- 172.21.15.39:0/4213320049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 msgr2=0x564835e17a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e17a00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f87ac002230 tx=0x7f87ac01ad90 comp rx=0 tx=0).stop 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 -- 172.21.15.39:0/4213320049 shutdown_connections 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564835a661b0 0x564835e89620 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564835a454d0 0x564835e17a00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564835a3d2f0 0x564835e16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87937fe640 0 client.17025 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 --2- 172.21.15.39:0/4213320049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564835a24d80 0x564835e16590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 -- 172.21.15.39:0/4213320049 >> 172.21.15.39:0/4213320049 conn(0x564835a5a300 msgr2=0x564835e02cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 -- 172.21.15.39:0/4213320049 shutdown_connections 2024-05-04T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 1 -- 172.21.15.39:0/4213320049 wait complete. 2024-05-04T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87c1382740 20 client.17025 trim_cache size 0 max 16384 2024-05-04T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:26:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:26:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 -- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f87a00b27e0 msgr2=0x7f87a00b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f87a00b27e0 0x7f87a00b4cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87a4005760 tx=0x7f87a4000bc0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 -- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b4103a90 msgr2=0x7f87b419a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b4103a90 0x7f87b419a810 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f879c00d950 tx=0x7f879c00de10 comp rx=0 tx=0).stop 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 -- 172.21.15.39:0/2036174942 shutdown_connections 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f87a00b27e0 0x7f87a00b4cf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b41045d0 0x7f87b41a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87b4103a90 0x7f87b419a810 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 --2- 172.21.15.39:0/2036174942 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87b41030b0 0x7f87b419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.800+0000 7f87ba4d1640 1 -- 172.21.15.39:0/2036174942 >> 172.21.15.39:0/2036174942 conn(0x7f87b40fd4b0 msgr2=0x7f87b4109fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.801+0000 7f87ba4d1640 1 -- 172.21.15.39:0/2036174942 shutdown_connections 2024-05-04T03:26:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:17.801+0000 7f87ba4d1640 1 -- 172.21.15.39:0/2036174942 wait complete. 2024-05-04T03:26:17.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.895+0000 7fb39affd700 1 -- 172.21.15.39:0/584153249 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7fb3ac0a8f10 con 0x564d2fe464c0 2024-05-04T03:26:17.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.895+0000 7fb39affd700 10 client.26576 handle_command_reply: tid=0 2024-05-04T03:26:17.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.895+0000 7fb3d09eeb80 1 client.26576 shutdown 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 msgr2=0x564d2fdd13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fdd13f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00bbb0 tx=0x7fb3ac0cb270 comp rx=0 tx=0).stop 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 shutdown_connections 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x564d2fe464c0 0x564d2fe488e0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2fdbf660 0x564d2fdd3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d2fdca970 0x564d2fdd1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb39affd700 0 client.26576 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 --2- 172.21.15.39:0/584153249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564d2fdbffb0 0x564d2fdd13f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 >> 172.21.15.39:0/584153249 conn(0x564d2fc42fd0 msgr2=0x564d2fdc70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 shutdown_connections 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 1 -- 172.21.15.39:0/584153249 wait complete. 2024-05-04T03:26:17.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.896+0000 7fb3d09eeb80 20 client.26576 trim_cache size 0 max 16384 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 42, 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "8ffce17a": { 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 42, 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:26:17.900 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 85.3, 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:17.901 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:17.902 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:17.902 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:17.902 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:17.902 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:26:17.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 -- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb3a40b2cc0 msgr2=0x7fb3a40b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb3a40b2cc0 0x7fb3a40b51d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000b5c0 tx=0x7fb3b0005fb0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 -- 172.21.15.39:0/2202979628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3bc103310 msgr2=0x7fb3bc104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3bc103310 0x7fb3bc104830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00b550 tx=0x7fb3ac00b860 comp rx=0 tx=0).stop 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 -- 172.21.15.39:0/2202979628 shutdown_connections 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb3a40b2cc0 0x7fb3a40b51d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3bc1029c0 0x7fb3bc1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3bc1089f0 0x7fb3bc104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 --2- 172.21.15.39:0/2202979628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3bc103310 0x7fb3bc104830 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 -- 172.21.15.39:0/2202979628 >> 172.21.15.39:0/2202979628 conn(0x7fb3bc0fe6f0 msgr2=0x7fb3bc106c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 -- 172.21.15.39:0/2202979628 shutdown_connections 2024-05-04T03:26:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:17.897+0000 7fb3c1fda700 1 -- 172.21.15.39:0/2202979628 wait complete. 2024-05-04T03:26:18.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:18 smithi039 ceph-mon[29715]: pgmap v851: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-05-04T03:26:18.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:18 smithi066 ceph-mon[33264]: pgmap v851: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-05-04T03:26:18.849 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8ffce17a', quiesce took 7.6 seconds. Will release after: 25.35 2024-05-04T03:26:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:18 smithi167 ceph-mon[33614]: pgmap v851: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.8 MiB/s wr, 431 op/s 2024-05-04T03:26:20.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:20 smithi066 ceph-mon[33264]: pgmap v852: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 613 op/s 2024-05-04T03:26:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:20 smithi167 ceph-mon[33614]: pgmap v852: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 613 op/s 2024-05-04T03:26:20.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:20 smithi039 ceph-mon[29715]: pgmap v852: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 613 op/s 2024-05-04T03:26:21.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 750.0 s, 203.2 tps, lat 80.869 ms stddev 31.447 2024-05-04T03:26:22.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:22 smithi066 ceph-mon[33264]: pgmap v853: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 690 op/s 2024-05-04T03:26:22.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:22 smithi167 ceph-mon[33614]: pgmap v853: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 690 op/s 2024-05-04T03:26:22.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:22 smithi039 ceph-mon[29715]: pgmap v853: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 690 op/s 2024-05-04T03:26:24.733 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:24 smithi167 ceph-mon[33614]: pgmap v854: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.3 MiB/s wr, 711 op/s 2024-05-04T03:26:24.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:24 smithi066 ceph-mon[33264]: pgmap v854: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.3 MiB/s wr, 711 op/s 2024-05-04T03:26:24.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:24 smithi039 ceph-mon[29715]: pgmap v854: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.3 MiB/s wr, 711 op/s 2024-05-04T03:26:26.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 755.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:26:26.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:26 smithi066 ceph-mon[33264]: pgmap v855: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 637 op/s 2024-05-04T03:26:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:26 smithi167 ceph-mon[33614]: pgmap v855: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 637 op/s 2024-05-04T03:26:26.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:26 smithi039 ceph-mon[29715]: pgmap v855: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 637 op/s 2024-05-04T03:26:28.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:28 smithi066 ceph-mon[33264]: pgmap v856: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-05-04T03:26:28.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:28 smithi167 ceph-mon[33614]: pgmap v856: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-05-04T03:26:28.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:28 smithi039 ceph-mon[29715]: pgmap v856: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-05-04T03:26:30.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:30 smithi066 ceph-mon[33264]: pgmap v857: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.6 MiB/s wr, 309 op/s 2024-05-04T03:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.913+0000 7f99f752d640 1 -- 172.21.15.39:0/2615413126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 msgr2=0x7f99f0107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.913+0000 7f99f752d640 1 --2- 172.21.15.39:0/2615413126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 0x7f99f0107c10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009ca0 tx=0x7f99e802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.913+0000 7f99f752d640 1 -- 172.21.15.39:0/2615413126 shutdown_connections 2024-05-04T03:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.913+0000 7f99f752d640 1 --2- 172.21.15.39:0/2615413126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 0x7f99f0107c10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.913+0000 7f99f752d640 1 --2- 172.21.15.39:0/2615413126 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f0104ea0 0x7f99f01072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.913+0000 7f99f752d640 1 --2- 172.21.15.39:0/2615413126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f0102540 0x7f99f0104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.913+0000 7f99f752d640 1 -- 172.21.15.39:0/2615413126 >> 172.21.15.39:0/2615413126 conn(0x7f99f00fdd50 msgr2=0x7f99f0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.914+0000 7f99f752d640 1 -- 172.21.15.39:0/2615413126 shutdown_connections 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.914+0000 7f99f752d640 1 -- 172.21.15.39:0/2615413126 wait complete. 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.914+0000 7f99f752d640 1 Processor -- start 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.914+0000 7f99f752d640 1 -- start start 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.914+0000 7f99f752d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f0102540 0x7f99f01a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.914+0000 7f99f752d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f0104ea0 0x7f99f01a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.914+0000 7f99f752d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 0x7f99f01a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f752d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0115c80 con 0x7f99f0107800 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f752d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0115b00 con 0x7f99f0104ea0 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f752d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0115e00 con 0x7f99f0102540 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 0x7f99f01a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 0x7f99f01a7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34266/0 (socket says 172.21.15.39:34266) 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 -- 172.21.15.39:0/1015183700 learned_addr learned my addr 172.21.15.39:0/1015183700 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f52c9640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f0102540 0x7f99f01a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f0104ea0 0x7f99f01a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 -- 172.21.15.39:0/1015183700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f0102540 msgr2=0x7f99f01a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f0102540 0x7f99f01a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 -- 172.21.15.39:0/1015183700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f0104ea0 msgr2=0x7f99f01a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f0104ea0 0x7f99f01a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 -- 172.21.15.39:0/1015183700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f01a8390 con 0x7f99f0107800 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f0104ea0 0x7f99f01a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f5aca640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 0x7f99f01a7d20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f99e802aeb0 tx=0x7f99e80039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99e27fc640 1 -- 172.21.15.39:0/1015183700 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8004040 con 0x7f99f0107800 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99e27fc640 1 -- 172.21.15.39:0/1015183700 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e8004670 con 0x7f99f0107800 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99e27fc640 1 -- 172.21.15.39:0/1015183700 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8030d50 con 0x7f99f0107800 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f752d640 1 -- 172.21.15.39:0/1015183700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f01a8620 con 0x7f99f0107800 2024-05-04T03:26:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.915+0000 7f99f752d640 1 -- 172.21.15.39:0/1015183700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f01a8bc0 con 0x7f99f0107800 2024-05-04T03:26:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.918+0000 7f99e27fc640 1 -- 172.21.15.39:0/1015183700 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f99e80305c0 con 0x7f99f0107800 2024-05-04T03:26:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.918+0000 7f99fe3dd740 1 Processor -- start 2024-05-04T03:26:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.918+0000 7f99fe3dd740 1 -- start start 2024-05-04T03:26:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05835a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df05835fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55df0579a5d0 con 0x55df0546b2f0 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55df05753890 con 0x55df054734d0 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df05796590 con 0x55df054941b0 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df05835fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df05835fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34272/0 (socket says 172.21.15.39:34272) 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 -- 172.21.15.39:0/2533056526 learned_addr learned my addr 172.21.15.39:0/2533056526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05835a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 -- 172.21.15.39:0/2533056526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 msgr2=0x55df05835a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99e27fc640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99dc0b2860 0x7f99dc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05835a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 -- 172.21.15.39:0/2533056526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 msgr2=0x55df0583cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 -- 172.21.15.39:0/2533056526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df054ac210 con 0x55df0546b2f0 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f5aca640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99dc0b2860 0x7f99dc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99f52c9640 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df05835fe0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b740 tx=0x7f99ec00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdce7303028a97c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.919+0000 7f99e27fc640 1 -- 172.21.15.39:0/1015183700 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f99e8103870 con 0x7f99f0107800 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.920+0000 7f99db7fe640 1 -- 172.21.15.39:0/2533056526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec003dd0 con 0x55df0546b2f0 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.920+0000 7f99db7fe640 1 -- 172.21.15.39:0/2533056526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99ec015070 con 0x55df0546b2f0 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.920+0000 7f99db7fe640 1 -- 172.21.15.39:0/2533056526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec004c70 con 0x55df0546b2f0 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 -- 172.21.15.39:0/2533056526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 msgr2=0x55df05835fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99f5aca640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99dc0b2860 0x7f99dc0b4d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f99e802ae80 tx=0x7f99e8097820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df05835fe0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b740 tx=0x7f99ec00bc00 comp rx=0 tx=0).stop 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 -- 172.21.15.39:0/2533056526 shutdown_connections 2024-05-04T03:26:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df05835fe0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/2533056526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05835a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 -- 172.21.15.39:0/2533056526 >> 172.21.15.39:0/2533056526 conn(0x55df05488300 msgr2=0x55df058313a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 -- 172.21.15.39:0/2533056526 shutdown_connections 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 -- 172.21.15.39:0/2533056526 wait complete. 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 -- start start 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.921+0000 7f99fe3dd740 1 Processor -- start 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df0546b2f0 0x55df0583f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054941b0 0x55df05846b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55df0583d930 con 0x55df054941b0 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55df05757370 con 0x55df054734d0 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df0583dab0 con 0x55df0546b2f0 2024-05-04T03:26:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99fe3dd740 10 client.0 _ll_register_callbacks cb 0x55df054aba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054941b0 0x55df05846b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054941b0 0x55df05846b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34288/0 (socket says 172.21.15.39:34288) 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 -- 172.21.15.39:0/332436956 learned_addr learned my addr 172.21.15.39:0/332436956 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f52c9640 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df0546b2f0 0x55df0583f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 -- 172.21.15.39:0/332436956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df0546b2f0 msgr2=0x55df0583f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df0546b2f0 0x55df0583f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 -- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 msgr2=0x55df0583fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 -- 172.21.15.39:0/332436956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df05578620 con 0x55df054941b0 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f52c9640 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054941b0 0x55df05846b40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f99e400cec0 tx=0x7f99e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfa18e4fb0a1939 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.922+0000 7f99d8ff9640 1 -- 172.21.15.39:0/332436956 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4004030 con 0x55df054941b0 2024-05-04T03:26:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99d8ff9640 1 -- 172.21.15.39:0/332436956 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f99e40041d0 con 0x55df054941b0 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 1 -- 172.21.15.39:0/332436956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99ec015070 con 0x55df054941b0 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99d8ff9640 1 -- 172.21.15.39:0/332436956 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f99e40043d0 con 0x55df054941b0 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 10 client.17037 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 1 -- 172.21.15.39:0/332436956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df05849ab0 con 0x55df054941b0 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99d8ff9640 1 -- 172.21.15.39:0/332436956 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99ec015070 con 0x55df054941b0 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99d8ff9640 1 -- 172.21.15.39:0/332436956 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f99e40035f0 con 0x55df054941b0 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 10 client.17037 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 20 client.17037 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55df05452d80 0x55df0584d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 4 client.17037 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99fe3dd740 1 -- 172.21.15.39:0/332436956 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df057a81e0 con 0x55df05452d80 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.923+0000 7f99f52c9640 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55df05452d80 0x55df0584d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.924+0000 7f99d8ff9640 10 client.17037 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.924+0000 7f99f52c9640 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55df05452d80 0x55df0584d1d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b468a899b6cbd2 server_cookie=bc725b19fdaf7201 in_seq=0 out_seq=0 2024-05-04T03:26:30.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:30 smithi167 ceph-mon[33614]: pgmap v857: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.6 MiB/s wr, 309 op/s 2024-05-04T03:26:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99d8ff9640 1 -- 172.21.15.39:0/332436956 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df057a81e0 con 0x55df05452d80 2024-05-04T03:26:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99d8ff9640 10 client.17037 handle_command_reply: tid=0 2024-05-04T03:26:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 client.17037 shutdown 2024-05-04T03:26:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 -- 172.21.15.39:0/332436956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054941b0 msgr2=0x55df05846b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054941b0 0x55df05846b40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f99e400cec0 tx=0x7f99e40046c0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 -- 172.21.15.39:0/332436956 shutdown_connections 2024-05-04T03:26:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55df05452d80 0x55df0584d1d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054941b0 0x55df05846b40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054734d0 0x55df0583fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/332436956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df0546b2f0 0x55df0583f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 -- 172.21.15.39:0/332436956 >> 172.21.15.39:0/332436956 conn(0x55df05488300 msgr2=0x55df05834300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99d8ff9640 0 client.17037 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 -- 172.21.15.39:0/332436956 shutdown_connections 2024-05-04T03:26:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 1 -- 172.21.15.39:0/332436956 wait complete. 2024-05-04T03:26:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.925+0000 7f99fe3dd740 20 client.17037 trim_cache size 0 max 16384 2024-05-04T03:26:30.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:30 smithi039 ceph-mon[29715]: pgmap v857: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.6 MiB/s wr, 309 op/s 2024-05-04T03:26:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99fe3dd740 1 Processor -- start 2024-05-04T03:26:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99fe3dd740 1 -- start start 2024-05-04T03:26:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df0583a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054734d0 0x55df058312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054941b0 0x55df05846ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01b1820 con 0x55df0546b2f0 2024-05-04T03:26:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01b3360 con 0x55df054941b0 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01b1ef0 con 0x55df054734d0 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99f4ac8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df0583a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99f4ac8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df0583a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34294/0 (socket says 172.21.15.39:34294) 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.941+0000 7f99f4ac8640 1 -- 172.21.15.39:0/960930622 learned_addr learned my addr 172.21.15.39:0/960930622 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99f4ac8640 1 -- 172.21.15.39:0/960930622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054734d0 msgr2=0x55df058312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/960930622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054734d0 0x55df058312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99f4ac8640 1 -- 172.21.15.39:0/960930622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054941b0 msgr2=0x55df05846ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/960930622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054941b0 0x55df05846ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99f4ac8640 1 -- 172.21.15.39:0/960930622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df058402a0 con 0x55df0546b2f0 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/960930622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df0583a130 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f99e4014960 tx=0x7f99e4011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be22347476e349c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99daffd640 1 -- 172.21.15.39:0/960930622 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4014110 con 0x55df0546b2f0 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99daffd640 1 -- 172.21.15.39:0/960930622 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e402f070 con 0x55df0546b2f0 2024-05-04T03:26:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.942+0000 7f99daffd640 1 -- 172.21.15.39:0/960930622 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e400f890 con 0x55df0546b2f0 2024-05-04T03:26:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 -- 172.21.15.39:0/960930622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 msgr2=0x55df0583a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/960930622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df0583a130 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f99e4014960 tx=0x7f99e4011f90 comp rx=0 tx=0).stop 2024-05-04T03:26:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 -- 172.21.15.39:0/960930622 shutdown_connections 2024-05-04T03:26:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/960930622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df054941b0 0x55df05846ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/960930622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054734d0 0x55df058312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/960930622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df0546b2f0 0x55df0583a130 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 -- 172.21.15.39:0/960930622 >> 172.21.15.39:0/960930622 conn(0x55df05488300 msgr2=0x55df05830bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 -- 172.21.15.39:0/960930622 shutdown_connections 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 -- 172.21.15.39:0/960930622 wait complete. 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 -- start start 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.943+0000 7f99fe3dd740 1 Processor -- start 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df0546b2f0 0x55df058444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054734d0 0x55df05844a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99fe3dd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05845930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01b2c90 con 0x55df054734d0 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01b25c0 con 0x55df0546b2f0 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99fe3dd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01af800 con 0x55df054941b0 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99fe3dd740 10 client.0 _ll_register_callbacks cb 0x7f99e400c730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054734d0 0x55df05844a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054734d0 0x55df05844a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34308/0 (socket says 172.21.15.39:34308) 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 -- 172.21.15.39:0/3050531662 learned_addr learned my addr 172.21.15.39:0/3050531662 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df0546b2f0 0x55df058444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05845930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 -- 172.21.15.39:0/3050531662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 msgr2=0x55df05845930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05845930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 -- 172.21.15.39:0/3050531662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df0546b2f0 msgr2=0x55df058444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df0546b2f0 0x55df058444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 -- 172.21.15.39:0/3050531662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df05845ef0 con 0x55df054734d0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05845930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99f52c9640 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054734d0 0x55df05844a00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b740 tx=0x7f99ec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e41609433534fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.944+0000 7f99d8ff9640 1 -- 172.21.15.39:0/3050531662 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec02d060 con 0x55df054734d0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99d8ff9640 1 -- 172.21.15.39:0/3050531662 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f99ec028450 con 0x55df054734d0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 1 -- 172.21.15.39:0/3050531662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e402f070 con 0x55df054734d0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99d8ff9640 1 -- 172.21.15.39:0/3050531662 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f99ec028650 con 0x55df054734d0 2024-05-04T03:26:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 10 client.17043 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 1 -- 172.21.15.39:0/3050531662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df05847280 con 0x55df054734d0 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99d8ff9640 1 -- 172.21.15.39:0/3050531662 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e402f070 con 0x55df054734d0 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99d8ff9640 1 -- 172.21.15.39:0/3050531662 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f99ec026b80 con 0x55df054734d0 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 10 client.17043 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 10 client.17043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 20 client.17043 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55df054281c0 0x55df058b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 4 client.17043 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99fe3dd740 1 -- 172.21.15.39:0/3050531662 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df057a2050 con 0x55df054281c0 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.945+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55df054281c0 0x55df058b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.946+0000 7f99f4ac8640 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55df054281c0 0x55df058b7810 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449d4158cde67a0f server_cookie=7e58a992e4690756 in_seq=0 out_seq=0 2024-05-04T03:26:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.946+0000 7f99d8ff9640 10 client.17043 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99d8ff9640 1 -- 172.21.15.39:0/3050531662 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df057a2050 con 0x55df054281c0 2024-05-04T03:26:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99d8ff9640 10 client.17043 handle_command_reply: tid=0 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 client.17043 shutdown 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 -- 172.21.15.39:0/3050531662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054734d0 msgr2=0x55df05844a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054734d0 0x55df05844a00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b740 tx=0x7f99ec00be50 comp rx=0 tx=0).stop 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 -- 172.21.15.39:0/3050531662 shutdown_connections 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55df054281c0 0x55df058b7810 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df054941b0 0x55df05845930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55df054734d0 0x55df05844a00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99d8ff9640 0 client.17043 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 --2- 172.21.15.39:0/3050531662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df0546b2f0 0x55df058444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 -- 172.21.15.39:0/3050531662 >> 172.21.15.39:0/3050531662 conn(0x55df05488300 msgr2=0x55df058519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 -- 172.21.15.39:0/3050531662 shutdown_connections 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 1 -- 172.21.15.39:0/3050531662 wait complete. 2024-05-04T03:26:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.955+0000 7f99fe3dd740 20 client.17043 trim_cache size 0 max 16384 2024-05-04T03:26:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:26:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:26:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:26:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 -- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99dc0b2860 msgr2=0x7f99dc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99dc0b2860 0x7f99dc0b4d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f99e802ae80 tx=0x7f99e8097820 comp rx=0 tx=0).stop 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 -- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 msgr2=0x7f99f01a7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 0x7f99f01a7d20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f99e802aeb0 tx=0x7f99e80039f0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 -- 172.21.15.39:0/1015183700 shutdown_connections 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99dc0b2860 0x7f99dc0b4d70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f0107800 0x7f99f01a7d20 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99f0104ea0 0x7f99f01a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 --2- 172.21.15.39:0/1015183700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99f0102540 0x7f99f01a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 -- 172.21.15.39:0/1015183700 >> 172.21.15.39:0/1015183700 conn(0x7f99f00fdd50 msgr2=0x7f99f0103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 -- 172.21.15.39:0/1015183700 shutdown_connections 2024-05-04T03:26:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:30.956+0000 7f99f752d640 1 -- 172.21.15.39:0/1015183700 wait complete. 2024-05-04T03:26:31.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 760.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:26:32.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:32 smithi066 ceph-mon[33264]: pgmap v858: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-05-04T03:26:32.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:32 smithi167 ceph-mon[33614]: pgmap v858: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-05-04T03:26:32.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:32 smithi039 ceph-mon[29715]: pgmap v858: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-05-04T03:26:34.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:34 smithi066 ceph-mon[33264]: pgmap v859: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 339 KiB/s wr, 52 op/s 2024-05-04T03:26:34.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:34 smithi167 ceph-mon[33614]: pgmap v859: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 339 KiB/s wr, 52 op/s 2024-05-04T03:26:34.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:34 smithi039 ceph-mon[29715]: pgmap v859: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 339 KiB/s wr, 52 op/s 2024-05-04T03:26:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 765.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:26:36.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:36 smithi066 ceph-mon[33264]: pgmap v860: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 171 KiB/s wr, 31 op/s 2024-05-04T03:26:36.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:36 smithi167 ceph-mon[33614]: pgmap v860: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 171 KiB/s wr, 31 op/s 2024-05-04T03:26:36.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:36 smithi039 ceph-mon[29715]: pgmap v860: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 171 KiB/s wr, 31 op/s 2024-05-04T03:26:38.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:38 smithi066 ceph-mon[33264]: pgmap v861: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 194 KiB/s wr, 37 op/s 2024-05-04T03:26:38.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:38 smithi167 ceph-mon[33614]: pgmap v861: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 194 KiB/s wr, 37 op/s 2024-05-04T03:26:38.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:38 smithi039 ceph-mon[29715]: pgmap v861: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 194 KiB/s wr, 37 op/s 2024-05-04T03:26:40.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:40 smithi066 ceph-mon[33264]: pgmap v862: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 213 KiB/s wr, 37 op/s 2024-05-04T03:26:40.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:40 smithi167 ceph-mon[33614]: pgmap v862: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 213 KiB/s wr, 37 op/s 2024-05-04T03:26:40.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:40 smithi039 ceph-mon[29715]: pgmap v862: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 213 KiB/s wr, 37 op/s 2024-05-04T03:26:41.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 770.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:26:42.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:42 smithi066 ceph-mon[33264]: pgmap v863: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 192 KiB/s wr, 35 op/s 2024-05-04T03:26:42.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:42 smithi167 ceph-mon[33614]: pgmap v863: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 192 KiB/s wr, 35 op/s 2024-05-04T03:26:42.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:42 smithi039 ceph-mon[29715]: pgmap v863: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 192 KiB/s wr, 35 op/s 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 -- 172.21.15.39:0/118814255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb01024e0 msgr2=0x7fbbb0109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 --2- 172.21.15.39:0/118814255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb01024e0 0x7fbbb0109270 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c02a410 comp rx=0 tx=0).stop 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 -- 172.21.15.39:0/118814255 shutdown_connections 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 --2- 172.21.15.39:0/118814255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbb0109900 0x7fbbb010be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 --2- 172.21.15.39:0/118814255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb01024e0 0x7fbbb0109270 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 --2- 172.21.15.39:0/118814255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb0101fe0 0x7fbbb0106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 -- 172.21.15.39:0/118814255 >> 172.21.15.39:0/118814255 conn(0x7fbbb00fd4f0 msgr2=0x7fbbb00ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 -- 172.21.15.39:0/118814255 shutdown_connections 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.068+0000 7fbbb7213640 1 -- 172.21.15.39:0/118814255 wait complete. 2024-05-04T03:26:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb7213640 1 Processor -- start 2024-05-04T03:26:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb7213640 1 -- start start 2024-05-04T03:26:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb7213640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbb0101fe0 0x7fbbb019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb7213640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb01024e0 0x7fbbb019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb7213640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb0109900 0x7fbbb01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb7213640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb010f600 con 0x7fbbb0109900 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb7213640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb010f480 con 0x7fbbb0101fe0 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb7213640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb010f780 con 0x7fbbb01024e0 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb57b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb0109900 0x7fbbb01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb57b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb0109900 0x7fbbb01a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46578/0 (socket says 172.21.15.39:46578) 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.069+0000 7fbbb57b0640 1 -- 172.21.15.39:0/1449621290 learned_addr learned my addr 172.21.15.39:0/1449621290 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbb0101fe0 0x7fbbb019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbaffff640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb01024e0 0x7fbbb019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbb57b0640 1 -- 172.21.15.39:0/1449621290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb01024e0 msgr2=0x7fbbb019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbb57b0640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb01024e0 0x7fbbb019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbb57b0640 1 -- 172.21.15.39:0/1449621290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbb0101fe0 msgr2=0x7fbbb019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbb57b0640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbb0101fe0 0x7fbbb019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbb57b0640 1 -- 172.21.15.39:0/1449621290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb01a1d80 con 0x7fbbb0109900 2024-05-04T03:26:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbb57b0640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb0109900 0x7fbbb01a1710 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbba800eb10 tx=0x7fbba800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbadffb640 1 -- 172.21.15.39:0/1449621290 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8004b20 con 0x7fbbb0109900 2024-05-04T03:26:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbadffb640 1 -- 172.21.15.39:0/1449621290 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbba8004cc0 con 0x7fbbb0109900 2024-05-04T03:26:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbb7213640 1 -- 172.21.15.39:0/1449621290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb01a2070 con 0x7fbbb0109900 2024-05-04T03:26:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.070+0000 7fbbadffb640 1 -- 172.21.15.39:0/1449621290 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8010690 con 0x7fbbb0109900 2024-05-04T03:26:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.071+0000 7fbbb7213640 1 -- 172.21.15.39:0/1449621290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01a25c0 con 0x7fbbb0109900 2024-05-04T03:26:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.073+0000 7fbbbe1a0740 1 Processor -- start 2024-05-04T03:26:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.073+0000 7fbbadffb640 1 -- 172.21.15.39:0/1449621290 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbba8005020 con 0x7fbbb0109900 2024-05-04T03:26:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbbe1a0740 1 -- start start 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 0x5647c535da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c535dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f9b4d0 0x5647c5364ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5647c52c25d0 con 0x5647c4f9b4d0 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5647c527b890 con 0x5647c4f932f0 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5647c52be590 con 0x5647c4fbc1b0 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f9b4d0 0x5647c5364ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f9b4d0 0x5647c5364ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46590/0 (socket says 172.21.15.39:46590) 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbaffff640 1 -- 172.21.15.39:0/2633969879 learned_addr learned my addr 172.21.15.39:0/2633969879 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbadffb640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbba00b2860 0x7fbba00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c535dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbaffff640 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 0x5647c535da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbb57b0640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbba00b2860 0x7fbba00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbaffff640 1 -- 172.21.15.39:0/2633969879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 msgr2=0x5647c535da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbaffff640 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 0x5647c535da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbaffff640 1 -- 172.21.15.39:0/2633969879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 msgr2=0x5647c535dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbadffb640 1 -- 172.21.15.39:0/1449621290 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbba80e2730 con 0x7fbbb0109900 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.074+0000 7fbbaffff640 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c535dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.075+0000 7fbbaffff640 1 -- 172.21.15.39:0/2633969879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c4fd4210 con 0x5647c4f9b4d0 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.075+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c535dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.075+0000 7fbbaffff640 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f9b4d0 0x5647c5364ee0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0096c0 tx=0x7fbb9c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb0b380b6f1528f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.075+0000 7fbbb57b0640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbba00b2860 0x7fbba00b4d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbba80a5de0 tx=0x7fbba800cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.075+0000 7fbb9affd640 1 -- 172.21.15.39:0/2633969879 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c035040 con 0x5647c4f9b4d0 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.075+0000 7fbb9affd640 1 -- 172.21.15.39:0/2633969879 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb9c004030 con 0x5647c4f9b4d0 2024-05-04T03:26:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.075+0000 7fbb9affd640 1 -- 172.21.15.39:0/2633969879 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0038f0 con 0x5647c4f9b4d0 2024-05-04T03:26:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2633969879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f9b4d0 msgr2=0x5647c5364ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f9b4d0 0x5647c5364ee0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0096c0 tx=0x7fbb9c005fd0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2633969879 shutdown_connections 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f9b4d0 0x5647c5364ee0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c535dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2633969879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 0x5647c535da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2633969879 >> 172.21.15.39:0/2633969879 conn(0x5647c4fb0300 msgr2=0x5647c53593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2633969879 shutdown_connections 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2633969879 wait complete. 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 -- start start 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.076+0000 7fbbbe1a0740 1 Processor -- start 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c5367670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c5367bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5647c53658d0 con 0x5647c4fbc1b0 2024-05-04T03:26:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5647c527f370 con 0x5647c4f932f0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5647c5365a50 con 0x5647c4f9b4d0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbbe1a0740 10 client.0 _ll_register_callbacks cb 0x5647c4fd3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46602/0 (socket says 172.21.15.39:46602) 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 -- 172.21.15.39:0/1063719903 learned_addr learned my addr 172.21.15.39:0/1063719903 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c5367bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c5367670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 -- 172.21.15.39:0/1063719903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 msgr2=0x5647c5367bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c5367bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 -- 172.21.15.39:0/1063719903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 msgr2=0x5647c5367670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c5367670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.077+0000 7fbbaffff640 1 -- 172.21.15.39:0/1063719903 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c50a0620 con 0x5647c4fbc1b0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbbaffff640 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536eb10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c005760 tx=0x7fbb9c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2d75e05276a609 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbb8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbb8ffff640 1 -- 172.21.15.39:0/1063719903 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c035070 con 0x5647c4fbc1b0 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbb8ffff640 1 -- 172.21.15.39:0/1063719903 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbb9c0036d0 con 0x5647c4fbc1b0 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/1063719903 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9c004030 con 0x5647c4fbc1b0 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbb8ffff640 1 -- 172.21.15.39:0/1063719903 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb9c000e60 con 0x5647c4fbc1b0 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbbbe1a0740 10 client.17055 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/1063719903 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647c53658d0 con 0x5647c4fbc1b0 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbb8ffff640 1 -- 172.21.15.39:0/1063719903 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb9c004030 con 0x5647c4fbc1b0 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbb8ffff640 1 -- 172.21.15.39:0/1063719903 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbb9c04b8d0 con 0x5647c4fbc1b0 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.078+0000 7fbbbe1a0740 10 client.17055 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.079+0000 7fbbbe1a0740 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.079+0000 7fbbbe1a0740 20 client.17055 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.079+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5647c4f7ad80 0x5647c5374bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.079+0000 7fbbbe1a0740 4 client.17055 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.079+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/1063719903 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647c52d01e0 con 0x5647c4f7ad80 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.079+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5647c4f7ad80 0x5647c5374bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.079+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5647c4f7ad80 0x5647c5374bd0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f48f071fe219159 server_cookie=b92da99164fb43b2 in_seq=0 out_seq=0 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.079+0000 7fbb8ffff640 10 client.17055 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.080+0000 7fbb8ffff640 1 -- 172.21.15.39:0/1063719903 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647c52d01e0 con 0x5647c4f7ad80 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.080+0000 7fbb8ffff640 10 client.17055 handle_command_reply: tid=0 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.080+0000 7fbbbe1a0740 1 client.17055 shutdown 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.080+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/1063719903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 msgr2=0x5647c536eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.080+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536eb10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c005760 tx=0x7fbb9c005c90 comp rx=0 tx=0).stop 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/1063719903 shutdown_connections 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5647c4f7ad80 0x5647c5374bd0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536eb10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbb8ffff640 0 client.17055 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c5367bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/1063719903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c5367670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/1063719903 >> 172.21.15.39:0/1063719903 conn(0x5647c4fb0300 msgr2=0x5647c535c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/1063719903 shutdown_connections 2024-05-04T03:26:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/1063719903 wait complete. 2024-05-04T03:26:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.081+0000 7fbbbe1a0740 20 client.17055 trim_cache size 0 max 16384 2024-05-04T03:26:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbbe1a0740 1 Processor -- start 2024-05-04T03:26:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbbe1a0740 1 -- start start 2024-05-04T03:26:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c5357970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c536ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01ab2a0 con 0x5647c4fbc1b0 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01acde0 con 0x5647c4f932f0 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01ab970 con 0x5647c4f9b4d0 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.096+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46606/0 (socket says 172.21.15.39:46606) 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 -- 172.21.15.39:0/3120529929 learned_addr learned my addr 172.21.15.39:0/3120529929 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c536ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c5357970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 -- 172.21.15.39:0/3120529929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 msgr2=0x5647c536ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c536ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 -- 172.21.15.39:0/3120529929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 msgr2=0x5647c5357970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c5357970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 -- 172.21.15.39:0/3120529929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c535d570 con 0x5647c4fbc1b0 2024-05-04T03:26:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c536ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbbaffff640 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536ef70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009800 tx=0x7fbb9c04d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea2c544e6032c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbb9a7fc640 1 -- 172.21.15.39:0/3120529929 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c035040 con 0x5647c4fbc1b0 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbb9a7fc640 1 -- 172.21.15.39:0/3120529929 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb9c052070 con 0x5647c4fbc1b0 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.097+0000 7fbb9a7fc640 1 -- 172.21.15.39:0/3120529929 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c03d6b0 con 0x5647c4fbc1b0 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/3120529929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 msgr2=0x5647c536ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536ef70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009800 tx=0x7fbb9c04d040 comp rx=0 tx=0).stop 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/3120529929 shutdown_connections 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4fbc1b0 0x5647c536ef70 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4f9b4d0 0x5647c536ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/3120529929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f932f0 0x5647c5357970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/3120529929 >> 172.21.15.39:0/3120529929 conn(0x5647c4fb0300 msgr2=0x5647c5358dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/3120529929 shutdown_connections 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/3120529929 wait complete. 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 -- start start 2024-05-04T03:26:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.098+0000 7fbbbe1a0740 1 Processor -- start 2024-05-04T03:26:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f932f0 0x5647c536ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f9b4d0 0x5647c536cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbbe1a0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 0x5647c536deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01ac710 con 0x5647c4f932f0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01ac040 con 0x5647c4f9b4d0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbbe1a0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01a9280 con 0x5647c4fbc1b0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbbe1a0740 10 client.0 _ll_register_callbacks cb 0x7fbba400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f932f0 0x5647c536ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f932f0 0x5647c536ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46618/0 (socket says 172.21.15.39:46618) 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 -- 172.21.15.39:0/2734143248 learned_addr learned my addr 172.21.15.39:0/2734143248 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f9b4d0 0x5647c536cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 0x5647c536deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 -- 172.21.15.39:0/2734143248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 msgr2=0x5647c536deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 0x5647c536deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 -- 172.21.15.39:0/2734143248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f9b4d0 msgr2=0x5647c536cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f9b4d0 0x5647c536cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.099+0000 7fbbaffff640 1 -- 172.21.15.39:0/2734143248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c536e470 con 0x5647c4f932f0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbaffff640 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f932f0 0x5647c536ca40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c04bd20 tx=0x7fbb9c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0de0f49426eef server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbb8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbb8ffff640 1 -- 172.21.15.39:0/2734143248 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c035070 con 0x5647c4f932f0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbb8ffff640 1 -- 172.21.15.39:0/2734143248 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbb9c059680 con 0x5647c4f932f0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2734143248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9c052070 con 0x5647c4f932f0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbb8ffff640 1 -- 172.21.15.39:0/2734143248 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbb9c059880 con 0x5647c4f932f0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 10 client.17061 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2734143248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647c53dce10 con 0x5647c4f932f0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbb8ffff640 1 -- 172.21.15.39:0/2734143248 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb9c052070 con 0x5647c4f932f0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbb8ffff640 1 -- 172.21.15.39:0/2734143248 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbb9c070020 con 0x5647c4f932f0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 10 client.17061 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 20 client.17061 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5647c4f501c0 0x5647c53df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 4 client.17061 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.100+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2734143248 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647c52ca050 con 0x5647c4f501c0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.101+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5647c4f501c0 0x5647c53df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.101+0000 7fbbb4faf640 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5647c4f501c0 0x5647c53df3f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d7fa3a7f7c9a45 server_cookie=bb6faa3cd2673a16 in_seq=0 out_seq=0 2024-05-04T03:26:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.101+0000 7fbb8ffff640 10 client.17061 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbb8ffff640 1 -- 172.21.15.39:0/2734143248 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647c52ca050 con 0x5647c4f501c0 2024-05-04T03:26:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbb8ffff640 10 client.17061 handle_command_reply: tid=0 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 client.17061 shutdown 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2734143248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f932f0 msgr2=0x5647c536ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f932f0 0x5647c536ca40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c04bd20 tx=0x7fbb9c032f60 comp rx=0 tx=0).stop 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2734143248 shutdown_connections 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5647c4f501c0 0x5647c53df3f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647c4fbc1b0 0x5647c536deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbb8ffff640 0 client.17061 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647c4f9b4d0 0x5647c536cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 --2- 172.21.15.39:0/2734143248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5647c4f932f0 0x5647c536ca40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2734143248 >> 172.21.15.39:0/2734143248 conn(0x5647c4fb0300 msgr2=0x5647c535c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2734143248 shutdown_connections 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 1 -- 172.21.15.39:0/2734143248 wait complete. 2024-05-04T03:26:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.106+0000 7fbbbe1a0740 20 client.17061 trim_cache size 0 max 16384 2024-05-04T03:26:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:26:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:26:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:26:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:26:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 -- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbba00b2860 msgr2=0x7fbba00b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbba00b2860 0x7fbba00b4d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbba80a5de0 tx=0x7fbba800cf70 comp rx=0 tx=0).stop 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 -- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb0109900 msgr2=0x7fbbb01a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb0109900 0x7fbbb01a1710 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbba800eb10 tx=0x7fbba800c2d0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 -- 172.21.15.39:0/1449621290 shutdown_connections 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbba00b2860 0x7fbba00b4d70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb0109900 0x7fbbb01a1710 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbb01024e0 0x7fbbb019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 --2- 172.21.15.39:0/1449621290 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbb0101fe0 0x7fbbb019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 -- 172.21.15.39:0/1449621290 >> 172.21.15.39:0/1449621290 conn(0x7fbbb00fd4f0 msgr2=0x7fbbb01068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 -- 172.21.15.39:0/1449621290 shutdown_connections 2024-05-04T03:26:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:44.107+0000 7fbbb7213640 1 -- 172.21.15.39:0/1449621290 wait complete. 2024-05-04T03:26:44.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:26:44.451 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:44.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:44 smithi066 ceph-mon[33264]: pgmap v864: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-05-04T03:26:44.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:44 smithi039 ceph-mon[29715]: pgmap v864: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-05-04T03:26:44.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:44 smithi167 ceph-mon[33614]: pgmap v864: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-05-04T03:26:46.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 -- 172.21.15.39:0/4001490577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3478108740 msgr2=0x7f34781100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:46.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 --2- 172.21.15.39:0/4001490577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3478108740 0x7f34781100a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f346c0099c0 tx=0x7f346c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 -- 172.21.15.39:0/4001490577 shutdown_connections 2024-05-04T03:26:46.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 --2- 172.21.15.39:0/4001490577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3478108740 0x7f34781100a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 --2- 172.21.15.39:0/4001490577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34781034c0 0x7f34781038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 --2- 172.21.15.39:0/4001490577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3478107df0 0x7f3478108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 -- 172.21.15.39:0/4001490577 >> 172.21.15.39:0/4001490577 conn(0x7f34780fedc0 msgr2=0x7f34781011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:46.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 -- 172.21.15.39:0/4001490577 shutdown_connections 2024-05-04T03:26:46.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.457+0000 7f347d04b700 1 -- 172.21.15.39:0/4001490577 wait complete. 2024-05-04T03:26:46.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.458+0000 7f347d04b700 1 Processor -- start 2024-05-04T03:26:46.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.458+0000 7f347d04b700 1 -- start start 2024-05-04T03:26:46.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.458+0000 7f347d04b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34781034c0 0x7f347819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:46.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.458+0000 7f347d04b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3478107df0 0x7f347819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:46.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.458+0000 7f347d04b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3478108740 0x7f347819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:46.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.458+0000 7f347d04b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3478113220 con 0x7f34781034c0 2024-05-04T03:26:46.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.458+0000 7f347d04b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f34781130a0 con 0x7f3478107df0 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.458+0000 7f347d04b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f34781133a0 con 0x7f3478108740 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34781034c0 0x7f347819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34781034c0 0x7f347819bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39852/0 (socket says 172.21.15.39:39852) 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 -- 172.21.15.39:0/2969574477 learned_addr learned my addr 172.21.15.39:0/2969574477 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f347659c700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3478107df0 0x7f347819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f347759e700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3478108740 0x7f347819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 -- 172.21.15.39:0/2969574477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3478108740 msgr2=0x7f347819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3478108740 0x7f347819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 -- 172.21.15.39:0/2969574477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3478107df0 msgr2=0x7f347819c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3478107df0 0x7f347819c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 -- 172.21.15.39:0/2969574477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34781a6de0 con 0x7f34781034c0 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f347659c700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3478107df0 0x7f347819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f3476d9d700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34781034c0 0x7f347819bb00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f346800c9b0 tx=0x7f346800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f345ffff700 1 -- 172.21.15.39:0/2969574477 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468007b00 con 0x7f34781034c0 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34781a70d0 con 0x7f34781034c0 2024-05-04T03:26:46.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f345ffff700 1 -- 172.21.15.39:0/2969574477 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3468007ca0 con 0x7f34781034c0 2024-05-04T03:26:46.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.459+0000 7f345ffff700 1 -- 172.21.15.39:0/2969574477 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468018660 con 0x7f34781034c0 2024-05-04T03:26:46.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.460+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34781a75e0 con 0x7f34781034c0 2024-05-04T03:26:46.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.461+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34780512c0 con 0x7f34781034c0 2024-05-04T03:26:46.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.463+0000 7f345ffff700 1 -- 172.21.15.39:0/2969574477 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f346800f450 con 0x7f34781034c0 2024-05-04T03:26:46.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.464+0000 7f345ffff700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f34600b2880 0x7f34600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:46.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.464+0000 7f347659c700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f34600b2880 0x7f34600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:46.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.464+0000 7f345ffff700 1 -- 172.21.15.39:0/2969574477 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f34680e25f0 con 0x7f34781034c0 2024-05-04T03:26:46.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.464+0000 7f347659c700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f34600b2880 0x7f34600b4d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3478102440 tx=0x7f3470009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:46.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.468+0000 7f345ffff700 1 -- 172.21.15.39:0/2969574477 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3468014030 con 0x7f34781034c0 2024-05-04T03:26:46.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.731+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f347802e5f0 con 0x7f34781034c0 2024-05-04T03:26:46.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:46 smithi039 ceph-mon[29715]: pgmap v865: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 181 KiB/s wr, 31 op/s 2024-05-04T03:26:46.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.732+0000 7f345ffff700 1 -- 172.21.15.39:0/2969574477 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f34680aa350 con 0x7f34781034c0 2024-05-04T03:26:46.735 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:26:46.735 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:26:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.734+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f34600b2880 msgr2=0x7f34600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f34600b2880 0x7f34600b4d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3478102440 tx=0x7f3470009040 comp rx=0 tx=0).stop 2024-05-04T03:26:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34781034c0 msgr2=0x7f347819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34781034c0 0x7f347819bb00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f346800c9b0 tx=0x7f346800cd70 comp rx=0 tx=0).stop 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 shutdown_connections 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f34600b2880 0x7f34600b4d90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34781034c0 0x7f347819bb00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3478108740 0x7f347819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 --2- 172.21.15.39:0/2969574477 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3478107df0 0x7f347819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 >> 172.21.15.39:0/2969574477 conn(0x7f34780fedc0 msgr2=0x7f3478107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 shutdown_connections 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:46.735+0000 7f347d04b700 1 -- 172.21.15.39:0/2969574477 wait complete. 2024-05-04T03:26:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:26:46.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 775.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:26:46.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:46 smithi066 ceph-mon[33264]: pgmap v865: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 181 KiB/s wr, 31 op/s 2024-05-04T03:26:46.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:46 smithi167 ceph-mon[33614]: pgmap v865: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 181 KiB/s wr, 31 op/s 2024-05-04T03:26:47.540 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 8ffce17a --release --await' 2024-05-04T03:26:47.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id 8ffce17a --release --await 2024-05-04T03:26:47.806 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:47.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:47 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2969574477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:47.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:47 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2969574477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:47.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:47 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2969574477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:48.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:48 smithi066 ceph-mon[33264]: pgmap v866: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 184 KiB/s wr, 34 op/s 2024-05-04T03:26:48.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:48 smithi167 ceph-mon[33614]: pgmap v866: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 184 KiB/s wr, 34 op/s 2024-05-04T03:26:48.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:48 smithi039 ceph-mon[29715]: pgmap v866: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 184 KiB/s wr, 34 op/s 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 -- 172.21.15.39:0/3950682917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 msgr2=0x7fc7ac1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/3950682917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 0x7fc7ac1100b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b0d0 tx=0x7fc7a400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 -- 172.21.15.39:0/3950682917 shutdown_connections 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/3950682917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 0x7fc7ac1100b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/3950682917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7ac1034d0 0x7fc7ac1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/3950682917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7ac107e00 0x7fc7ac108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 -- 172.21.15.39:0/3950682917 >> 172.21.15.39:0/3950682917 conn(0x7fc7ac0fef20 msgr2=0x7fc7ac101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 -- 172.21.15.39:0/3950682917 shutdown_connections 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.577+0000 7fc7b26e0700 1 -- 172.21.15.39:0/3950682917 wait complete. 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.578+0000 7fc7b26e0700 1 Processor -- start 2024-05-04T03:26:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.578+0000 7fc7b26e0700 1 -- start start 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.578+0000 7fc7b26e0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7ac1034d0 0x7fc7ac1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.578+0000 7fc7b26e0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7ac107e00 0x7fc7ac1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.578+0000 7fc7b26e0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 0x7fc7ac19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.578+0000 7fc7b26e0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac113230 con 0x7fc7ac108750 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.578+0000 7fc7b26e0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1130b0 con 0x7fc7ac1034d0 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.578+0000 7fc7b26e0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1133b0 con 0x7fc7ac107e00 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 0x7fc7ac19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 0x7fc7ac19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39874/0 (socket says 172.21.15.39:39874) 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/2412446079 learned_addr learned my addr 172.21.15.39:0/2412446079 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7ac107e00 0x7fc7ac1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7abfff700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7ac1034d0 0x7fc7ac1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/2412446079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7ac107e00 msgr2=0x7fc7ac1a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7ac107e00 0x7fc7ac1a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/2412446079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7ac1034d0 msgr2=0x7fc7ac1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7ac1034d0 0x7fc7ac1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/2412446079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac19b1b0 con 0x7fc7ac108750 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 0x7fc7ac19ab40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4007d90 tx=0x7fc7a4007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7a97fa700 1 -- 172.21.15.39:0/2412446079 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a401f030 con 0x7fc7ac108750 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7a97fa700 1 -- 172.21.15.39:0/2412446079 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7a401bb90 con 0x7fc7ac108750 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7a97fa700 1 -- 172.21.15.39:0/2412446079 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4012bd0 con 0x7fc7ac108750 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.579+0000 7fc7b26e0700 1 -- 172.21.15.39:0/2412446079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac19b440 con 0x7fc7ac108750 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.580+0000 7fc7b26e0700 1 -- 172.21.15.39:0/2412446079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac19b930 con 0x7fc7ac108750 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.582+0000 7fc7a97fa700 1 -- 172.21.15.39:0/2412446079 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fc7a4022030 con 0x7fc7ac108750 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.583+0000 7fc7a97fa700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7940b2900 0x7fc7940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.583+0000 7fc7abfff700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7940b2900 0x7fc7940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.583+0000 7fc7a97fa700 1 -- 172.21.15.39:0/2412446079 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc7a400f020 con 0x7fc7ac108750 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.583+0000 7fc7c10f4b80 1 Processor -- start 2024-05-04T03:26:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.583+0000 7fc7c10f4b80 1 -- start start 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7abfff700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7940b2900 0x7fc7940b4e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc79c005950 tx=0x7fc79c005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffdcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdd420 0x55578ffe7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55578feffa40 con 0x55578ffe7c40 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55578fe64ea0 con 0x55578ffdd420 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55578fefba00 con 0x55578ffdcad0 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffea060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39876/0 (socket says 172.21.15.39:39876) 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/2962356071 learned_addr learned my addr 172.21.15.39:0/2962356071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7abfff700 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffdcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/2962356071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 msgr2=0x55578ffdcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdd420 0x55578ffe7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffdcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/2962356071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdd420 msgr2=0x55578ffe7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdd420 0x55578ffe7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/2962356071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55578fe4c9d0 con 0x55578ffe7c40 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7abfff700 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffdcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffea060 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc7a401bfd0 tx=0x7fc7a400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f444a713bd76419 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7927fc700 1 -- 172.21.15.39:0/2962356071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40b38c0 con 0x55578ffe7c40 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7927fc700 1 -- 172.21.15.39:0/2962356071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7a401bd40 con 0x55578ffe7c40 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.584+0000 7fc7927fc700 1 -- 172.21.15.39:0/2962356071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40b3430 con 0x55578ffe7c40 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2962356071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 msgr2=0x55578ffea060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffea060 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc7a401bfd0 tx=0x7fc7a400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2962356071 shutdown_connections 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffea060 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffdcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2962356071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdd420 0x55578ffe7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2962356071 >> 172.21.15.39:0/2962356071 conn(0x55578fe5ff40 msgr2=0x55578ffd89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2962356071 shutdown_connections 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.585+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2962356071 wait complete. 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.586+0000 7fc7c10f4b80 1 -- start start 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.586+0000 7fc7c10f4b80 1 Processor -- start 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.586+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffe4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.586+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdd420 0x55578ffe5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.586+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 0x55578ffe57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.586+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55578ffea7d0 con 0x55578ffdd420 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.586+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55578feb8fd0 con 0x55578ffe7c40 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.586+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55578ffea950 con 0x55578ffdcad0 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7c10f4b80 10 client.0 _ll_register_callbacks cb 0x55578fe97a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdd420 0x55578ffe5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdd420 0x55578ffe5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39882/0 (socket says 172.21.15.39:39882) 2024-05-04T03:26:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 -- 172.21.15.39:0/2614672505 learned_addr learned my addr 172.21.15.39:0/2614672505 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 0x55578ffe57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7abfff700 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffe4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 -- 172.21.15.39:0/2614672505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 msgr2=0x55578ffe4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffe4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 -- 172.21.15.39:0/2614672505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 msgr2=0x55578ffe57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 0x55578ffe57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 -- 172.21.15.39:0/2614672505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578feb6860 con 0x55578ffdd420 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdd420 0x55578ffe5290 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0011660 tx=0x7fc7a0011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4af23f3da09070f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc78b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc78b7fe700 1 -- 172.21.15.39:0/2614672505 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0012920 con 0x55578ffdd420 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2614672505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a401bd40 con 0x55578ffdd420 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc78b7fe700 1 -- 172.21.15.39:0/2614672505 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc7a0012ac0 con 0x55578ffdd420 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc78b7fe700 1 -- 172.21.15.39:0/2614672505 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7a0012cc0 con 0x55578ffdd420 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7c10f4b80 10 client.17079 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.587+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2614672505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55578fff5460 con 0x55578ffdd420 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc78b7fe700 1 -- 172.21.15.39:0/2614672505 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7a401bd40 con 0x55578ffdd420 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc78b7fe700 1 -- 172.21.15.39:0/2614672505 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc7a001a3f0 con 0x55578ffdd420 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc7c10f4b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc7c10f4b80 10 client.17079 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc7c10f4b80 20 client.17079 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55578fff7b40 0x55578fff9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc7c10f4b80 4 client.17079 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2614672505 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55578ff0d420 con 0x55578fff7b40 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.588+0000 7fc7abfff700 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55578fff7b40 0x55578fff9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.589+0000 7fc78b7fe700 10 client.17079 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.589+0000 7fc7abfff700 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55578fff7b40 0x55578fff9f60 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85afac8d8a169e13 server_cookie=ffb60be492b977a1 in_seq=0 out_seq=0 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.589+0000 7fc78b7fe700 1 -- 172.21.15.39:0/2614672505 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55578ff0d420 con 0x55578fff7b40 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.589+0000 7fc78b7fe700 10 client.17079 handle_command_reply: tid=0 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.589+0000 7fc7c10f4b80 1 client.17079 shutdown 2024-05-04T03:26:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2614672505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdd420 msgr2=0x55578ffe5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdd420 0x55578ffe5290 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0011660 tx=0x7fc7a0011970 comp rx=0 tx=0).stop 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2614672505 shutdown_connections 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55578fff7b40 0x55578fff9f60 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 0x55578ffe57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdcad0 0x55578ffe4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/2614672505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdd420 0x55578ffe5290 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc78b7fe700 0 client.17079 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2614672505 >> 172.21.15.39:0/2614672505 conn(0x55578fe5ff40 msgr2=0x55578ffde570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2614672505 shutdown_connections 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/2614672505 wait complete. 2024-05-04T03:26:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.590+0000 7fc7c10f4b80 20 client.17079 trim_cache size 0 max 16384 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.610+0000 7fc7c10f4b80 1 Processor -- start 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.610+0000 7fc7c10f4b80 1 -- start start 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdcad0 0x55578ffd8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 0x55578ffd8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1aea20 con 0x55578ffe7c40 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1b0560 con 0x55578ffdcad0 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1af0f0 con 0x55578ffdd420 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39898/0 (socket says 172.21.15.39:39898) 2024-05-04T03:26:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/565563620 learned_addr learned my addr 172.21.15.39:0/565563620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7abfff700 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdcad0 0x55578ffd8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 0x55578ffd8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/565563620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 msgr2=0x55578ffd8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 0x55578ffd8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/565563620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdcad0 msgr2=0x55578ffd8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdcad0 0x55578ffd8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 -- 172.21.15.39:0/565563620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55578ffd03a0 con 0x55578ffe7c40 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7abfff700 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdcad0 0x55578ffd8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.611+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffff6f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40a6c50 tx=0x7fc7a40aaad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a75030f87b7b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc791ffb700 1 -- 172.21.15.39:0/565563620 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40c3070 con 0x55578ffe7c40 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc791ffb700 1 -- 172.21.15.39:0/565563620 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7a40b38c0 con 0x55578ffe7c40 2024-05-04T03:26:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc791ffb700 1 -- 172.21.15.39:0/565563620 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40b3430 con 0x55578ffe7c40 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/565563620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 msgr2=0x55578ffff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffff6f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40a6c50 tx=0x7fc7a40aaad0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/565563620 shutdown_connections 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffe7c40 0x55578ffff6f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffdcad0 0x55578ffd8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/565563620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 0x55578ffd8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.612+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/565563620 >> 172.21.15.39:0/565563620 conn(0x55578fe5ff40 msgr2=0x55578ffd7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.613+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/565563620 shutdown_connections 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.613+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/565563620 wait complete. 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.613+0000 7fc7c10f4b80 1 -- start start 2024-05-04T03:26:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.613+0000 7fc7c10f4b80 1 Processor -- start 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.613+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdcad0 0x55578ffedf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.613+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 0x55578ffee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7c10f4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 0x5557900656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1afe90 con 0x55578ffdcad0 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1af7c0 con 0x55578ffe7c40 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7c10f4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1ac9c0 con 0x55578ffdd420 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7c10f4b80 10 client.0 _ll_register_callbacks cb 0x55578fe97a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdcad0 0x55578ffedf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdcad0 0x55578ffedf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39912/0 (socket says 172.21.15.39:39912) 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 -- 172.21.15.39:0/3866928064 learned_addr learned my addr 172.21.15.39:0/3866928064 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7b0c7d700 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 0x5557900656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 0x55578ffee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 -- 172.21.15.39:0/3866928064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 msgr2=0x55578ffee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 0x55578ffee490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 -- 172.21.15.39:0/3866928064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 msgr2=0x5557900656c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 0x5557900656c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 -- 172.21.15.39:0/3866928064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555790065c30 con 0x55578ffdcad0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc7abfff700 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdcad0 0x55578ffedf50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55578ffd9380 tx=0x7fc79c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77f67b33ce79dcd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.614+0000 7fc78b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.615+0000 7fc78b7fe700 1 -- 172.21.15.39:0/3866928064 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c01e840 con 0x55578ffdcad0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.615+0000 7fc78b7fe700 1 -- 172.21.15.39:0/3866928064 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fc79c01e9e0 con 0x55578ffdcad0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.615+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/3866928064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a40b38c0 con 0x55578ffdcad0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.615+0000 7fc78b7fe700 1 -- 172.21.15.39:0/3866928064 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc79c01ebe0 con 0x55578ffdcad0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.615+0000 7fc7c10f4b80 10 client.17085 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.615+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/3866928064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555790066f30 con 0x55578ffdcad0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.615+0000 7fc78b7fe700 1 -- 172.21.15.39:0/3866928064 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc79c02b070 con 0x55578ffdcad0 2024-05-04T03:26:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.616+0000 7fc78b7fe700 1 -- 172.21.15.39:0/3866928064 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc79c03b3f0 con 0x55578ffdcad0 2024-05-04T03:26:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.616+0000 7fc7c10f4b80 10 client.17085 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.616+0000 7fc7c10f4b80 10 client.17085 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:26:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.616+0000 7fc7c10f4b80 20 client.17085 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.616+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555790067620 0x555790069a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.616+0000 7fc7c10f4b80 4 client.17085 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8ffce17a", "release": true, "await": true}] 2024-05-04T03:26:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.616+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/3866928064 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8ffce17a", "release": true, "await": true}) -- 0x7fc7a40b38c0 con 0x555790067620 2024-05-04T03:26:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.616+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555790067620 0x555790069a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:49.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.617+0000 7fc78b7fe700 10 client.17085 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:49.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.617+0000 7fc7ab7fe700 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555790067620 0x555790069a40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd546a598ced438 server_cookie=c2d243dee985d224 in_seq=0 out_seq=0 2024-05-04T03:26:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc78b7fe700 1 -- 172.21.15.39:0/3866928064 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7fc7a40b38c0 con 0x555790067620 2024-05-04T03:26:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc78b7fe700 10 client.17085 handle_command_reply: tid=0 2024-05-04T03:26:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 client.17085 shutdown 2024-05-04T03:26:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/3866928064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdcad0 msgr2=0x55578ffedf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdcad0 0x55578ffedf50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55578ffd9380 tx=0x7fc79c0047c0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/3866928064 shutdown_connections 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x555790067620 0x555790069a40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc78b7fe700 0 client.17085 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55578ffdcad0 0x55578ffedf50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55578ffe7c40 0x5557900656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 --2- 172.21.15.39:0/3866928064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55578ffdd420 0x55578ffee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/3866928064 >> 172.21.15.39:0/3866928064 conn(0x55578fe5ff40 msgr2=0x55578ffd8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/3866928064 shutdown_connections 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 1 -- 172.21.15.39:0/3866928064 wait complete. 2024-05-04T03:26:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.623+0000 7fc7c10f4b80 20 client.17085 trim_cache size 0 max 16384 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 44, 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "8ffce17a": { 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 44, 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:26:49.627 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 85.3, 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:26:49.628 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 -- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7940b2900 msgr2=0x7fc7940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7940b2900 0x7fc7940b4e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc79c005950 tx=0x7fc79c005ed0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 -- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 msgr2=0x7fc7ac19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 0x7fc7ac19ab40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4007d90 tx=0x7fc7a4007dc0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 -- 172.21.15.39:0/2412446079 shutdown_connections 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fc7940b2900 0x7fc7940b4e10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7ac108750 0x7fc7ac19ab40 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7ac1034d0 0x7fc7ac1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 --2- 172.21.15.39:0/2412446079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7ac107e00 0x7fc7ac1a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:49.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 -- 172.21.15.39:0/2412446079 >> 172.21.15.39:0/2412446079 conn(0x7fc7ac0fef20 msgr2=0x7fc7ac101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:49.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 -- 172.21.15.39:0/2412446079 shutdown_connections 2024-05-04T03:26:49.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:26:49.624+0000 7fc7b26e0700 1 -- 172.21.15.39:0/2412446079 wait complete. 2024-05-04T03:26:50.474 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8ffce17a', seconds elapsed: 39.2 2024-05-04T03:26:50.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:50 smithi066 ceph-mon[33264]: pgmap v867: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-04T03:26:50.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:50 smithi167 ceph-mon[33614]: pgmap v867: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-04T03:26:50.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:50 smithi039 ceph-mon[29715]: pgmap v867: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 179 KiB/s wr, 31 op/s 2024-05-04T03:26:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 780.0 s, 546.9 tps, lat 1252.557 ms stddev 6042.501 2024-05-04T03:26:52.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:52 smithi066 ceph-mon[33264]: pgmap v868: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 271 KiB/s wr, 41 op/s 2024-05-04T03:26:52.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:52 smithi167 ceph-mon[33614]: pgmap v868: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 271 KiB/s wr, 41 op/s 2024-05-04T03:26:52.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:52 smithi039 ceph-mon[29715]: pgmap v868: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 271 KiB/s wr, 41 op/s 2024-05-04T03:26:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:54 smithi066 ceph-mon[33264]: pgmap v869: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 175 op/s 2024-05-04T03:26:54.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:54 smithi167 ceph-mon[33614]: pgmap v869: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 175 op/s 2024-05-04T03:26:54.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:54 smithi039 ceph-mon[29715]: pgmap v869: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 175 op/s 2024-05-04T03:26:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 785.0 s, 1327.4 tps, lat 75.266 ms stddev 25.339 2024-05-04T03:26:56.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:56 smithi066 ceph-mon[33264]: pgmap v870: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 297 op/s 2024-05-04T03:26:56.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:56 smithi167 ceph-mon[33614]: pgmap v870: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 297 op/s 2024-05-04T03:26:56.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:56 smithi039 ceph-mon[29715]: pgmap v870: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 297 op/s 2024-05-04T03:26:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 -- 172.21.15.39:0/976235558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8370102b80 msgr2=0x7f8370103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 --2- 172.21.15.39:0/976235558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8370102b80 0x7f8370103880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8368009ca0 tx=0x7f836802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 -- 172.21.15.39:0/976235558 shutdown_connections 2024-05-04T03:26:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 --2- 172.21.15.39:0/976235558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370108ac0 0x7f837010aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 --2- 172.21.15.39:0/976235558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8370103dc0 0x7f83701041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 --2- 172.21.15.39:0/976235558 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8370102b80 0x7f8370103880 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 -- 172.21.15.39:0/976235558 >> 172.21.15.39:0/976235558 conn(0x7f83700fdd90 msgr2=0x7f8370100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 -- 172.21.15.39:0/976235558 shutdown_connections 2024-05-04T03:26:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.257+0000 7f8377d08640 1 -- 172.21.15.39:0/976235558 wait complete. 2024-05-04T03:26:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.258+0000 7f8377d08640 1 Processor -- start 2024-05-04T03:26:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8377d08640 1 -- start start 2024-05-04T03:26:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8377d08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370102b80 0x7f8370197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8375aa4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370102b80 0x7f8370197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8375aa4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370102b80 0x7f8370197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35164/0 (socket says 172.21.15.39:35164) 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8375aa4640 1 -- 172.21.15.39:0/1637163237 learned_addr learned my addr 172.21.15.39:0/1637163237 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8377d08640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8370103dc0 0x7f83701979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8377d08640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8370108ac0 0x7f8370193fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f837010e560 con 0x7f8370102b80 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f837010e3e0 con 0x7f8370108ac0 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.259+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f837010e6e0 con 0x7f8370103dc0 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.260+0000 7f8375aa4640 1 -- 172.21.15.39:0/1637163237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8370103dc0 msgr2=0x7f83701979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.260+0000 7f8375aa4640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8370103dc0 0x7f83701979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.260+0000 7f8375aa4640 1 -- 172.21.15.39:0/1637163237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8370108ac0 msgr2=0x7f8370193fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-04T03:26:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.260+0000 7f8375aa4640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8370108ac0 0x7f8370193fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.260+0000 7f8375aa4640 1 -- 172.21.15.39:0/1637163237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83701947e0 con 0x7f8370102b80 2024-05-04T03:26:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.260+0000 7f8375aa4640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370102b80 0x7f8370197470 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f836802aeb0 tx=0x7f8368003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.261+0000 7f8366ffd640 1 -- 172.21.15.39:0/1637163237 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368004240 con 0x7f8370102b80 2024-05-04T03:26:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.261+0000 7f8366ffd640 1 -- 172.21.15.39:0/1637163237 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8368003e10 con 0x7f8370102b80 2024-05-04T03:26:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.261+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8370194a70 con 0x7f8370102b80 2024-05-04T03:26:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.261+0000 7f8366ffd640 1 -- 172.21.15.39:0/1637163237 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836803b410 con 0x7f8370102b80 2024-05-04T03:26:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.262+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83701a8c30 con 0x7f8370102b80 2024-05-04T03:26:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f837ebb7740 1 Processor -- start 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f837ebb7740 1 -- start start 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f8366ffd640 1 -- 172.21.15.39:0/1637163237 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f83680305c0 con 0x7f8370102b80 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda6f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda6f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda6fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f837ebb7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563eda65b5d0 con 0x563eda32c2f0 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f837ebb7740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563eda614890 con 0x563eda3344d0 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.266+0000 7f837ebb7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563eda657590 con 0x563eda3551b0 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83762a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda6f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83762a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda6f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:60616/0 (socket says 172.21.15.39:60616) 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83762a5640 1 -- 172.21.15.39:0/1239834709 learned_addr learned my addr 172.21.15.39:0/1239834709 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83752a3640 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda6fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f8375aa4640 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda6f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83762a5640 1 -- 172.21.15.39:0/1239834709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 msgr2=0x563eda6fdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83762a5640 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda6fdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83762a5640 1 -- 172.21.15.39:0/1239834709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 msgr2=0x563eda6f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83762a5640 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda6f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.267+0000 7f83762a5640 1 -- 172.21.15.39:0/1239834709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eda36d210 con 0x563eda3551b0 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.268+0000 7f8366ffd640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f835c0b28e0 0x7f835c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.268+0000 7f8375aa4640 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda6f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.268+0000 7f8375aa4640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f835c0b28e0 0x7f835c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.268+0000 7f8366ffd640 1 -- 172.21.15.39:0/1637163237 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8368103b00 con 0x7f8370102b80 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.268+0000 7f83762a5640 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda6f6a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8360009b00 tx=0x7f836000e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=686224ce65360282 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.269+0000 7f8375aa4640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f835c0b28e0 0x7f835c0b4df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f83680c8340 tx=0x7f8368107410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.269+0000 7f835bfff640 1 -- 172.21.15.39:0/1239834709 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836000ed00 con 0x563eda3551b0 2024-05-04T03:26:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.269+0000 7f835bfff640 1 -- 172.21.15.39:0/1239834709 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8360005020 con 0x563eda3551b0 2024-05-04T03:26:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.269+0000 7f835bfff640 1 -- 172.21.15.39:0/1239834709 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360004a00 con 0x563eda3551b0 2024-05-04T03:26:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 -- 172.21.15.39:0/1239834709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 msgr2=0x563eda6f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda6f6a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8360009b00 tx=0x7f836000e350 comp rx=0 tx=0).stop 2024-05-04T03:26:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 -- 172.21.15.39:0/1239834709 shutdown_connections 2024-05-04T03:26:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda6fdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda6f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1239834709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda6f6a60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 -- 172.21.15.39:0/1239834709 >> 172.21.15.39:0/1239834709 conn(0x563eda349300 msgr2=0x563eda6f23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 -- 172.21.15.39:0/1239834709 shutdown_connections 2024-05-04T03:26:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.270+0000 7f837ebb7740 1 -- 172.21.15.39:0/1239834709 wait complete. 2024-05-04T03:26:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.271+0000 7f837ebb7740 1 -- start start 2024-05-04T03:26:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.271+0000 7f837ebb7740 1 Processor -- start 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda32c2f0 0x563eda7006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 0x563eda700be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda32c2f0 0x563eda7006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda707b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda32c2f0 0x563eda7006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:47900/0 (socket says 172.21.15.39:47900) 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f837ebb7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563eda6fe930 con 0x563eda3344d0 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 -- 172.21.15.39:0/4213539796 learned_addr learned my addr 172.21.15.39:0/4213539796 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x563eda618370 con 0x563eda32c2f0 2024-05-04T03:26:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563eda6feab0 con 0x563eda3551b0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83752a3640 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 0x563eda700be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda707b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f837ebb7740 10 client.0 _ll_register_callbacks cb 0x563eda36ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 -- 172.21.15.39:0/4213539796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 msgr2=0x563eda707b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda707b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 -- 172.21.15.39:0/4213539796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 msgr2=0x563eda700be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 0x563eda700be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f83762a5640 1 -- 172.21.15.39:0/4213539796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eda439620 con 0x563eda32c2f0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.272+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8360005020 con 0x563eda32c2f0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.273+0000 7f83762a5640 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda32c2f0 0x563eda7006a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8360000c00 tx=0x7f83600097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc7cc528d530f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.273+0000 7f83597fa640 10 client.26648 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.273+0000 7f83597fa640 1 -- 172.21.15.39:0/4213539796 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360004b90 con 0x563eda32c2f0 2024-05-04T03:26:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.273+0000 7f83597fa640 1 -- 172.21.15.39:0/4213539796 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8360004d30 con 0x563eda32c2f0 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.273+0000 7f83597fa640 1 -- 172.21.15.39:0/4213539796 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8360010430 con 0x563eda32c2f0 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.273+0000 7f837ebb7740 10 client.26648 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.273+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=32}) -- 0x563eda70a9a0 con 0x563eda32c2f0 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.274+0000 7f83597fa640 1 -- 172.21.15.39:0/4213539796 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f83600105d0 con 0x563eda32c2f0 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.274+0000 7f837ebb7740 10 client.26648 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.274+0000 7f837ebb7740 10 client.26648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.274+0000 7f837ebb7740 20 client.26648 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.274+0000 7f837ebb7740 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563eda313d80 0x563eda70dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.274+0000 7f837ebb7740 4 client.26648 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.274+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eda6691e0 con 0x563eda313d80 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.274+0000 7f83752a3640 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563eda313d80 0x563eda70dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.275+0000 7f83752a3640 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563eda313d80 0x563eda70dd00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e18b7b7d929aa4e server_cookie=94ae2870990ba612 in_seq=0 out_seq=0 2024-05-04T03:26:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.275+0000 7f83597fa640 10 client.26648 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.275+0000 7f83597fa640 1 -- 172.21.15.39:0/4213539796 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563eda6691e0 con 0x563eda313d80 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.275+0000 7f83597fa640 10 client.26648 handle_command_reply: tid=0 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.275+0000 7f837ebb7740 1 client.26648 shutdown 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda32c2f0 msgr2=0x563eda7006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda32c2f0 0x563eda7006a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8360000c00 tx=0x7f83600097f0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 shutdown_connections 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563eda313d80 0x563eda70dd00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f83597fa640 0 client.26648 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda3551b0 0x563eda707b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 0x563eda700be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 --2- 172.21.15.39:0/4213539796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda32c2f0 0x563eda7006a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 >> 172.21.15.39:0/4213539796 conn(0x563eda349300 msgr2=0x563eda6f13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 shutdown_connections 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 1 -- 172.21.15.39:0/4213539796 wait complete. 2024-05-04T03:26:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.276+0000 7f837ebb7740 20 client.26648 trim_cache size 0 max 16384 2024-05-04T03:26:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.306+0000 7f837ebb7740 1 Processor -- start 2024-05-04T03:26:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.306+0000 7f837ebb7740 1 -- start start 2024-05-04T03:26:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.307+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda313d80 0x563eda6ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.307+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda707ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.307+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda708010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.307+0000 7f83752a3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda707ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.307+0000 7f83762a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda313d80 0x563eda6ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.307+0000 7f83752a3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda707ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35188/0 (socket says 172.21.15.39:35188) 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.307+0000 7f83762a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda313d80 0x563eda6ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:60638/0 (socket says 172.21.15.39:60638) 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.307+0000 7f83752a3640 1 -- 172.21.15.39:0/2347912590 learned_addr learned my addr 172.21.15.39:0/2347912590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f837ebb7740 1 -- 172.21.15.39:0/2347912590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b1a30 con 0x563eda32c2f0 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f837ebb7740 1 -- 172.21.15.39:0/2347912590 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b3570 con 0x563eda3344d0 2024-05-04T03:26:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f837ebb7740 1 -- 172.21.15.39:0/2347912590 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b2100 con 0x563eda313d80 2024-05-04T03:26:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f83752a3640 1 -- 172.21.15.39:0/2347912590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda313d80 msgr2=0x563eda6ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f83762a5640 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda708010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f83752a3640 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda313d80 0x563eda6ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f83752a3640 1 -- 172.21.15.39:0/2347912590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 msgr2=0x563eda708010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f83752a3640 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda708010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.308+0000 7f83752a3640 1 -- 172.21.15.39:0/2347912590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eda701710 con 0x563eda32c2f0 2024-05-04T03:26:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.309+0000 7f83762a5640 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda708010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.309+0000 7f83752a3640 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda707ad0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f836c000c00 tx=0x7f836c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503ecf8bd65dfb61 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.309+0000 7f835b7fe640 1 -- 172.21.15.39:0/2347912590 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c003d20 con 0x563eda32c2f0 2024-05-04T03:26:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.310+0000 7f835b7fe640 1 -- 172.21.15.39:0/2347912590 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f836c021070 con 0x563eda32c2f0 2024-05-04T03:26:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.310+0000 7f835b7fe640 1 -- 172.21.15.39:0/2347912590 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c010850 con 0x563eda32c2f0 2024-05-04T03:26:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.311+0000 7f837ebb7740 1 -- 172.21.15.39:0/2347912590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 msgr2=0x563eda707ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.311+0000 7f837ebb7740 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda707ad0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f836c000c00 tx=0x7f836c004410 comp rx=0 tx=0).stop 2024-05-04T03:26:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.311+0000 7f837ebb7740 1 -- 172.21.15.39:0/2347912590 shutdown_connections 2024-05-04T03:26:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.311+0000 7f837ebb7740 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda3344d0 0x563eda708010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.311+0000 7f837ebb7740 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda32c2f0 0x563eda707ad0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.311+0000 7f837ebb7740 1 --2- 172.21.15.39:0/2347912590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda313d80 0x563eda6ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.311+0000 7f837ebb7740 1 -- 172.21.15.39:0/2347912590 >> 172.21.15.39:0/2347912590 conn(0x563eda349300 msgr2=0x563eda6f13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.311+0000 7f837ebb7740 1 -- 172.21.15.39:0/2347912590 shutdown_connections 2024-05-04T03:26:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.312+0000 7f837ebb7740 1 -- 172.21.15.39:0/2347912590 wait complete. 2024-05-04T03:26:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.313+0000 7f837ebb7740 1 -- start start 2024-05-04T03:26:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.313+0000 7f837ebb7740 1 Processor -- start 2024-05-04T03:26:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.314+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda313d80 0x563eda705200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.314+0000 7f837ebb7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda32c2f0 0x563eda705740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.314+0000 7f83762a5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda313d80 0x563eda705200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f83762a5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda313d80 0x563eda705200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:47916/0 (socket says 172.21.15.39:47916) 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f83762a5640 1 -- 172.21.15.39:0/1162893130 learned_addr learned my addr 172.21.15.39:0/1162893130 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 0x563eda706590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b2ea0 con 0x563eda3344d0 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b27d0 con 0x563eda313d80 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f83752a3640 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda32c2f0 0x563eda705740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83701afa10 con 0x563eda32c2f0 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f837ebb7740 10 client.0 _ll_register_callbacks cb 0x7f836810b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.315+0000 7f83762a5640 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 0x563eda706590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.316+0000 7f83762a5640 1 -- 172.21.15.39:0/1162893130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda32c2f0 msgr2=0x563eda705740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.316+0000 7f83762a5640 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda32c2f0 0x563eda705740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.316+0000 7f83762a5640 1 -- 172.21.15.39:0/1162893130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 msgr2=0x563eda706590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:26:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.316+0000 7f83762a5640 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 0x563eda706590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.316+0000 7f83762a5640 1 -- 172.21.15.39:0/1162893130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eda706b50 con 0x563eda313d80 2024-05-04T03:26:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.316+0000 7f83762a5640 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda313d80 0x563eda705200 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8360009230 tx=0x7f83600097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da386e2474aff461 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.316+0000 7f83597fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.316+0000 7f83597fa640 1 -- 172.21.15.39:0/1162893130 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360009470 con 0x563eda313d80 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.317+0000 7f83597fa640 1 -- 172.21.15.39:0/1162893130 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8360004ba0 con 0x563eda313d80 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.317+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836c021070 con 0x563eda313d80 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.317+0000 7f83597fa640 1 -- 172.21.15.39:0/1162893130 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f836000ee70 con 0x563eda313d80 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.317+0000 7f837ebb7740 10 client.26651 fetch_fsmap learned FSMap version 32 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.317+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563eda707d80 con 0x563eda313d80 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.318+0000 7f83597fa640 1 -- 172.21.15.39:0/1162893130 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8360005020 con 0x563eda313d80 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.318+0000 7f83597fa640 1 -- 172.21.15.39:0/1162893130 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8360003790 con 0x563eda313d80 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.318+0000 7f837ebb7740 10 client.26651 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.318+0000 7f837ebb7740 10 client.26651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.318+0000 7f837ebb7740 20 client.26651 populate_metadata read hostname 'smithi039' 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.318+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563eda3551b0 0x563eda778080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.318+0000 7f837ebb7740 4 client.26651 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:26:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.319+0000 7f83752a3640 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563eda3551b0 0x563eda778080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.319+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f836c021070 con 0x563eda3551b0 2024-05-04T03:26:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.319+0000 7f83752a3640 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563eda3551b0 0x563eda778080 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d5992b8922f2da server_cookie=6612326cd111c9ce in_seq=0 out_seq=0 2024-05-04T03:26:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.319+0000 7f83597fa640 10 client.26651 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.328+0000 7f83597fa640 1 -- 172.21.15.39:0/1162893130 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f836c021070 con 0x563eda3551b0 2024-05-04T03:26:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.328+0000 7f83597fa640 10 client.26651 handle_command_reply: tid=0 2024-05-04T03:26:57.330 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.328+0000 7f837ebb7740 1 client.26651 shutdown 2024-05-04T03:26:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.329+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda313d80 msgr2=0x563eda705200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.329+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda313d80 0x563eda705200 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8360009230 tx=0x7f83600097f0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.330+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 shutdown_connections 2024-05-04T03:26:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.330+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x563eda3551b0 0x563eda778080 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.330+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563eda3344d0 0x563eda706590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.330+0000 7f83597fa640 0 client.26651 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:26:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.330+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563eda32c2f0 0x563eda705740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.330+0000 7f837ebb7740 1 --2- 172.21.15.39:0/1162893130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563eda313d80 0x563eda705200 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.330+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 >> 172.21.15.39:0/1162893130 conn(0x563eda349300 msgr2=0x563eda6f13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.330+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 shutdown_connections 2024-05-04T03:26:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.331+0000 7f837ebb7740 1 -- 172.21.15.39:0/1162893130 wait complete. 2024-05-04T03:26:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.331+0000 7f837ebb7740 20 client.26651 trim_cache size 0 max 16384 2024-05-04T03:26:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:26:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:26:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:26:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:26:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f835c0b28e0 msgr2=0x7f835c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f835c0b28e0 0x7f835c0b4df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f83680c8340 tx=0x7f8368107410 comp rx=0 tx=0).stop 2024-05-04T03:26:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370102b80 msgr2=0x7f8370197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370102b80 0x7f8370197470 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f836802aeb0 tx=0x7f8368003c20 comp rx=0 tx=0).stop 2024-05-04T03:26:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 shutdown_connections 2024-05-04T03:26:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f835c0b28e0 0x7f835c0b4df0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8370108ac0 0x7f8370193fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8370103dc0 0x7f83701979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 --2- 172.21.15.39:0/1637163237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8370102b80 0x7f8370197470 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.333+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 >> 172.21.15.39:0/1637163237 conn(0x7f83700fdd90 msgr2=0x7f83701001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.334+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 shutdown_connections 2024-05-04T03:26:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:26:57.334+0000 7f8377d08640 1 -- 172.21.15.39:0/1637163237 wait complete. 2024-05-04T03:26:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:26:58 smithi066 ceph-mon[33264]: pgmap v871: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.5 MiB/s wr, 564 op/s 2024-05-04T03:26:58.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:26:58 smithi167 ceph-mon[33614]: pgmap v871: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.5 MiB/s wr, 564 op/s 2024-05-04T03:26:58.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:26:58 smithi039 ceph-mon[29715]: pgmap v871: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 9.5 MiB/s wr, 564 op/s 2024-05-04T03:27:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:00 smithi066 ceph-mon[33264]: pgmap v872: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 806 op/s 2024-05-04T03:27:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:00 smithi167 ceph-mon[33614]: pgmap v872: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 806 op/s 2024-05-04T03:27:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:00 smithi039 ceph-mon[29715]: pgmap v872: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 806 op/s 2024-05-04T03:27:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 790.0 s, 1313.6 tps, lat 76.104 ms stddev 27.544 2024-05-04T03:27:02.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:02 smithi066 ceph-mon[33264]: pgmap v873: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 15 MiB/s wr, 982 op/s 2024-05-04T03:27:02.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:02 smithi167 ceph-mon[33614]: pgmap v873: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 15 MiB/s wr, 982 op/s 2024-05-04T03:27:02.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:02 smithi039 ceph-mon[29715]: pgmap v873: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 15 MiB/s wr, 982 op/s 2024-05-04T03:27:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:04 smithi066 ceph-mon[33264]: pgmap v874: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 19 MiB/s wr, 1.25k op/s 2024-05-04T03:27:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:04 smithi167 ceph-mon[33614]: pgmap v874: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 19 MiB/s wr, 1.25k op/s 2024-05-04T03:27:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:04 smithi039 ceph-mon[29715]: pgmap v874: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 19 MiB/s wr, 1.25k op/s 2024-05-04T03:27:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 795.0 s, 1396.4 tps, lat 71.521 ms stddev 21.348 2024-05-04T03:27:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:06 smithi066 ceph-mon[33264]: pgmap v875: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 1.28k op/s 2024-05-04T03:27:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:06 smithi167 ceph-mon[33614]: pgmap v875: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 1.28k op/s 2024-05-04T03:27:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:06 smithi039 ceph-mon[29715]: pgmap v875: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 1.28k op/s 2024-05-04T03:27:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:07 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:27:07.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:07 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:27:08.068 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:07 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:27:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: pgmap v876: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 21 MiB/s wr, 1.42k op/s 2024-05-04T03:27:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:08 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: pgmap v876: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 21 MiB/s wr, 1.42k op/s 2024-05-04T03:27:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:08.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:08 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: pgmap v876: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 21 MiB/s wr, 1.42k op/s 2024-05-04T03:27:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.070 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.070 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:08 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:09 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-05-04T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:09 smithi167 ceph-mon[33614]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-05-04T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:09 smithi167 ceph-mon[33614]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-04T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:09 smithi167 ceph-mon[33614]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-04T03:27:09.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:27:09.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:27:09.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:27:09.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:09 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-05-04T03:27:09.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:09 smithi039 ceph-mon[29715]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-05-04T03:27:09.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:09 smithi039 ceph-mon[29715]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-04T03:27:09.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:09 smithi039 ceph-mon[29715]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-04T03:27:09.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:27:09.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:27:09.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:27:10.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:09 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-05-04T03:27:10.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:09 smithi066 ceph-mon[33264]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-05-04T03:27:10.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:09 smithi066 ceph-mon[33264]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-04T03:27:10.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:09 smithi066 ceph-mon[33264]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-04T03:27:10.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:27:10.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:27:10.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:27:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.487+0000 7f4fa2d46640 1 -- 172.21.15.39:0/1260387895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c1032e0 msgr2=0x7f4f9c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.487+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/1260387895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c1032e0 0x7f4f9c1036f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00fc00 tx=0x7f4f8c02e690 comp rx=0 tx=0).stop 2024-05-04T03:27:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.488+0000 7f4fa2d46640 1 -- 172.21.15.39:0/1260387895 shutdown_connections 2024-05-04T03:27:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.488+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/1260387895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f9c107fe0 0x7f4f9c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.488+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/1260387895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c1032e0 0x7f4f9c1036f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.488+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/1260387895 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9c1022a0 0x7f4f9c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.488+0000 7f4fa2d46640 1 -- 172.21.15.39:0/1260387895 >> 172.21.15.39:0/1260387895 conn(0x7f4f9c0fcea0 msgr2=0x7f4f9c0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.488+0000 7f4fa2d46640 1 -- 172.21.15.39:0/1260387895 shutdown_connections 2024-05-04T03:27:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.488+0000 7f4fa2d46640 1 -- 172.21.15.39:0/1260387895 wait complete. 2024-05-04T03:27:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.489+0000 7f4fa2d46640 1 Processor -- start 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.489+0000 7f4fa2d46640 1 -- start start 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.489+0000 7f4fa2d46640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9c1022a0 0x7f4f9c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa2d46640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c1032e0 0x7f4f9c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa2d46640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f9c107fe0 0x7f4f9c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa2d46640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10d260 con 0x7f4f9c107fe0 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa2d46640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10d0e0 con 0x7f4f9c1022a0 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa2d46640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c10d3e0 con 0x7f4f9c1032e0 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1d44640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9c1022a0 0x7f4f9c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1d44640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9c1022a0 0x7f4f9c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:52578/0 (socket says 172.21.15.39:52578) 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1d44640 1 -- 172.21.15.39:0/3729297324 learned_addr learned my addr 172.21.15.39:0/3729297324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1543640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c1032e0 0x7f4f9c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa2545640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f9c107fe0 0x7f4f9c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1d44640 1 -- 172.21.15.39:0/3729297324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c1032e0 msgr2=0x7f4f9c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c1032e0 0x7f4f9c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1d44640 1 -- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f9c107fe0 msgr2=0x7f4f9c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f9c107fe0 0x7f4f9c19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.490+0000 7f4fa1d44640 1 -- 172.21.15.39:0/3729297324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c19fa10 con 0x7f4f9c1022a0 2024-05-04T03:27:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.491+0000 7f4fa2545640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f9c107fe0 0x7f4f9c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:27:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.491+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9c1022a0 0x7f4f9c197ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4f98007860 tx=0x7f4f98007c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.491+0000 7f4f92ffd640 1 -- 172.21.15.39:0/3729297324 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98019030 con 0x7f4f9c1022a0 2024-05-04T03:27:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.491+0000 7f4f92ffd640 1 -- 172.21.15.39:0/3729297324 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f9800f8c0 con 0x7f4f9c1022a0 2024-05-04T03:27:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.491+0000 7f4fa2d46640 1 -- 172.21.15.39:0/3729297324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9c19fd00 con 0x7f4f9c1022a0 2024-05-04T03:27:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.491+0000 7f4f92ffd640 1 -- 172.21.15.39:0/3729297324 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98003850 con 0x7f4f9c1022a0 2024-05-04T03:27:10.495 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.491+0000 7f4fa2d46640 1 -- 172.21.15.39:0/3729297324 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9c1a01e0 con 0x7f4f9c1022a0 2024-05-04T03:27:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.496+0000 7f4f92ffd640 1 -- 172.21.15.39:0/3729297324 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4f9801c070 con 0x7f4f9c1022a0 2024-05-04T03:27:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.496+0000 7f4faae5c740 1 Processor -- start 2024-05-04T03:27:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.496+0000 7f4faae5c740 1 -- start start 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.497+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a362046c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.497+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a3620471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.497+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a36204e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.497+0000 7f4faae5c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a361fab5d0 con 0x55a361c7c2f0 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.497+0000 7f4f92ffd640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4f880b28e0 0x7f4f880b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.497+0000 7f4faae5c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a361f64890 con 0x55a361c844d0 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.497+0000 7f4faae5c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a361fa7590 con 0x55a361ca51b0 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa2545640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a362046c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa2545640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a362046c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:44690/0 (socket says 172.21.15.39:44690) 2024-05-04T03:27:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa2545640 1 -- 172.21.15.39:0/3869332885 learned_addr learned my addr 172.21.15.39:0/3869332885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a3620471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa2545640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4f880b28e0 0x7f4f880b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa1543640 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a36204e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa1543640 1 -- 172.21.15.39:0/3869332885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 msgr2=0x55a362046c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa1543640 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a362046c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa1543640 1 -- 172.21.15.39:0/3869332885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 msgr2=0x55a3620471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4f92ffd640 1 -- 172.21.15.39:0/3729297324 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4f980e1b80 con 0x7f4f9c1022a0 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.498+0000 7f4fa1543640 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a3620471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.499+0000 7f4fa1543640 1 -- 172.21.15.39:0/3869332885 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a361cbd210 con 0x55a361c844d0 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.499+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a3620471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.499+0000 7f4fa2545640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4f880b28e0 0x7f4f880b4df0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4f94011dd0 tx=0x7f4f940114a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.499+0000 7f4fa1543640 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a36204e0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0057c0 tx=0x7f4f8c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc6d8060005c10d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.499+0000 7f4f87fff640 1 -- 172.21.15.39:0/3869332885 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c039030 con 0x55a361c844d0 2024-05-04T03:27:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.500+0000 7f4f87fff640 1 -- 172.21.15.39:0/3869332885 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f8c045070 con 0x55a361c844d0 2024-05-04T03:27:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.500+0000 7f4f87fff640 1 -- 172.21.15.39:0/3869332885 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c041c20 con 0x55a361c844d0 2024-05-04T03:27:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 -- 172.21.15.39:0/3869332885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 msgr2=0x55a36204e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a36204e0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0057c0 tx=0x7f4f8c005fd0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 -- 172.21.15.39:0/3869332885 shutdown_connections 2024-05-04T03:27:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a36204e0b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a3620471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3869332885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a362046c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 -- 172.21.15.39:0/3869332885 >> 172.21.15.39:0/3869332885 conn(0x55a361c99300 msgr2=0x55a3620425c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 -- 172.21.15.39:0/3869332885 shutdown_connections 2024-05-04T03:27:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.501+0000 7f4faae5c740 1 -- 172.21.15.39:0/3869332885 wait complete. 2024-05-04T03:27:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.502+0000 7f4faae5c740 1 -- start start 2024-05-04T03:27:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.502+0000 7f4faae5c740 1 Processor -- start 2024-05-04T03:27:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a362059020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a362059560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4faae5c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a36204eaa0 con 0x55a361c7c2f0 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4faae5c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55a361f68370 con 0x55a361c844d0 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4faae5c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a36204ec20 con 0x55a361ca51b0 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4faae5c740 10 client.0 _ll_register_callbacks cb 0x55a361cbca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4fa2545640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4fa2545640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:44704/0 (socket says 172.21.15.39:44704) 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4fa2545640 1 -- 172.21.15.39:0/3590137275 learned_addr learned my addr 172.21.15.39:0/3590137275 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4fa1543640 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a362059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.503+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a362059020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4fa2545640 1 -- 172.21.15.39:0/3590137275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 msgr2=0x55a362059560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4fa2545640 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a362059560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4fa2545640 1 -- 172.21.15.39:0/3590137275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 msgr2=0x55a362059020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4fa2545640 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a362059020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4fa2545640 1 -- 172.21.15.39:0/3590137275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a361d89620 con 0x55a361ca51b0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4fa2545640 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620604c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f94003ba0 tx=0x7f4f94009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de62143178b29f68 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4f857fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4f857fa640 1 -- 172.21.15.39:0/3590137275 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9402b060 con 0x55a361ca51b0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4f857fa640 1 -- 172.21.15.39:0/3590137275 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4f9401a040 con 0x55a361ca51b0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.504+0000 7f4faae5c740 1 -- 172.21.15.39:0/3590137275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c045070 con 0x55a361ca51b0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.505+0000 7f4f857fa640 1 -- 172.21.15.39:0/3590137275 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4f940047d0 con 0x55a361ca51b0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.505+0000 7f4faae5c740 10 client.26692 fetch_fsmap learned FSMap version 32 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.505+0000 7f4faae5c740 1 -- 172.21.15.39:0/3590137275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a362063460 con 0x55a361ca51b0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.505+0000 7f4f857fa640 1 -- 172.21.15.39:0/3590137275 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f8c045070 con 0x55a361ca51b0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.505+0000 7f4f857fa640 1 -- 172.21.15.39:0/3590137275 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4f94026410 con 0x55a361ca51b0 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4faae5c740 10 client.26692 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:27:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4faae5c740 10 client.26692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:27:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4faae5c740 20 client.26692 populate_metadata read hostname 'smithi039' 2024-05-04T03:27:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a361c63d80 0x55a362066b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4faae5c740 4 client.26692 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:27:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4faae5c740 1 -- 172.21.15.39:0/3590137275 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a361fb91e0 con 0x55a361c63d80 2024-05-04T03:27:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a361c63d80 0x55a362066b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4f857fa640 10 client.26692 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.506+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a361c63d80 0x55a362066b30 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e0738115fd24f4 server_cookie=b9c871bec62869f8 in_seq=0 out_seq=0 2024-05-04T03:27:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.507+0000 7f4f857fa640 1 -- 172.21.15.39:0/3590137275 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a361fb91e0 con 0x55a361c63d80 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.507+0000 7f4f857fa640 10 client.26692 handle_command_reply: tid=0 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 client.26692 shutdown 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 -- 172.21.15.39:0/3590137275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 msgr2=0x55a3620604c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620604c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4f94003ba0 tx=0x7f4f94009920 comp rx=0 tx=0).stop 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 -- 172.21.15.39:0/3590137275 shutdown_connections 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a361c63d80 0x55a362066b30 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620604c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c844d0 0x55a362059560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 --2- 172.21.15.39:0/3590137275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c7c2f0 0x55a362059020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 -- 172.21.15.39:0/3590137275 >> 172.21.15.39:0/3590137275 conn(0x55a361c99300 msgr2=0x55a3620454d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4f857fa640 0 client.26692 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 -- 172.21.15.39:0/3590137275 shutdown_connections 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.508+0000 7f4faae5c740 1 -- 172.21.15.39:0/3590137275 wait complete. 2024-05-04T03:27:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.509+0000 7f4faae5c740 20 client.26692 trim_cache size 0 max 16384 2024-05-04T03:27:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.537+0000 7f4faae5c740 1 Processor -- start 2024-05-04T03:27:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.537+0000 7f4faae5c740 1 -- start start 2024-05-04T03:27:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.537+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a362041f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.537+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c844d0 0x55a3620424c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.537+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.537+0000 7f4faae5c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1a8fb0 con 0x55a361c844d0 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.537+0000 7f4faae5c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1aaaf0 con 0x55a361c7c2f0 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.537+0000 7f4faae5c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1a9680 con 0x55a361ca51b0 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1543640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c844d0 0x55a3620424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1543640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c844d0 0x55a3620424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35140/0 (socket says 172.21.15.39:35140) 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa2545640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1543640 1 -- 172.21.15.39:0/2290707455 learned_addr learned my addr 172.21.15.39:0/2290707455 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/2290707455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a362041f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1d44640 1 -- 172.21.15.39:0/2290707455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 msgr2=0x55a3620603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/2290707455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1d44640 1 -- 172.21.15.39:0/2290707455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c844d0 msgr2=0x55a3620424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/2290707455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c844d0 0x55a3620424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1d44640 1 -- 172.21.15.39:0/2290707455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a361c63fb0 con 0x55a361c7c2f0 2024-05-04T03:27:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.538+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/2290707455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a362041f80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4f980a90e0 tx=0x7f4f9800b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536f2443fdb0a3d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.539+0000 7f4f877fe640 1 -- 172.21.15.39:0/2290707455 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980df740 con 0x55a361c7c2f0 2024-05-04T03:27:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.539+0000 7f4f877fe640 1 -- 172.21.15.39:0/2290707455 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f9800fd70 con 0x55a361c7c2f0 2024-05-04T03:27:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.539+0000 7f4f877fe640 1 -- 172.21.15.39:0/2290707455 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980dfaf0 con 0x55a361c7c2f0 2024-05-04T03:27:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 -- 172.21.15.39:0/2290707455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 msgr2=0x55a362041f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 --2- 172.21.15.39:0/2290707455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a362041f80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4f980a90e0 tx=0x7f4f9800b3f0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 -- 172.21.15.39:0/2290707455 shutdown_connections 2024-05-04T03:27:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 --2- 172.21.15.39:0/2290707455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361ca51b0 0x55a3620603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 --2- 172.21.15.39:0/2290707455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361c844d0 0x55a3620424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 --2- 172.21.15.39:0/2290707455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a362041f80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 -- 172.21.15.39:0/2290707455 >> 172.21.15.39:0/2290707455 conn(0x55a361c99300 msgr2=0x55a36204acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 -- 172.21.15.39:0/2290707455 shutdown_connections 2024-05-04T03:27:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.540+0000 7f4faae5c740 1 -- 172.21.15.39:0/2290707455 wait complete. 2024-05-04T03:27:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.541+0000 7f4faae5c740 1 -- start start 2024-05-04T03:27:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.541+0000 7f4faae5c740 1 Processor -- start 2024-05-04T03:27:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a36205d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4faae5c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361c844d0 0x55a36205de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4fa1d44640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a36205d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4fa1d44640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a36205d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:52616/0 (socket says 172.21.15.39:52616) 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4fa1d44640 1 -- 172.21.15.39:0/549239882 learned_addr learned my addr 172.21.15.39:0/549239882 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4faae5c740 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361ca51b0 0x55a3620ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4fa1543640 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361c844d0 0x55a36205de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1aa420 con 0x55a361ca51b0 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1a9d50 con 0x55a361c7c2f0 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1a6f90 con 0x55a361c844d0 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4fa2545640 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361ca51b0 0x55a3620ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.542+0000 7f4faae5c740 10 client.0 _ll_register_callbacks cb 0x7f4f94011ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.543+0000 7f4fa1543640 1 -- 172.21.15.39:0/549239882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 msgr2=0x55a36205d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.543+0000 7f4fa1543640 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a36205d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.543+0000 7f4fa1543640 1 -- 172.21.15.39:0/549239882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361ca51b0 msgr2=0x55a3620ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.543+0000 7f4fa1543640 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361ca51b0 0x55a3620ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.543+0000 7f4fa1543640 1 -- 172.21.15.39:0/549239882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a362060db0 con 0x55a361c844d0 2024-05-04T03:27:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.543+0000 7f4fa1543640 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361c844d0 0x55a36205de60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c02ec80 tx=0x7f4f8c00f4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f071f037cedfe0f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.543+0000 7f4f857fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.544+0000 7f4f857fa640 1 -- 172.21.15.39:0/549239882 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c039040 con 0x55a361c844d0 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.544+0000 7f4f857fa640 1 -- 172.21.15.39:0/549239882 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4f8c004220 con 0x55a361c844d0 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.544+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f9800fd70 con 0x55a361c844d0 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.545+0000 7f4f857fa640 1 -- 172.21.15.39:0/549239882 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4f8c004420 con 0x55a361c844d0 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.545+0000 7f4faae5c740 10 client.26698 fetch_fsmap learned FSMap version 32 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.545+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3620ce2b0 con 0x55a361c844d0 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4f857fa640 1 -- 172.21.15.39:0/549239882 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f9800fd70 con 0x55a361c844d0 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4f857fa640 1 -- 172.21.15.39:0/549239882 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4f8c03e040 con 0x55a361c844d0 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4faae5c740 10 client.26698 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:27:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4faae5c740 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:27:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4faae5c740 20 client.26698 populate_metadata read hostname 'smithi039' 2024-05-04T03:27:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4faae5c740 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a361c391c0 0x55a3620d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4faae5c740 4 client.26698 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:27:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a361fb3050 con 0x55a361c391c0 2024-05-04T03:27:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.546+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a361c391c0 0x55a3620d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.547+0000 7f4fa1d44640 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a361c391c0 0x55a3620d0880 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfce6b26d2836455 server_cookie=e61d5f0a0c235511 in_seq=0 out_seq=0 2024-05-04T03:27:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.547+0000 7f4f857fa640 10 client.26698 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.561+0000 7f4f857fa640 1 -- 172.21.15.39:0/549239882 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a361fb3050 con 0x55a361c391c0 2024-05-04T03:27:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.561+0000 7f4f857fa640 10 client.26698 handle_command_reply: tid=0 2024-05-04T03:27:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.561+0000 7f4faae5c740 1 client.26698 shutdown 2024-05-04T03:27:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.561+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361c844d0 msgr2=0x55a36205de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.561+0000 7f4faae5c740 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361c844d0 0x55a36205de60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c02ec80 tx=0x7f4f8c00f4f0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 shutdown_connections 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55a361c391c0 0x55a3620d0880 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a361ca51b0 0x55a3620ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a361c844d0 0x55a36205de60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 1 --2- 172.21.15.39:0/549239882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a361c7c2f0 0x55a36205d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 >> 172.21.15.39:0/549239882 conn(0x55a361c99300 msgr2=0x55a362041f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4f857fa640 0 client.26698 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 shutdown_connections 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 1 -- 172.21.15.39:0/549239882 wait complete. 2024-05-04T03:27:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.562+0000 7f4faae5c740 20 client.26698 trim_cache size 0 max 16384 2024-05-04T03:27:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:27:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:27:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:27:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:27:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.565+0000 7f4fa2d46640 1 -- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4f880b28e0 msgr2=0x7f4f880b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4f880b28e0 0x7f4f880b4df0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4f94011dd0 tx=0x7f4f940114a0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 -- 172.21.15.39:0/3729297324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9c1022a0 msgr2=0x7f4f9c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9c1022a0 0x7f4f9c197ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4f98007860 tx=0x7f4f98007c60 comp rx=0 tx=0).stop 2024-05-04T03:27:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 -- 172.21.15.39:0/3729297324 shutdown_connections 2024-05-04T03:27:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4f880b28e0 0x7f4f880b4df0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f9c107fe0 0x7f4f9c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f9c1032e0 0x7f4f9c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 --2- 172.21.15.39:0/3729297324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f9c1022a0 0x7f4f9c197ed0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 -- 172.21.15.39:0/3729297324 >> 172.21.15.39:0/3729297324 conn(0x7f4f9c0fcea0 msgr2=0x7f4f9c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 -- 172.21.15.39:0/3729297324 shutdown_connections 2024-05-04T03:27:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:10.566+0000 7f4fa2d46640 1 -- 172.21.15.39:0/3729297324 wait complete. 2024-05-04T03:27:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:10 smithi167 ceph-mon[33614]: pgmap v877: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2024-05-04T03:27:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:10 smithi039 ceph-mon[29715]: pgmap v877: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2024-05-04T03:27:11.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:10 smithi066 ceph-mon[33264]: pgmap v877: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2024-05-04T03:27:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 800.0 s, 1344.0 tps, lat 74.545 ms stddev 20.932 2024-05-04T03:27:12.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:12 smithi167 ceph-mon[33614]: pgmap v878: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-05-04T03:27:12.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:12 smithi039 ceph-mon[29715]: pgmap v878: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-05-04T03:27:13.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:12 smithi066 ceph-mon[33264]: pgmap v878: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 19 MiB/s wr, 1.35k op/s 2024-05-04T03:27:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:14 smithi167 ceph-mon[33614]: pgmap v879: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-04T03:27:14.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:14 smithi039 ceph-mon[29715]: pgmap v879: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-04T03:27:15.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:14 smithi066 ceph-mon[33264]: pgmap v879: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 20 MiB/s wr, 1.45k op/s 2024-05-04T03:27:16.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 805.0 s, 1373.4 tps, lat 72.749 ms stddev 19.675 2024-05-04T03:27:16.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:16 smithi167 ceph-mon[33614]: pgmap v880: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-04T03:27:16.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:16 smithi039 ceph-mon[29715]: pgmap v880: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-04T03:27:17.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:16 smithi066 ceph-mon[33264]: pgmap v880: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.34k op/s 2024-05-04T03:27:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:18 smithi167 ceph-mon[33614]: pgmap v881: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.42k op/s 2024-05-04T03:27:18.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:18 smithi039 ceph-mon[29715]: pgmap v881: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.42k op/s 2024-05-04T03:27:19.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:18 smithi066 ceph-mon[33264]: pgmap v881: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 1.42k op/s 2024-05-04T03:27:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:20 smithi167 ceph-mon[33614]: pgmap v882: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-04T03:27:20.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:20 smithi039 ceph-mon[29715]: pgmap v882: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-04T03:27:21.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:20 smithi066 ceph-mon[33264]: pgmap v882: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-04T03:27:21.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 810.0 s, 1315.2 tps, lat 76.042 ms stddev 23.999 2024-05-04T03:27:22.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:21 smithi066 ceph-mon[33264]: pgmap v883: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 18 MiB/s wr, 1.38k op/s 2024-05-04T03:27:22.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:21 smithi167 ceph-mon[33614]: pgmap v883: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 18 MiB/s wr, 1.38k op/s 2024-05-04T03:27:22.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:21 smithi039 ceph-mon[29715]: pgmap v883: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 18 MiB/s wr, 1.38k op/s 2024-05-04T03:27:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.726+0000 7f169b59e640 1 -- 172.21.15.39:0/2813357407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c106f60 msgr2=0x7f169c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.726+0000 7f169b59e640 1 --2- 172.21.15.39:0/2813357407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c106f60 0x7f169c107370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1684009860 tx=0x7f168402a340 comp rx=0 tx=0).stop 2024-05-04T03:27:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.727+0000 7f169b59e640 1 -- 172.21.15.39:0/2813357407 shutdown_connections 2024-05-04T03:27:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.727+0000 7f169b59e640 1 --2- 172.21.15.39:0/2813357407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f169c10bc60 0x7f169c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.727+0000 7f169b59e640 1 --2- 172.21.15.39:0/2813357407 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c106f60 0x7f169c107370 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.727+0000 7f169b59e640 1 --2- 172.21.15.39:0/2813357407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c105d20 0x7f169c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.729 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.727+0000 7f169b59e640 1 -- 172.21.15.39:0/2813357407 >> 172.21.15.39:0/2813357407 conn(0x7f169c100f10 msgr2=0x7f169c1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.727+0000 7f169b59e640 1 -- 172.21.15.39:0/2813357407 shutdown_connections 2024-05-04T03:27:23.730 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.727+0000 7f169b59e640 1 -- 172.21.15.39:0/2813357407 wait complete. 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169b59e640 1 Processor -- start 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169b59e640 1 -- start start 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169b59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c105d20 0x7f169c07ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169b59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f169c106f60 0x7f169c07bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c10bc60 0x7f169c0785a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169b59e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1116e0 con 0x7f169c10bc60 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169b59e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f169c111560 con 0x7f169c105d20 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169b59e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f169c111860 con 0x7f169c106f60 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f169a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c105d20 0x7f169c07ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c105d20 0x7f169c07ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:46286/0 (socket says 172.21.15.39:46286) 2024-05-04T03:27:23.732 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169a59c640 1 -- 172.21.15.39:0/2026359008 learned_addr learned my addr 172.21.15.39:0/2026359008 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.728+0000 7f1693fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f169c106f60 0x7f169c07bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169ad9d640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c10bc60 0x7f169c0785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169a59c640 1 -- 172.21.15.39:0/2026359008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f169c106f60 msgr2=0x7f169c07bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169a59c640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f169c106f60 0x7f169c07bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169a59c640 1 -- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c10bc60 msgr2=0x7f169c0785a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169a59c640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c10bc60 0x7f169c0785a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169a59c640 1 -- 172.21.15.39:0/2026359008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c078dc0 con 0x7f169c105d20 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169ad9d640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c10bc60 0x7f169c0785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f1693fff640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f169c106f60 0x7f169c07bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f169a59c640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c105d20 0x7f169c07ba50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e9c0 tx=0x7f168c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.729+0000 7f16937fe640 1 -- 172.21.15.39:0/2026359008 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c00ccc0 con 0x7f169c105d20 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.730+0000 7f169b59e640 1 -- 172.21.15.39:0/2026359008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c0790b0 con 0x7f169c105d20 2024-05-04T03:27:23.733 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.730+0000 7f169b59e640 1 -- 172.21.15.39:0/2026359008 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c06ca10 con 0x7f169c105d20 2024-05-04T03:27:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.730+0000 7f16937fe640 1 -- 172.21.15.39:0/2026359008 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f168c0035f0 con 0x7f169c105d20 2024-05-04T03:27:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.731+0000 7f16937fe640 1 -- 172.21.15.39:0/2026359008 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c00ce60 con 0x7f169c105d20 2024-05-04T03:27:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.733+0000 7f16a7bda740 1 Processor -- start 2024-05-04T03:27:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.733+0000 7f16a7bda740 1 -- start start 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172702c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d1727031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d17270a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f16a7bda740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d1726675d0 con 0x55d1723404d0 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f16a7bda740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d172620890 con 0x55d1723382f0 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f16a7bda740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d172663590 con 0x55d1723611b0 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f169ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172702c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f169ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172702c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36846/0 (socket says 172.21.15.39:36846) 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f169ad9d640 1 -- 172.21.15.39:0/50757902 learned_addr learned my addr 172.21.15.39:0/50757902 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f169ad9d640 1 -- 172.21.15.39:0/50757902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 msgr2=0x55d1727031b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f169ad9d640 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d1727031b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f169ad9d640 1 -- 172.21.15.39:0/50757902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 msgr2=0x55d17270a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:23.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f1693fff640 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d17270a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.734+0000 7f169ad9d640 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d17270a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.735+0000 7f169ad9d640 1 -- 172.21.15.39:0/50757902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d172379210 con 0x55d1723611b0 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.735+0000 7f1693fff640 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d17270a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.735+0000 7f169ad9d640 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172702c30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1694009fd0 tx=0x7f16940218f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ba90b869fc3d876 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.736+0000 7f1690ff9640 1 -- 172.21.15.39:0/50757902 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694022770 con 0x55d1723611b0 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.736+0000 7f1690ff9640 1 -- 172.21.15.39:0/50757902 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1694022da0 con 0x55d1723611b0 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.736+0000 7f1690ff9640 1 -- 172.21.15.39:0/50757902 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169401f3d0 con 0x55d1723611b0 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.736+0000 7f16937fe640 1 -- 172.21.15.39:0/2026359008 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f168c010750 con 0x7f169c105d20 2024-05-04T03:27:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 -- 172.21.15.39:0/50757902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 msgr2=0x55d172702c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172702c30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1694009fd0 tx=0x7f16940218f0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 -- 172.21.15.39:0/50757902 shutdown_connections 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d17270a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d1727031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 --2- 172.21.15.39:0/50757902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172702c30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 -- 172.21.15.39:0/50757902 >> 172.21.15.39:0/50757902 conn(0x55d172355300 msgr2=0x55d1726fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 -- 172.21.15.39:0/50757902 shutdown_connections 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.737+0000 7f16a7bda740 1 -- 172.21.15.39:0/50757902 wait complete. 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.738+0000 7f16937fe640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16880b27c0 0x7f16880b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.738+0000 7f16a7bda740 1 -- start start 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.738+0000 7f16937fe640 1 -- 172.21.15.39:0/2026359008 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f168c0e2530 con 0x7f169c105d20 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.738+0000 7f16a7bda740 1 Processor -- start 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d17271d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723404d0 0x55d17271ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723611b0 0x55d172724e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f16a7bda740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d17270ab50 con 0x55d1723611b0 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f16a7bda740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d172624370 con 0x55d1723382f0 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f16a7bda740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d17270acd0 con 0x55d1723404d0 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f1693fff640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16880b27c0 0x7f16880b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f16a7bda740 10 client.0 _ll_register_callbacks cb 0x55d172378a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f1693fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723404d0 0x55d17271ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f1693fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723404d0 0x55d17271ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36860/0 (socket says 172.21.15.39:36860) 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f1693fff640 1 -- 172.21.15.39:0/298604607 learned_addr learned my addr 172.21.15.39:0/298604607 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f169ad9d640 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723611b0 0x55d172724e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.739+0000 7f169a59c640 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d17271d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.740+0000 7f1693fff640 1 -- 172.21.15.39:0/298604607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 msgr2=0x55d17271d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.740+0000 7f1693fff640 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d17271d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.740+0000 7f1693fff640 1 -- 172.21.15.39:0/298604607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723611b0 msgr2=0x55d172724e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.740+0000 7f1693fff640 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723611b0 0x55d172724e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.740+0000 7f1693fff640 1 -- 172.21.15.39:0/298604607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d172445620 con 0x55d1723404d0 2024-05-04T03:27:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.740+0000 7f169ad9d640 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723611b0 0x55d172724e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:27:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.741+0000 7f1693fff640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16880b27c0 0x7f16880b4cd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1684000c00 tx=0x7f1684030760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.741+0000 7f1693fff640 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723404d0 0x55d17271ded0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1684005eb0 tx=0x7f1684044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c92b46d14bc2ca40 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.741+0000 7f1681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:27:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.741+0000 7f1681ffb640 1 -- 172.21.15.39:0/298604607 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684051060 con 0x55d1723404d0 2024-05-04T03:27:23.746 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.741+0000 7f16a7bda740 1 -- 172.21.15.39:0/298604607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1694022da0 con 0x55d1723404d0 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.741+0000 7f1681ffb640 1 -- 172.21.15.39:0/298604607 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1684004250 con 0x55d1723404d0 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f1681ffb640 1 -- 172.21.15.39:0/298604607 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1684051060 con 0x55d1723404d0 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f16a7bda740 10 client.26710 fetch_fsmap learned FSMap version 32 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f16a7bda740 1 -- 172.21.15.39:0/298604607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d172727d90 con 0x55d1723404d0 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f1681ffb640 1 -- 172.21.15.39:0/298604607 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1694022da0 con 0x55d1723404d0 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f1681ffb640 1 -- 172.21.15.39:0/298604607 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1684004700 con 0x55d1723404d0 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f16a7bda740 10 client.26710 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f16a7bda740 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f16a7bda740 20 client.26710 populate_metadata read hostname 'smithi039' 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f16a7bda740 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d17231fd80 0x55d17272b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f16a7bda740 4 client.26710 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.742+0000 7f16a7bda740 1 -- 172.21.15.39:0/298604607 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1726751e0 con 0x55d17231fd80 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.743+0000 7f169a59c640 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d17231fd80 0x55d17272b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.744+0000 7f1681ffb640 10 client.26710 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.744+0000 7f169a59c640 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d17231fd80 0x55d17272b930 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb40bc624135f8b server_cookie=7cb8ba2814aa5163 in_seq=0 out_seq=0 2024-05-04T03:27:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f1681ffb640 1 -- 172.21.15.39:0/298604607 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1726751e0 con 0x55d17231fd80 2024-05-04T03:27:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f1681ffb640 10 client.26710 handle_command_reply: tid=0 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 client.26710 shutdown 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 -- 172.21.15.39:0/298604607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723404d0 msgr2=0x55d17271ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723404d0 0x55d17271ded0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1684005eb0 tx=0x7f1684044000 comp rx=0 tx=0).stop 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 -- 172.21.15.39:0/298604607 shutdown_connections 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d17231fd80 0x55d17272b930 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723611b0 0x55d172724e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723404d0 0x55d17271ded0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 --2- 172.21.15.39:0/298604607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d17271d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f16a7bda740 1 -- 172.21.15.39:0/298604607 >> 172.21.15.39:0/298604607 conn(0x55d172355300 msgr2=0x55d1726fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.745+0000 7f1681ffb640 0 client.26710 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.746+0000 7f16a7bda740 1 -- 172.21.15.39:0/298604607 shutdown_connections 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.746+0000 7f16a7bda740 1 -- 172.21.15.39:0/298604607 wait complete. 2024-05-04T03:27:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.746+0000 7f16a7bda740 20 client.26710 trim_cache size 0 max 16384 2024-05-04T03:27:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.776+0000 7f16a7bda740 1 Processor -- start 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.777+0000 7f16a7bda740 1 -- start start 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.777+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723382f0 0x55d1726fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.777+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.777+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723611b0 0x55d172725290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.777+0000 7f169a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723382f0 0x55d1726fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.777+0000 7f169a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723382f0 0x55d1726fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36866/0 (socket says 172.21.15.39:36866) 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.777+0000 7f169a59c640 1 -- 172.21.15.39:0/3656481254 learned_addr learned my addr 172.21.15.39:0/3656481254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.777+0000 7f16a7bda740 1 -- 172.21.15.39:0/3656481254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f167c00e530 con 0x55d1723404d0 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f169ad9d640 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723611b0 0x55d172725290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f1693fff640 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f16a7bda740 1 -- 172.21.15.39:0/3656481254 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f167c010070 con 0x55d1723611b0 2024-05-04T03:27:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f16a7bda740 1 -- 172.21.15.39:0/3656481254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f167c00ec00 con 0x55d1723382f0 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f169a59c640 1 -- 172.21.15.39:0/3656481254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723611b0 msgr2=0x55d172725290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f169a59c640 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723611b0 0x55d172725290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f169a59c640 1 -- 172.21.15.39:0/3656481254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 msgr2=0x55d172724d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f169a59c640 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f169a59c640 1 -- 172.21.15.39:0/3656481254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d17270a460 con 0x55d1723382f0 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.778+0000 7f169ad9d640 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723611b0 0x55d172725290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.779+0000 7f169a59c640 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723382f0 0x55d1726fe690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f168c0a6d70 tx=0x7f168c0a5af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4a94c985b25faae server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.779+0000 7f1683fff640 1 -- 172.21.15.39:0/3656481254 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0a9b00 con 0x55d1723382f0 2024-05-04T03:27:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.779+0000 7f1683fff640 1 -- 172.21.15.39:0/3656481254 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d1726751e0 con 0x55d1723382f0 2024-05-04T03:27:23.782 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.779+0000 7f1683fff640 1 -- 172.21.15.39:0/3656481254 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0a5d90 con 0x55d1723382f0 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.780+0000 7f16a7bda740 1 -- 172.21.15.39:0/3656481254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723382f0 msgr2=0x55d1726fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.780+0000 7f16a7bda740 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723382f0 0x55d1726fe690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f168c0a6d70 tx=0x7f168c0a5af0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 -- 172.21.15.39:0/3656481254 shutdown_connections 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723611b0 0x55d172725290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 --2- 172.21.15.39:0/3656481254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723382f0 0x55d1726fe690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 -- 172.21.15.39:0/3656481254 >> 172.21.15.39:0/3656481254 conn(0x55d172355300 msgr2=0x55d1726fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 -- 172.21.15.39:0/3656481254 shutdown_connections 2024-05-04T03:27:23.783 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 -- 172.21.15.39:0/3656481254 wait complete. 2024-05-04T03:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 -- start start 2024-05-04T03:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.781+0000 7f16a7bda740 1 Processor -- start 2024-05-04T03:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d172724a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f16a7bda740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172791d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f16a7bda740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f167c00f9a0 con 0x55d1723404d0 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f16a7bda740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f167c00f2d0 con 0x55d1723382f0 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f16a7bda740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f167c00c4b0 con 0x55d1723611b0 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f16a7bda740 10 client.0 _ll_register_callbacks cb 0x7f168c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f169ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172791d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f1693fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f169ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172791d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36882/0 (socket says 172.21.15.39:36882) 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f169ad9d640 1 -- 172.21.15.39:0/1036213735 learned_addr learned my addr 172.21.15.39:0/1036213735 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.782+0000 7f169a59c640 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d172724a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.783+0000 7f1693fff640 1 -- 172.21.15.39:0/1036213735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 msgr2=0x55d172791d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.783+0000 7f1693fff640 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172791d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.783+0000 7f1693fff640 1 -- 172.21.15.39:0/1036213735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 msgr2=0x55d172724a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.783+0000 7f1693fff640 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d172724a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.783+0000 7f1693fff640 1 -- 172.21.15.39:0/1036213735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1727922e0 con 0x55d1723404d0 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.783+0000 7f1693fff640 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724f60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f168404b780 tx=0x7f168402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c131d088561f35f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.783+0000 7f1681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.784+0000 7f1681ffb640 1 -- 172.21.15.39:0/1036213735 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684063060 con 0x55d1723404d0 2024-05-04T03:27:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.784+0000 7f1681ffb640 1 -- 172.21.15.39:0/1036213735 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1684004250 con 0x55d1723404d0 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.784+0000 7f16a7bda740 1 -- 172.21.15.39:0/1036213735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1726751e0 con 0x55d1723404d0 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.784+0000 7f1681ffb640 1 -- 172.21.15.39:0/1036213735 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1684037040 con 0x55d1723404d0 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.784+0000 7f16a7bda740 10 client.17136 fetch_fsmap learned FSMap version 32 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.784+0000 7f16a7bda740 1 -- 172.21.15.39:0/1036213735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d172793570 con 0x55d1723404d0 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.784+0000 7f1681ffb640 1 -- 172.21.15.39:0/1036213735 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1684055e10 con 0x55d1723404d0 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.785+0000 7f1681ffb640 1 -- 172.21.15.39:0/1036213735 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1684004700 con 0x55d1723404d0 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.785+0000 7f16a7bda740 10 client.17136 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.785+0000 7f16a7bda740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.785+0000 7f16a7bda740 20 client.17136 populate_metadata read hostname 'smithi039' 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.785+0000 7f16a7bda740 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d1722f51c0 0x55d172795f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.785+0000 7f16a7bda740 4 client.17136 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.785+0000 7f16a7bda740 1 -- 172.21.15.39:0/1036213735 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1726751e0 con 0x55d1722f51c0 2024-05-04T03:27:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.785+0000 7f169a59c640 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d1722f51c0 0x55d172795f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.786+0000 7f1681ffb640 10 client.17136 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.786+0000 7f169a59c640 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d1722f51c0 0x55d172795f40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680569e256c0f445 server_cookie=a8a53694cab27e9a in_seq=0 out_seq=0 2024-05-04T03:27:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.796+0000 7f1681ffb640 1 -- 172.21.15.39:0/1036213735 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1726751e0 con 0x55d1722f51c0 2024-05-04T03:27:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.796+0000 7f1681ffb640 10 client.17136 handle_command_reply: tid=0 2024-05-04T03:27:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.796+0000 7f16a7bda740 1 client.17136 shutdown 2024-05-04T03:27:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 -- 172.21.15.39:0/1036213735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 msgr2=0x55d172724f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724f60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f168404b780 tx=0x7f168402a340 comp rx=0 tx=0).stop 2024-05-04T03:27:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 -- 172.21.15.39:0/1036213735 shutdown_connections 2024-05-04T03:27:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55d1722f51c0 0x55d172795f40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1723611b0 0x55d172791d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1723404d0 0x55d172724f60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 --2- 172.21.15.39:0/1036213735 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d1723382f0 0x55d172724a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 -- 172.21.15.39:0/1036213735 >> 172.21.15.39:0/1036213735 conn(0x55d172355300 msgr2=0x55d1726fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f1681ffb640 0 client.17136 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.797+0000 7f16a7bda740 1 -- 172.21.15.39:0/1036213735 shutdown_connections 2024-05-04T03:27:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.800+0000 7f16a7bda740 1 -- 172.21.15.39:0/1036213735 wait complete. 2024-05-04T03:27:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.800+0000 7f16a7bda740 20 client.17136 trim_cache size 0 max 16384 2024-05-04T03:27:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:27:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:27:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:27:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:27:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 -- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16880b27c0 msgr2=0x7f16880b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16880b27c0 0x7f16880b4cd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1684000c00 tx=0x7f1684030760 comp rx=0 tx=0).stop 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 -- 172.21.15.39:0/2026359008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c105d20 msgr2=0x7f169c07ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c105d20 0x7f169c07ba50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e9c0 tx=0x7f168c00ee80 comp rx=0 tx=0).stop 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 -- 172.21.15.39:0/2026359008 shutdown_connections 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f16880b27c0 0x7f16880b4cd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f169c10bc60 0x7f169c0785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f169c106f60 0x7f169c07bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 --2- 172.21.15.39:0/2026359008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f169c105d20 0x7f169c07ba50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.801+0000 7f169b59e640 1 -- 172.21.15.39:0/2026359008 >> 172.21.15.39:0/2026359008 conn(0x7f169c100f10 msgr2=0x7f169c107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.802+0000 7f169b59e640 1 -- 172.21.15.39:0/2026359008 shutdown_connections 2024-05-04T03:27:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:23.802+0000 7f169b59e640 1 -- 172.21.15.39:0/2026359008 wait complete. 2024-05-04T03:27:24.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:24 smithi039 ceph-mon[29715]: pgmap v884: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:24.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:24 smithi066 ceph-mon[33264]: pgmap v884: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:24.879 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:24 smithi167 ceph-mon[33614]: pgmap v884: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:26.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:26 smithi039 ceph-mon[29715]: pgmap v885: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-05-04T03:27:26.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 815.0 s, 1372.6 tps, lat 72.745 ms stddev 23.478 2024-05-04T03:27:26.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:26 smithi066 ceph-mon[33264]: pgmap v885: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-05-04T03:27:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:26 smithi167 ceph-mon[33614]: pgmap v885: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-05-04T03:27:28.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:28 smithi039 ceph-mon[29715]: pgmap v886: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:28.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:28 smithi066 ceph-mon[33264]: pgmap v886: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:28.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:28 smithi167 ceph-mon[33614]: pgmap v886: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:30.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:30 smithi039 ceph-mon[29715]: pgmap v887: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:30.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:30 smithi066 ceph-mon[33264]: pgmap v887: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:30.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:30 smithi167 ceph-mon[33614]: pgmap v887: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 18 MiB/s wr, 1.43k op/s 2024-05-04T03:27:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 820.0 s, 1263.0 tps, lat 79.150 ms stddev 53.052 2024-05-04T03:27:32.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:32 smithi066 ceph-mon[33264]: pgmap v888: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-04T03:27:32.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:32 smithi167 ceph-mon[33614]: pgmap v888: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-04T03:27:32.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:32 smithi039 ceph-mon[29715]: pgmap v888: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-04T03:27:34.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:34 smithi066 ceph-mon[33264]: pgmap v889: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-04T03:27:34.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:34 smithi167 ceph-mon[33614]: pgmap v889: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-04T03:27:34.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:34 smithi039 ceph-mon[29715]: pgmap v889: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-04T03:27:36.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 825.0 s, 1437.2 tps, lat 69.700 ms stddev 20.256 2024-05-04T03:27:36.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:36 smithi066 ceph-mon[33264]: pgmap v890: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-04T03:27:36.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:36 smithi167 ceph-mon[33614]: pgmap v890: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-04T03:27:36.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:36 smithi039 ceph-mon[29715]: pgmap v890: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 16 MiB/s wr, 1.34k op/s 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 -- 172.21.15.39:0/20914251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10a3d0 msgr2=0x7fbffc10a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 --2- 172.21.15.39:0/20914251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10a3d0 0x7fbffc10a7e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009900 tx=0x7fbff802a440 comp rx=0 tx=0).stop 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 -- 172.21.15.39:0/20914251 shutdown_connections 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 --2- 172.21.15.39:0/20914251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc10bcc0 0x7fbffc10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 --2- 172.21.15.39:0/20914251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc10b370 0x7fbffc10b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 --2- 172.21.15.39:0/20914251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10a3d0 0x7fbffc10a7e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 -- 172.21.15.39:0/20914251 >> 172.21.15.39:0/20914251 conn(0x7fbffc070a90 msgr2=0x7fbffc072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 -- 172.21.15.39:0/20914251 shutdown_connections 2024-05-04T03:27:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.963+0000 7fc003cc8640 1 -- 172.21.15.39:0/20914251 wait complete. 2024-05-04T03:27:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.964+0000 7fc003cc8640 1 Processor -- start 2024-05-04T03:27:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.964+0000 7fc003cc8640 1 -- start start 2024-05-04T03:27:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc003cc8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc10a3d0 0x7fbffc1a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc003cc8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10b370 0x7fbffc1a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc003cc8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc10bcc0 0x7fbffc1b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc003cc8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc11a100 con 0x7fbffc10a3d0 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc003cc8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc119f80 con 0x7fbffc10b370 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc003cc8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc11a280 con 0x7fbffc10bcc0 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc001263640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10b370 0x7fbffc1a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc002265640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc10bcc0 0x7fbffc1b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc002265640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc10bcc0 0x7fbffc1b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36046/0 (socket says 172.21.15.39:36046) 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.965+0000 7fc002265640 1 -- 172.21.15.39:0/3600224892 learned_addr learned my addr 172.21.15.39:0/3600224892 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc001a64640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc10a3d0 0x7fbffc1a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc002265640 1 -- 172.21.15.39:0/3600224892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10b370 msgr2=0x7fbffc1a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc002265640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10b370 0x7fbffc1a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc002265640 1 -- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc10a3d0 msgr2=0x7fbffc1a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc002265640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc10a3d0 0x7fbffc1a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc002265640 1 -- 172.21.15.39:0/3600224892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc1b0be0 con 0x7fbffc10bcc0 2024-05-04T03:27:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc001263640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10b370 0x7fbffc1a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:27:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc001a64640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc10a3d0 0x7fbffc1a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:27:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fc002265640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc10bcc0 0x7fbffc1b0620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009fd0 tx=0x7fbfec00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.966+0000 7fbff2ffd640 1 -- 172.21.15.39:0/3600224892 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011830 con 0x7fbffc10bcc0 2024-05-04T03:27:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.967+0000 7fbff2ffd640 1 -- 172.21.15.39:0/3600224892 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfec016070 con 0x7fbffc10bcc0 2024-05-04T03:27:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.967+0000 7fbff2ffd640 1 -- 172.21.15.39:0/3600224892 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec010d20 con 0x7fbffc10bcc0 2024-05-04T03:27:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.967+0000 7fc003cc8640 1 -- 172.21.15.39:0/3600224892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc1b0ed0 con 0x7fbffc10bcc0 2024-05-04T03:27:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.967+0000 7fc003cc8640 1 -- 172.21.15.39:0/3600224892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1b1300 con 0x7fbffc10bcc0 2024-05-04T03:27:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.971+0000 7fbff2ffd640 1 -- 172.21.15.39:0/3600224892 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbfec01b460 con 0x7fbffc10bcc0 2024-05-04T03:27:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.972+0000 7fbff2ffd640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfe80b28e0 0x7fbfe80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.972+0000 7fbff2ffd640 1 -- 172.21.15.39:0/3600224892 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbfec0e1860 con 0x7fbffc10bcc0 2024-05-04T03:27:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.973+0000 7fc001a64640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfe80b28e0 0x7fbfe80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.973+0000 7fc00ab7c740 1 Processor -- start 2024-05-04T03:27:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.973+0000 7fc00ab7c740 1 -- start start 2024-05-04T03:27:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2474a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc001a64640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfe80b28e0 0x7fbfe80b4df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbff8005f70 tx=0x7fbff8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe2474fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe247bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc00ab7c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bfe23d95d0 con 0x55bfe20b24d0 2024-05-04T03:27:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc00ab7c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2392890 con 0x55bfe20aa2f0 2024-05-04T03:27:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc00ab7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bfe23d5590 con 0x55bfe20d31b0 2024-05-04T03:27:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc001a64640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2474a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc001a64640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2474a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:36052/0 (socket says 172.21.15.39:36052) 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc001a64640 1 -- 172.21.15.39:0/881537167 learned_addr learned my addr 172.21.15.39:0/881537167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.974+0000 7fc001263640 1 --2- 172.21.15.39:0/881537167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe2474fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.975+0000 7fc001263640 1 -- 172.21.15.39:0/881537167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 msgr2=0x55bfe2474a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.975+0000 7fc001263640 1 --2- 172.21.15.39:0/881537167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2474a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.975+0000 7fc001263640 1 -- 172.21.15.39:0/881537167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 msgr2=0x55bfe247bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.975+0000 7fc001263640 1 --2- 172.21.15.39:0/881537167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe247bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.975+0000 7fc001263640 1 -- 172.21.15.39:0/881537167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe20eb210 con 0x55bfe20aa2f0 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.975+0000 7fc001263640 1 --2- 172.21.15.39:0/881537167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe2474fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005fd0 tx=0x7fbff400e860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2780d21a5ac922 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.976+0000 7fbfe7fff640 1 -- 172.21.15.39:0/881537167 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4014070 con 0x55bfe20aa2f0 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.976+0000 7fbfe7fff640 1 -- 172.21.15.39:0/881537167 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbff4004d20 con 0x55bfe20aa2f0 2024-05-04T03:27:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.976+0000 7fbfe7fff640 1 -- 172.21.15.39:0/881537167 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff40055e0 con 0x55bfe20aa2f0 2024-05-04T03:27:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.977+0000 7fc00ab7c740 1 -- 172.21.15.39:0/881537167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 msgr2=0x55bfe2474fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.977+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/881537167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe2474fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005fd0 tx=0x7fbff400e860 comp rx=0 tx=0).stop 2024-05-04T03:27:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.977+0000 7fc00ab7c740 1 -- 172.21.15.39:0/881537167 shutdown_connections 2024-05-04T03:27:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.977+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/881537167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe247bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.977+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/881537167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe2474fe0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.977+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/881537167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2474a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.977+0000 7fc00ab7c740 1 -- 172.21.15.39:0/881537167 >> 172.21.15.39:0/881537167 conn(0x55bfe20c7300 msgr2=0x55bfe24703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.977+0000 7fc00ab7c740 1 -- 172.21.15.39:0/881537167 shutdown_connections 2024-05-04T03:27:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.978+0000 7fc00ab7c740 1 -- 172.21.15.39:0/881537167 wait complete. 2024-05-04T03:27:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.979+0000 7fc00ab7c740 1 -- start start 2024-05-04T03:27:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.979+0000 7fc00ab7c740 1 Processor -- start 2024-05-04T03:27:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe247e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe247ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2485c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc00ab7c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bfe247c930 con 0x55bfe20b24d0 2024-05-04T03:27:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc00ab7c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2396370 con 0x55bfe20aa2f0 2024-05-04T03:27:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc00ab7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bfe247cab0 con 0x55bfe20d31b0 2024-05-04T03:27:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc00ab7c740 10 client.0 _ll_register_callbacks cb 0x55bfe20eaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:27:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc001263640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe247ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc001a64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe247e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc001263640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe247ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36490/0 (socket says 172.21.15.39:36490) 2024-05-04T03:27:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc001263640 1 -- 172.21.15.39:0/2738176137 learned_addr learned my addr 172.21.15.39:0/2738176137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:36.984 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.980+0000 7fc002265640 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2485c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.981+0000 7fc001a64640 1 -- 172.21.15.39:0/2738176137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 msgr2=0x55bfe2485c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.981+0000 7fc001a64640 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2485c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.981+0000 7fc001a64640 1 -- 172.21.15.39:0/2738176137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 msgr2=0x55bfe247ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.981+0000 7fc001a64640 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe247ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.981+0000 7fc001a64640 1 -- 172.21.15.39:0/2738176137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe21b7620 con 0x55bfe20aa2f0 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.981+0000 7fc001a64640 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe247e7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55bfe2475c30 tx=0x7fbff8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f839302f0457708 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.981+0000 7fbfe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.981+0000 7fbfe57fa640 1 -- 172.21.15.39:0/2738176137 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8053670 con 0x55bfe20aa2f0 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.982+0000 7fbfe57fa640 1 -- 172.21.15.39:0/2738176137 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbff8053810 con 0x55bfe20aa2f0 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.982+0000 7fc00ab7c740 1 -- 172.21.15.39:0/2738176137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff4004d20 con 0x55bfe20aa2f0 2024-05-04T03:27:36.985 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.982+0000 7fbfe57fa640 1 -- 172.21.15.39:0/2738176137 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbff8053a10 con 0x55bfe20aa2f0 2024-05-04T03:27:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.982+0000 7fc00ab7c740 10 client.26696 fetch_fsmap learned FSMap version 32 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.982+0000 7fc00ab7c740 1 -- 172.21.15.39:0/2738176137 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bfe2488be0 con 0x55bfe20aa2f0 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.982+0000 7fbfe57fa640 1 -- 172.21.15.39:0/2738176137 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbff4004d20 con 0x55bfe20aa2f0 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.982+0000 7fbfe57fa640 1 -- 172.21.15.39:0/2738176137 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbff804abf0 con 0x55bfe20aa2f0 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.983+0000 7fc00ab7c740 10 client.26696 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.983+0000 7fc00ab7c740 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.983+0000 7fc00ab7c740 20 client.26696 populate_metadata read hostname 'smithi039' 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.983+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfe2091d80 0x55bfe248c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.983+0000 7fc00ab7c740 4 client.26696 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.983+0000 7fc00ab7c740 1 -- 172.21.15.39:0/2738176137 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe23e71e0 con 0x55bfe2091d80 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.983+0000 7fc001263640 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfe2091d80 0x55bfe248c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.984+0000 7fbfe57fa640 10 client.26696 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:36.987 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.984+0000 7fc001263640 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfe2091d80 0x55bfe248c2b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7edabdc31a57af61 server_cookie=8cf682f464475450 in_seq=0 out_seq=0 2024-05-04T03:27:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fbfe57fa640 1 -- 172.21.15.39:0/2738176137 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bfe23e71e0 con 0x55bfe2091d80 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fbfe57fa640 10 client.26696 handle_command_reply: tid=0 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 client.26696 shutdown 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 -- 172.21.15.39:0/2738176137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 msgr2=0x55bfe247e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe247e7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55bfe2475c30 tx=0x7fbff8009260 comp rx=0 tx=0).stop 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 -- 172.21.15.39:0/2738176137 shutdown_connections 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfe2091d80 0x55bfe248c2b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20d31b0 0x55bfe2485c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fbfe57fa640 0 client.26696 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe247ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/2738176137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20aa2f0 0x55bfe247e7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 -- 172.21.15.39:0/2738176137 >> 172.21.15.39:0/2738176137 conn(0x55bfe20c7300 msgr2=0x55bfe2473300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 -- 172.21.15.39:0/2738176137 shutdown_connections 2024-05-04T03:27:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.985+0000 7fc00ab7c740 1 -- 172.21.15.39:0/2738176137 wait complete. 2024-05-04T03:27:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:36.986+0000 7fc00ab7c740 20 client.26696 trim_cache size 0 max 16384 2024-05-04T03:27:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.015+0000 7fc00ab7c740 1 Processor -- start 2024-05-04T03:27:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.015+0000 7fc00ab7c740 1 -- start start 2024-05-04T03:27:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.016+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe2091d80 0x55bfe2470410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.016+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2485bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.016+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20b24d0 0x55bfe2486110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.016+0000 7fc00ab7c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ba150 con 0x55bfe2091d80 2024-05-04T03:27:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.016+0000 7fc00ab7c740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1bbc90 con 0x55bfe20b24d0 2024-05-04T03:27:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.016+0000 7fc002265640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20b24d0 0x55bfe2486110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.016+0000 7fc002265640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20b24d0 0x55bfe2486110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:33738/0 (socket says 172.21.15.39:33738) 2024-05-04T03:27:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.016+0000 7fc002265640 1 -- 172.21.15.39:0/985939136 learned_addr learned my addr 172.21.15.39:0/985939136 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.017+0000 7fc00ab7c740 1 -- 172.21.15.39:0/985939136 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ba820 con 0x55bfe20aa2f0 2024-05-04T03:27:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.017+0000 7fc001263640 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2485bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.017+0000 7fc002265640 1 -- 172.21.15.39:0/985939136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 msgr2=0x55bfe2485bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.017+0000 7fc001a64640 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe2091d80 0x55bfe2470410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.017+0000 7fc002265640 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2485bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.017+0000 7fc002265640 1 -- 172.21.15.39:0/985939136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe2091d80 msgr2=0x55bfe2470410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.017+0000 7fc002265640 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe2091d80 0x55bfe2470410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.017+0000 7fc002265640 1 -- 172.21.15.39:0/985939136 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe246f980 con 0x55bfe20b24d0 2024-05-04T03:27:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.018+0000 7fc001263640 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2485bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:27:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.018+0000 7fc001a64640 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe2091d80 0x55bfe2470410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:27:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.018+0000 7fc002265640 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20b24d0 0x55bfe2486110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0a8dc0 tx=0x7fbfec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c3cb0cab8035a4c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.019+0000 7fbfe77fe640 1 -- 172.21.15.39:0/985939136 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0a8e30 con 0x55bfe20b24d0 2024-05-04T03:27:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.019+0000 7fbfe77fe640 1 -- 172.21.15.39:0/985939136 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfec0ad1e0 con 0x55bfe20b24d0 2024-05-04T03:27:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.019+0000 7fbfe77fe640 1 -- 172.21.15.39:0/985939136 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0b1a60 con 0x55bfe20b24d0 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 -- 172.21.15.39:0/985939136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20b24d0 msgr2=0x55bfe2486110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20b24d0 0x55bfe2486110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0a8dc0 tx=0x7fbfec0046c0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 -- 172.21.15.39:0/985939136 shutdown_connections 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe20b24d0 0x55bfe2486110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2485bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/985939136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe2091d80 0x55bfe2470410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 -- 172.21.15.39:0/985939136 >> 172.21.15.39:0/985939136 conn(0x55bfe20c7300 msgr2=0x55bfe246fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 -- 172.21.15.39:0/985939136 shutdown_connections 2024-05-04T03:27:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.020+0000 7fc00ab7c740 1 -- 172.21.15.39:0/985939136 wait complete. 2024-05-04T03:27:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.021+0000 7fc00ab7c740 1 -- start start 2024-05-04T03:27:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.021+0000 7fc00ab7c740 1 Processor -- start 2024-05-04T03:27:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.022+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe2091d80 0x55bfe24836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.022+0000 7fc001a64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe2091d80 0x55bfe24836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.022+0000 7fc00ab7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2483c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.022+0000 7fc001a64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe2091d80 0x55bfe24836e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:33746/0 (socket says 172.21.15.39:33746) 2024-05-04T03:27:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.022+0000 7fc001a64640 1 -- 172.21.15.39:0/4173630547 learned_addr learned my addr 172.21.15.39:0/4173630547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.022+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe2484b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.023+0000 7fc002265640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe2484b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.023+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1bb5c0 con 0x55bfe20b24d0 2024-05-04T03:27:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.023+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1baef0 con 0x55bfe2091d80 2024-05-04T03:27:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.023+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1b8130 con 0x55bfe20aa2f0 2024-05-04T03:27:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.023+0000 7fc001263640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2483c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:37.027 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.023+0000 7fc00ab7c740 10 client.0 _ll_register_callbacks cb 0x7fbff8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:27:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.024+0000 7fc002265640 1 -- 172.21.15.39:0/4173630547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 msgr2=0x55bfe2483c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.024+0000 7fc002265640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2483c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.024+0000 7fc002265640 1 -- 172.21.15.39:0/4173630547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe2091d80 msgr2=0x55bfe24836e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.024+0000 7fc002265640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe2091d80 0x55bfe24836e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.024+0000 7fc002265640 1 -- 172.21.15.39:0/4173630547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe24850c0 con 0x55bfe20b24d0 2024-05-04T03:27:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.024+0000 7fc001263640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2483c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:27:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.024+0000 7fc001a64640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe2091d80 0x55bfe24836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:27:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.025+0000 7fc002265640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe2484b00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbfec01f000 tx=0x7fbfec010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaeb33bd437a6ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.025+0000 7fbfe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:27:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.025+0000 7fbfe57fa640 1 -- 172.21.15.39:0/4173630547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec088060 con 0x55bfe20b24d0 2024-05-04T03:27:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.025+0000 7fbfe57fa640 1 -- 172.21.15.39:0/4173630547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbfec0b1a10 con 0x55bfe20b24d0 2024-05-04T03:27:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.025+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfec0ad1e0 con 0x55bfe20b24d0 2024-05-04T03:27:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.026+0000 7fbfe57fa640 1 -- 172.21.15.39:0/4173630547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbfec0c9070 con 0x55bfe20b24d0 2024-05-04T03:27:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.026+0000 7fc00ab7c740 10 client.17142 fetch_fsmap learned FSMap version 32 2024-05-04T03:27:37.029 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.026+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bfe24862a0 con 0x55bfe20b24d0 2024-05-04T03:27:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fbfe57fa640 1 -- 172.21.15.39:0/4173630547 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbfec0b1c90 con 0x55bfe20b24d0 2024-05-04T03:27:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fbfe57fa640 1 -- 172.21.15.39:0/4173630547 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbfec0ae370 con 0x55bfe20b24d0 2024-05-04T03:27:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fc00ab7c740 10 client.17142 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:27:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fc00ab7c740 10 client.17142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:27:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fc00ab7c740 20 client.17142 populate_metadata read hostname 'smithi039' 2024-05-04T03:27:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfe20d31b0 0x55bfe24f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fc00ab7c740 4 client.17142 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:27:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbfec0ad1e0 con 0x55bfe20d31b0 2024-05-04T03:27:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.027+0000 7fc001a64640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfe20d31b0 0x55bfe24f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.028+0000 7fbfe57fa640 10 client.17142 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.028+0000 7fc001a64640 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfe20d31b0 0x55bfe24f68a0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3512b90c6c1b9cda server_cookie=a6becd37562d2f20 in_seq=0 out_seq=0 2024-05-04T03:27:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.037+0000 7fbfe57fa640 1 -- 172.21.15.39:0/4173630547 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbfec0ad1e0 con 0x55bfe20d31b0 2024-05-04T03:27:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.037+0000 7fbfe57fa640 10 client.17142 handle_command_reply: tid=0 2024-05-04T03:27:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.038+0000 7fc00ab7c740 1 client.17142 shutdown 2024-05-04T03:27:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.038+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 msgr2=0x55bfe2484b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:37.041 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.038+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe2484b00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbfec01f000 tx=0x7fbfec010640 comp rx=0 tx=0).stop 2024-05-04T03:27:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.038+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 shutdown_connections 2024-05-04T03:27:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.038+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55bfe20d31b0 0x55bfe24f68a0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.039+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfe20b24d0 0x55bfe2484b00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.039+0000 7fbfe57fa640 0 client.17142 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.039+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bfe20aa2f0 0x55bfe2483c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.039+0000 7fc00ab7c740 1 --2- 172.21.15.39:0/4173630547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfe2091d80 0x55bfe24836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.039+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 >> 172.21.15.39:0/4173630547 conn(0x55bfe20c7300 msgr2=0x55bfe246e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.040+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 shutdown_connections 2024-05-04T03:27:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.040+0000 7fc00ab7c740 1 -- 172.21.15.39:0/4173630547 wait complete. 2024-05-04T03:27:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.040+0000 7fc00ab7c740 20 client.17142 trim_cache size 0 max 16384 2024-05-04T03:27:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:27:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:27:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:27:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:27:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.042+0000 7fc003cc8640 1 -- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfe80b28e0 msgr2=0x7fbfe80b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.042+0000 7fc003cc8640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfe80b28e0 0x7fbfe80b4df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbff8005f70 tx=0x7fbff8005fd0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.042+0000 7fc003cc8640 1 -- 172.21.15.39:0/3600224892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc10bcc0 msgr2=0x7fbffc1b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.042+0000 7fc003cc8640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc10bcc0 0x7fbffc1b0620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009fd0 tx=0x7fbfec00b8e0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.046+0000 7fc003cc8640 1 -- 172.21.15.39:0/3600224892 shutdown_connections 2024-05-04T03:27:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.046+0000 7fc003cc8640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbfe80b28e0 0x7fbfe80b4df0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.046+0000 7fc003cc8640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbffc10bcc0 0x7fbffc1b0620 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.046+0000 7fc003cc8640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbffc10b370 0x7fbffc1a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.046+0000 7fc003cc8640 1 --2- 172.21.15.39:0/3600224892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc10a3d0 0x7fbffc1a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.046+0000 7fc003cc8640 1 -- 172.21.15.39:0/3600224892 >> 172.21.15.39:0/3600224892 conn(0x7fbffc070a90 msgr2=0x7fbffc1118a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:37.050 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.047+0000 7fc003cc8640 1 -- 172.21.15.39:0/3600224892 shutdown_connections 2024-05-04T03:27:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:37.047+0000 7fc003cc8640 1 -- 172.21.15.39:0/3600224892 wait complete. 2024-05-04T03:27:38.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:38 smithi066 ceph-mon[33264]: pgmap v891: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-04T03:27:38.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:38 smithi167 ceph-mon[33614]: pgmap v891: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-04T03:27:38.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:38 smithi039 ceph-mon[29715]: pgmap v891: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2024-05-04T03:27:40.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:40 smithi066 ceph-mon[33264]: pgmap v892: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-05-04T03:27:40.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:40 smithi167 ceph-mon[33614]: pgmap v892: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-05-04T03:27:40.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:40 smithi039 ceph-mon[29715]: pgmap v892: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-05-04T03:27:41.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 830.0 s, 1311.6 tps, lat 76.295 ms stddev 21.322 2024-05-04T03:27:42.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:42 smithi066 ceph-mon[33264]: pgmap v893: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:42.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:42 smithi167 ceph-mon[33614]: pgmap v893: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:42.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:42 smithi039 ceph-mon[29715]: pgmap v893: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:44.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:44 smithi066 ceph-mon[33264]: pgmap v894: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-05-04T03:27:44.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:44 smithi167 ceph-mon[33614]: pgmap v894: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-05-04T03:27:44.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:44 smithi039 ceph-mon[29715]: pgmap v894: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.43k op/s 2024-05-04T03:27:46.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:46 smithi039 ceph-mon[29715]: pgmap v895: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:46.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 835.0 s, 471.7 tps, lat 92.943 ms stddev 92.857 2024-05-04T03:27:46.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:46 smithi066 ceph-mon[33264]: pgmap v895: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:46.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:46 smithi167 ceph-mon[33614]: pgmap v895: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:48.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:48 smithi066 ceph-mon[33264]: pgmap v896: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:48.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:48 smithi167 ceph-mon[33614]: pgmap v896: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:48.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:48 smithi039 ceph-mon[29715]: pgmap v896: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-05-04T03:27:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.213+0000 7fed833e3640 1 -- 172.21.15.39:0/504569413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c10a450 msgr2=0x7fed7c10af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.213+0000 7fed833e3640 1 --2- 172.21.15.39:0/504569413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c10a450 0x7fed7c10af50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fed78009900 tx=0x7fed7802a3f0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.213+0000 7fed833e3640 1 -- 172.21.15.39:0/504569413 shutdown_connections 2024-05-04T03:27:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.213+0000 7fed833e3640 1 --2- 172.21.15.39:0/504569413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c110190 0x7fed7c1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.213+0000 7fed833e3640 1 --2- 172.21.15.39:0/504569413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed7c10b490 0x7fed7c10b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.213+0000 7fed833e3640 1 --2- 172.21.15.39:0/504569413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c10a450 0x7fed7c10af50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.213+0000 7fed833e3640 1 -- 172.21.15.39:0/504569413 >> 172.21.15.39:0/504569413 conn(0x7fed7c078f10 msgr2=0x7fed7c07b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.214+0000 7fed833e3640 1 -- 172.21.15.39:0/504569413 shutdown_connections 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.214+0000 7fed833e3640 1 -- 172.21.15.39:0/504569413 wait complete. 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed833e3640 1 Processor -- start 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed833e3640 1 -- start start 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed833e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c10a450 0x7fed7c1a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed833e3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c10b490 0x7fed7c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed833e3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed7c110190 0x7fed7c1a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed833e3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c115c60 con 0x7fed7c10b490 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed833e3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c115ae0 con 0x7fed7c110190 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed833e3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c115de0 con 0x7fed7c10a450 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed8097e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c10b490 0x7fed7c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed8097e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c10b490 0x7fed7c1a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55228/0 (socket says 172.21.15.39:55228) 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed8097e640 1 -- 172.21.15.39:0/791161206 learned_addr learned my addr 172.21.15.39:0/791161206 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed81980640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed7c110190 0x7fed7c1a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.215+0000 7fed8117f640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c10a450 0x7fed7c1a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed8097e640 1 -- 172.21.15.39:0/791161206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c10a450 msgr2=0x7fed7c1a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed8097e640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c10a450 0x7fed7c1a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed8097e640 1 -- 172.21.15.39:0/791161206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed7c110190 msgr2=0x7fed7c1a7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed8097e640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed7c110190 0x7fed7c1a7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed8097e640 1 -- 172.21.15.39:0/791161206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c1a8400 con 0x7fed7c10b490 2024-05-04T03:27:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed8097e640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c10b490 0x7fed7c1a0e00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ea60 tx=0x7fed6c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed727fc640 1 -- 172.21.15.39:0/791161206 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00cde0 con 0x7fed7c10b490 2024-05-04T03:27:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed727fc640 1 -- 172.21.15.39:0/791161206 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed6c0035f0 con 0x7fed7c10b490 2024-05-04T03:27:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed727fc640 1 -- 172.21.15.39:0/791161206 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c010690 con 0x7fed7c10b490 2024-05-04T03:27:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed833e3640 1 -- 172.21.15.39:0/791161206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c1a86f0 con 0x7fed7c10b490 2024-05-04T03:27:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.216+0000 7fed833e3640 1 -- 172.21.15.39:0/791161206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c1a8c90 con 0x7fed7c10b490 2024-05-04T03:27:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.220+0000 7fed727fc640 1 -- 172.21.15.39:0/791161206 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fed6c004000 con 0x7fed7c10b490 2024-05-04T03:27:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.220+0000 7fed8a36f740 1 Processor -- start 2024-05-04T03:27:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.220+0000 7fed8a36f740 1 -- start start 2024-05-04T03:27:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699e11b0 0x561d69d82a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d82fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed81980640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699e11b0 0x561d69d82a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed81980640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699e11b0 0x561d69d82a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55234/0 (socket says 172.21.15.39:55234) 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed727fc640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fed680b2860 0x7fed680b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 0x561d69d89ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed8a36f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561d69ce75d0 con 0x561d699e11b0 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed8a36f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561d69ca0890 con 0x561d699c04d0 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed8a36f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561d69ce3590 con 0x561d699b82f0 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.221+0000 7fed81980640 1 -- 172.21.15.39:0/588583401 learned_addr learned my addr 172.21.15.39:0/588583401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed8117f640 1 --2- 172.21.15.39:0/588583401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d82fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed727fc640 1 -- 172.21.15.39:0/791161206 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fed6c0e2440 con 0x7fed7c10b490 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed81980640 1 -- 172.21.15.39:0/588583401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 msgr2=0x561d69d82fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed81980640 1 --2- 172.21.15.39:0/588583401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d82fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed81980640 1 -- 172.21.15.39:0/588583401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 msgr2=0x561d69d89ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed81980640 1 --2- 172.21.15.39:0/588583401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 0x561d69d89ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed81980640 1 -- 172.21.15.39:0/588583401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d699f9210 con 0x561d699e11b0 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed8097e640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fed680b2860 0x7fed680b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed81980640 1 --2- 172.21.15.39:0/588583401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699e11b0 0x561d69d82a60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fed74011440 tx=0x7fed74011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbfe92501cb1bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.222+0000 7fed677fe640 1 -- 172.21.15.39:0/588583401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74004d20 con 0x561d699e11b0 2024-05-04T03:27:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.223+0000 7fed677fe640 1 -- 172.21.15.39:0/588583401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed74004050 con 0x561d699e11b0 2024-05-04T03:27:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.223+0000 7fed677fe640 1 -- 172.21.15.39:0/588583401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74005240 con 0x561d699e11b0 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 -- 172.21.15.39:0/588583401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699e11b0 msgr2=0x561d69d82a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8097e640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fed680b2860 0x7fed680b4d70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed6c000c00 tx=0x7fed6c00ef90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 --2- 172.21.15.39:0/588583401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699e11b0 0x561d69d82a60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fed74011440 tx=0x7fed74011900 comp rx=0 tx=0).stop 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 -- 172.21.15.39:0/588583401 shutdown_connections 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 --2- 172.21.15.39:0/588583401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 0x561d69d89ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 --2- 172.21.15.39:0/588583401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d82fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 --2- 172.21.15.39:0/588583401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699e11b0 0x561d69d82a60 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 -- 172.21.15.39:0/588583401 >> 172.21.15.39:0/588583401 conn(0x561d699d5300 msgr2=0x561d69d7e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 -- 172.21.15.39:0/588583401 shutdown_connections 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.224+0000 7fed8a36f740 1 -- 172.21.15.39:0/588583401 wait complete. 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.225+0000 7fed8a36f740 1 -- start start 2024-05-04T03:27:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.225+0000 7fed8a36f740 1 Processor -- start 2024-05-04T03:27:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.225+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d8c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.225+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699c04d0 0x561d69d8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699e11b0 0x561d69d93b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed8a36f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561d69d8a930 con 0x561d699c04d0 2024-05-04T03:27:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed8a36f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x561d69ca4370 con 0x561d699e11b0 2024-05-04T03:27:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed8a36f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561d69d8aab0 con 0x561d699b82f0 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d8c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d8c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:58292/0 (socket says 172.21.15.39:58292) 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 -- 172.21.15.39:0/1594557853 learned_addr learned my addr 172.21.15.39:0/1594557853 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699e11b0 0x561d69d93b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed8a36f740 10 client.0 _ll_register_callbacks cb 0x561d699f8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed8117f640 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699c04d0 0x561d69d8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 -- 172.21.15.39:0/1594557853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699e11b0 msgr2=0x561d69d93b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699e11b0 0x561d69d93b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 -- 172.21.15.39:0/1594557853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699c04d0 msgr2=0x561d69d8cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699c04d0 0x561d69d8cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.226+0000 7fed81980640 1 -- 172.21.15.39:0/1594557853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d69ac5620 con 0x561d699b82f0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.227+0000 7fed81980640 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d8c6a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fed74000f80 tx=0x7fed74009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83cc31e596bcc7d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.227+0000 7fed64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.227+0000 7fed64ff9640 1 -- 172.21.15.39:0/1594557853 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74005240 con 0x561d699b82f0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.227+0000 7fed64ff9640 1 -- 172.21.15.39:0/1594557853 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fed740053e0 con 0x561d699b82f0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.227+0000 7fed8a36f740 1 -- 172.21.15.39:0/1594557853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed74004050 con 0x561d699b82f0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.227+0000 7fed64ff9640 1 -- 172.21.15.39:0/1594557853 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed74003e70 con 0x561d699b82f0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.227+0000 7fed8a36f740 10 client.26740 fetch_fsmap learned FSMap version 32 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.227+0000 7fed8a36f740 1 -- 172.21.15.39:0/1594557853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561d69d96a70 con 0x561d699b82f0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.228+0000 7fed64ff9640 1 -- 172.21.15.39:0/1594557853 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed74004050 con 0x561d699b82f0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.228+0000 7fed64ff9640 1 -- 172.21.15.39:0/1594557853 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fed74025070 con 0x561d699b82f0 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.228+0000 7fed8a36f740 10 client.26740 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:27:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.228+0000 7fed8a36f740 10 client.26740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:27:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.228+0000 7fed8a36f740 20 client.26740 populate_metadata read hostname 'smithi039' 2024-05-04T03:27:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.228+0000 7fed8a36f740 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561d6999fd80 0x561d69d99f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.228+0000 7fed8a36f740 4 client.26740 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:27:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.228+0000 7fed8a36f740 1 -- 172.21.15.39:0/1594557853 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d69cf51e0 con 0x561d6999fd80 2024-05-04T03:27:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.229+0000 7fed8117f640 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561d6999fd80 0x561d69d99f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.229+0000 7fed8117f640 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561d6999fd80 0x561d69d99f40 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967533897742a81f server_cookie=1ccafae3b10f9118 in_seq=0 out_seq=0 2024-05-04T03:27:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.229+0000 7fed64ff9640 10 client.26740 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed64ff9640 1 -- 172.21.15.39:0/1594557853 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d69cf51e0 con 0x561d6999fd80 2024-05-04T03:27:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed64ff9640 10 client.26740 handle_command_reply: tid=0 2024-05-04T03:27:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 client.26740 shutdown 2024-05-04T03:27:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 -- 172.21.15.39:0/1594557853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 msgr2=0x561d69d8c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d8c6a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fed74000f80 tx=0x7fed74009fb0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 -- 172.21.15.39:0/1594557853 shutdown_connections 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561d6999fd80 0x561d69d99f40 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699e11b0 0x561d69d93b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699c04d0 0x561d69d8cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed64ff9640 0 client.26740 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 --2- 172.21.15.39:0/1594557853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d8c6a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 -- 172.21.15.39:0/1594557853 >> 172.21.15.39:0/1594557853 conn(0x561d699d5300 msgr2=0x561d69d81300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 -- 172.21.15.39:0/1594557853 shutdown_connections 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.230+0000 7fed8a36f740 1 -- 172.21.15.39:0/1594557853 wait complete. 2024-05-04T03:27:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.231+0000 7fed8a36f740 20 client.26740 trim_cache size 0 max 16384 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.258+0000 7fed8a36f740 1 Processor -- start 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.258+0000 7fed8a36f740 1 -- start start 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.258+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d6999fd80 0x561d69d861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.258+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d93ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 0x561d69d94010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed8a36f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b19f0 con 0x561d6999fd80 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed8a36f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b3530 con 0x561d699c04d0 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed8a36f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b20c0 con 0x561d699b82f0 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed8117f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d93ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed81980640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d6999fd80 0x561d69d861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed8117f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d93ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:58300/0 (socket says 172.21.15.39:58300) 2024-05-04T03:27:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed8117f640 1 -- 172.21.15.39:0/2515036981 learned_addr learned my addr 172.21.15.39:0/2515036981 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.259+0000 7fed81980640 1 --2- 172.21.15.39:0/2515036981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 0x561d69d94010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.260+0000 7fed8117f640 1 -- 172.21.15.39:0/2515036981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 msgr2=0x561d69d94010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.260+0000 7fed8117f640 1 --2- 172.21.15.39:0/2515036981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 0x561d69d94010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.260+0000 7fed8117f640 1 -- 172.21.15.39:0/2515036981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d6999fd80 msgr2=0x561d69d861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.260+0000 7fed8117f640 1 --2- 172.21.15.39:0/2515036981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d6999fd80 0x561d69d861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.260+0000 7fed8117f640 1 -- 172.21.15.39:0/2515036981 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d69d89e90 con 0x561d699b82f0 2024-05-04T03:27:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.260+0000 7fed8117f640 1 --2- 172.21.15.39:0/2515036981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d93ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed780057c0 tx=0x7fed78005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bb60a1529c94f63 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.261+0000 7fed66ffd640 1 -- 172.21.15.39:0/2515036981 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78035040 con 0x561d699b82f0 2024-05-04T03:27:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.261+0000 7fed66ffd640 1 -- 172.21.15.39:0/2515036981 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed78046a90 con 0x561d699b82f0 2024-05-04T03:27:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.261+0000 7fed66ffd640 1 -- 172.21.15.39:0/2515036981 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78031500 con 0x561d699b82f0 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 -- 172.21.15.39:0/2515036981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 msgr2=0x561d69d93ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 --2- 172.21.15.39:0/2515036981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d93ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed780057c0 tx=0x7fed78005720 comp rx=0 tx=0).stop 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 -- 172.21.15.39:0/2515036981 shutdown_connections 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 --2- 172.21.15.39:0/2515036981 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d699c04d0 0x561d69d94010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 --2- 172.21.15.39:0/2515036981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699b82f0 0x561d69d93ad0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 --2- 172.21.15.39:0/2515036981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d6999fd80 0x561d69d861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 -- 172.21.15.39:0/2515036981 >> 172.21.15.39:0/2515036981 conn(0x561d699d5300 msgr2=0x561d69d85ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 -- 172.21.15.39:0/2515036981 shutdown_connections 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 -- 172.21.15.39:0/2515036981 wait complete. 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.262+0000 7fed8a36f740 1 -- start start 2024-05-04T03:27:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.263+0000 7fed8a36f740 1 Processor -- start 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d6999fd80 0x561d69d934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699b82f0 0x561d69d93a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8a36f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699c04d0 0x561d69e00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8a36f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b2e60 con 0x561d699b82f0 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8a36f740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b2790 con 0x561d6999fd80 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8a36f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1af9d0 con 0x561d699c04d0 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8a36f740 10 client.0 _ll_register_callbacks cb 0x7fed6c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8117f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699b82f0 0x561d69d93a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed81980640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d6999fd80 0x561d69d934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed81980640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d6999fd80 0x561d69d934d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:53390/0 (socket says 172.21.15.39:53390) 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed81980640 1 -- 172.21.15.39:0/105806776 learned_addr learned my addr 172.21.15.39:0/105806776 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed8117f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699b82f0 0x561d69d93a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55240/0 (socket says 172.21.15.39:55240) 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed81980640 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699c04d0 0x561d69e00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed81980640 1 -- 172.21.15.39:0/105806776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699c04d0 msgr2=0x561d69e00690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed81980640 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699c04d0 0x561d69e00690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed81980640 1 -- 172.21.15.39:0/105806776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699b82f0 msgr2=0x561d69d93a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.264+0000 7fed81980640 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699b82f0 0x561d69d93a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.265+0000 7fed81980640 1 -- 172.21.15.39:0/105806776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d69e00c50 con 0x561d6999fd80 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.265+0000 7fed81980640 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d6999fd80 0x561d69d934d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed74014040 tx=0x7fed74011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee8047a295c5c44 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.265+0000 7fed64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.265+0000 7fed64ff9640 1 -- 172.21.15.39:0/105806776 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740048a0 con 0x561d6999fd80 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.265+0000 7fed64ff9640 1 -- 172.21.15.39:0/105806776 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fed7400f890 con 0x561d6999fd80 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.265+0000 7fed8a36f740 1 -- 172.21.15.39:0/105806776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed78046a90 con 0x561d6999fd80 2024-05-04T03:27:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed64ff9640 1 -- 172.21.15.39:0/105806776 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed7400fa90 con 0x561d6999fd80 2024-05-04T03:27:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed8a36f740 10 client.26717 fetch_fsmap learned FSMap version 32 2024-05-04T03:27:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed8a36f740 1 -- 172.21.15.39:0/105806776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561d69e01ee0 con 0x561d6999fd80 2024-05-04T03:27:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed64ff9640 1 -- 172.21.15.39:0/105806776 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed78046a90 con 0x561d6999fd80 2024-05-04T03:27:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed64ff9640 1 -- 172.21.15.39:0/105806776 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fed74025040 con 0x561d6999fd80 2024-05-04T03:27:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed8a36f740 10 client.26717 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:27:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed8a36f740 10 client.26717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:27:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed8a36f740 20 client.26717 populate_metadata read hostname 'smithi039' 2024-05-04T03:27:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed8a36f740 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561d699e11b0 0x561d69e04880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:27:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed8a36f740 4 client.26717 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:27:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.266+0000 7fed8a36f740 1 -- 172.21.15.39:0/105806776 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d69cef050 con 0x561d699e11b0 2024-05-04T03:27:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.267+0000 7fed8117f640 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561d699e11b0 0x561d69e04880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:27:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.267+0000 7fed64ff9640 10 client.26717 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.268+0000 7fed8117f640 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561d699e11b0 0x561d69e04880 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d71a0d888289e1e server_cookie=a7b93eeb46c12a29 in_seq=0 out_seq=0 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed64ff9640 1 -- 172.21.15.39:0/105806776 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d69cef050 con 0x561d699e11b0 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed64ff9640 10 client.26717 handle_command_reply: tid=0 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 client.26717 shutdown 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 -- 172.21.15.39:0/105806776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d6999fd80 msgr2=0x561d69d934d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d6999fd80 0x561d69d934d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed74014040 tx=0x7fed74011f90 comp rx=0 tx=0).stop 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 -- 172.21.15.39:0/105806776 shutdown_connections 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x561d699e11b0 0x561d69e04880 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d699c04d0 0x561d69e00690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d699b82f0 0x561d69d93a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 --2- 172.21.15.39:0/105806776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561d6999fd80 0x561d69d934d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 -- 172.21.15.39:0/105806776 >> 172.21.15.39:0/105806776 conn(0x561d699d5300 msgr2=0x561d69d811f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed64ff9640 0 client.26717 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.276+0000 7fed8a36f740 1 -- 172.21.15.39:0/105806776 shutdown_connections 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.277+0000 7fed8a36f740 1 -- 172.21.15.39:0/105806776 wait complete. 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.277+0000 7fed8a36f740 20 client.26717 trim_cache size 0 max 16384 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:27:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:27:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 -- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fed680b2860 msgr2=0x7fed680b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fed680b2860 0x7fed680b4d70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed6c000c00 tx=0x7fed6c00ef90 comp rx=0 tx=0).stop 2024-05-04T03:27:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 -- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c10b490 msgr2=0x7fed7c1a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:27:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c10b490 0x7fed7c1a0e00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ea60 tx=0x7fed6c00ef20 comp rx=0 tx=0).stop 2024-05-04T03:27:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 -- 172.21.15.39:0/791161206 shutdown_connections 2024-05-04T03:27:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fed680b2860 0x7fed680b4d70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed7c110190 0x7fed7c1a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c10b490 0x7fed7c1a0e00 secure :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ea60 tx=0x7fed6c00ef20 comp rx=0 tx=0).stop 2024-05-04T03:27:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 --2- 172.21.15.39:0/791161206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c10a450 0x7fed7c1a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:27:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 -- 172.21.15.39:0/791161206 >> 172.21.15.39:0/791161206 conn(0x7fed7c078f10 msgr2=0x7fed7c10c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:27:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 -- 172.21.15.39:0/791161206 shutdown_connections 2024-05-04T03:27:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:27:50.278+0000 7fed833e3640 1 -- 172.21.15.39:0/791161206 wait complete. 2024-05-04T03:27:50.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:50 smithi066 ceph-mon[33264]: pgmap v897: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-05-04T03:27:50.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:50 smithi167 ceph-mon[33614]: pgmap v897: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-05-04T03:27:50.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:50 smithi039 ceph-mon[29715]: pgmap v897: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 1.11k op/s 2024-05-04T03:27:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 840.0 s, 718.6 tps, lat 216.497 ms stddev 840.538 2024-05-04T03:27:52.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:52 smithi066 ceph-mon[33264]: pgmap v898: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 924 op/s 2024-05-04T03:27:52.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:52 smithi167 ceph-mon[33614]: pgmap v898: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 924 op/s 2024-05-04T03:27:52.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:52 smithi039 ceph-mon[29715]: pgmap v898: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 9.4 MiB/s wr, 924 op/s 2024-05-04T03:27:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:54 smithi066 ceph-mon[33264]: pgmap v899: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.0 MiB/s wr, 868 op/s 2024-05-04T03:27:54.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:54 smithi167 ceph-mon[33614]: pgmap v899: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.0 MiB/s wr, 868 op/s 2024-05-04T03:27:54.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:54 smithi039 ceph-mon[29715]: pgmap v899: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.0 MiB/s wr, 868 op/s 2024-05-04T03:27:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 845.0 s, 1275.6 tps, lat 78.723 ms stddev 23.048 2024-05-04T03:27:56.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:56 smithi066 ceph-mon[33264]: pgmap v900: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-05-04T03:27:56.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:56 smithi167 ceph-mon[33614]: pgmap v900: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-05-04T03:27:56.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:56 smithi039 ceph-mon[29715]: pgmap v900: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-05-04T03:27:58.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:27:58 smithi066 ceph-mon[33264]: pgmap v901: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.3 MiB/s wr, 810 op/s 2024-05-04T03:27:58.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:27:58 smithi167 ceph-mon[33614]: pgmap v901: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.3 MiB/s wr, 810 op/s 2024-05-04T03:27:58.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:27:58 smithi039 ceph-mon[29715]: pgmap v901: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.3 MiB/s wr, 810 op/s 2024-05-04T03:28:00.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:00 smithi066 ceph-mon[33264]: pgmap v902: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.7 MiB/s wr, 865 op/s 2024-05-04T03:28:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:00 smithi167 ceph-mon[33614]: pgmap v902: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.7 MiB/s wr, 865 op/s 2024-05-04T03:28:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:00 smithi039 ceph-mon[29715]: pgmap v902: 97 pgs: 97 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.7 MiB/s wr, 865 op/s 2024-05-04T03:28:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 850.0 s, 1122.2 tps, lat 89.032 ms stddev 31.144 2024-05-04T03:28:02.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:02 smithi066 ceph-mon[33264]: pgmap v903: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 10 MiB/s wr, 983 op/s 2024-05-04T03:28:02.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:02 smithi167 ceph-mon[33614]: pgmap v903: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 10 MiB/s wr, 983 op/s 2024-05-04T03:28:02.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:02 smithi039 ceph-mon[29715]: pgmap v903: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 10 MiB/s wr, 983 op/s 2024-05-04T03:28:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 -- 172.21.15.39:0/2133553752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c109bb0 msgr2=0x7ff52c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 --2- 172.21.15.39:0/2133553752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c109bb0 0x7ff52c10bfd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff524009be0 tx=0x7ff52402a6d0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 -- 172.21.15.39:0/2133553752 shutdown_connections 2024-05-04T03:28:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 --2- 172.21.15.39:0/2133553752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c109bb0 0x7ff52c10bfd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 --2- 172.21.15.39:0/2133553752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff52c104eb0 0x7ff52c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 --2- 172.21.15.39:0/2133553752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c103c70 0x7ff52c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 -- 172.21.15.39:0/2133553752 >> 172.21.15.39:0/2133553752 conn(0x7ff52c0fee60 msgr2=0x7ff52c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 -- 172.21.15.39:0/2133553752 shutdown_connections 2024-05-04T03:28:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.487+0000 7ff533966640 1 -- 172.21.15.39:0/2133553752 wait complete. 2024-05-04T03:28:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.488+0000 7ff533966640 1 Processor -- start 2024-05-04T03:28:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.489+0000 7ff533966640 1 -- start start 2024-05-04T03:28:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.489+0000 7ff533966640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c103c70 0x7ff52c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.489+0000 7ff533966640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c104eb0 0x7ff52c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.489+0000 7ff533966640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff52c109bb0 0x7ff52c1a7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.489+0000 7ff533966640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c10f630 con 0x7ff52c109bb0 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.489+0000 7ff533966640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c10f4b0 con 0x7ff52c104eb0 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.489+0000 7ff533966640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c10f7b0 con 0x7ff52c103c70 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff531702640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c103c70 0x7ff52c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff531702640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c103c70 0x7ff52c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:42840/0 (socket says 172.21.15.39:42840) 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff531702640 1 -- 172.21.15.39:0/2860233154 learned_addr learned my addr 172.21.15.39:0/2860233154 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff530f01640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c104eb0 0x7ff52c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff531f03640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff52c109bb0 0x7ff52c1a7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff530f01640 1 -- 172.21.15.39:0/2860233154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c103c70 msgr2=0x7ff52c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff530f01640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c103c70 0x7ff52c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff530f01640 1 -- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff52c109bb0 msgr2=0x7ff52c1a7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff530f01640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff52c109bb0 0x7ff52c1a7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.490+0000 7ff530f01640 1 -- 172.21.15.39:0/2860233154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c1a8210 con 0x7ff52c104eb0 2024-05-04T03:28:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.491+0000 7ff530f01640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c104eb0 0x7ff52c198400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00e790 tx=0x7ff51c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.491+0000 7ff5227fc640 1 -- 172.21.15.39:0/2860233154 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00eea0 con 0x7ff52c104eb0 2024-05-04T03:28:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.491+0000 7ff533966640 1 -- 172.21.15.39:0/2860233154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c1a8500 con 0x7ff52c104eb0 2024-05-04T03:28:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.491+0000 7ff5227fc640 1 -- 172.21.15.39:0/2860233154 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff51c00ccc0 con 0x7ff52c104eb0 2024-05-04T03:28:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.491+0000 7ff5227fc640 1 -- 172.21.15.39:0/2860233154 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c018680 con 0x7ff52c104eb0 2024-05-04T03:28:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.491+0000 7ff533966640 1 -- 172.21.15.39:0/2860233154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c1a89e0 con 0x7ff52c104eb0 2024-05-04T03:28:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.495+0000 7ff5227fc640 1 -- 172.21.15.39:0/2860233154 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7ff51c010430 con 0x7ff52c104eb0 2024-05-04T03:28:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.496+0000 7ff53a817740 1 Processor -- start 2024-05-04T03:28:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.496+0000 7ff5227fc640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5180b28b0 0x7ff5180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.496+0000 7ff53a817740 1 -- start start 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.496+0000 7ff53a817740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff53a817740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531f03640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5180b28b0 0x7ff5180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:40094/0 (socket says 172.21.15.39:40094) 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531702640 1 -- 172.21.15.39:0/313044296 learned_addr learned my addr 172.21.15.39:0/313044296 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff53a817740 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff53a817740 1 -- 172.21.15.39:0/313044296 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560a376395d0 con 0x560a373124d0 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff53a817740 1 -- 172.21.15.39:0/313044296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560a375f2890 con 0x560a373331b0 2024-05-04T03:28:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff53a817740 1 -- 172.21.15.39:0/313044296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560a37635590 con 0x560a3730a2f0 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff5227fc640 1 -- 172.21.15.39:0/2860233154 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff51c0e23c0 con 0x7ff52c104eb0 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff530f01640 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531702640 1 -- 172.21.15.39:0/313044296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 msgr2=0x560a376d4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531702640 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376d4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531702640 1 -- 172.21.15.39:0/313044296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 msgr2=0x560a376dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531702640 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.497+0000 7ff531702640 1 -- 172.21.15.39:0/313044296 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3734b210 con 0x560a373331b0 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.498+0000 7ff531f03640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5180b28b0 0x7ff5180b4dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff524009420 tx=0x7ff5240304c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.498+0000 7ff531702640 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d4a60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff528009d10 tx=0x7ff528011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0780f9ef2b6a9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.498+0000 7ff5177fe640 1 -- 172.21.15.39:0/313044296 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528004220 con 0x560a373331b0 2024-05-04T03:28:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.499+0000 7ff5177fe640 1 -- 172.21.15.39:0/313044296 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5280043c0 con 0x560a373331b0 2024-05-04T03:28:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.499+0000 7ff5177fe640 1 -- 172.21.15.39:0/313044296 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528019650 con 0x560a373331b0 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.500+0000 7ff53a817740 1 -- 172.21.15.39:0/313044296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 msgr2=0x560a376d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.500+0000 7ff53a817740 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d4a60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff528009d10 tx=0x7ff528011810 comp rx=0 tx=0).stop 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.501+0000 7ff53a817740 1 -- 172.21.15.39:0/313044296 shutdown_connections 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.501+0000 7ff53a817740 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.501+0000 7ff53a817740 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.501+0000 7ff53a817740 1 --2- 172.21.15.39:0/313044296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d4a60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.501+0000 7ff53a817740 1 -- 172.21.15.39:0/313044296 >> 172.21.15.39:0/313044296 conn(0x560a37327300 msgr2=0x560a376d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.501+0000 7ff53a817740 1 -- 172.21.15.39:0/313044296 shutdown_connections 2024-05-04T03:28:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.501+0000 7ff53a817740 1 -- 172.21.15.39:0/313044296 wait complete. 2024-05-04T03:28:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.502+0000 7ff53a817740 1 -- start start 2024-05-04T03:28:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.502+0000 7ff53a817740 1 Processor -- start 2024-05-04T03:28:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.504+0000 7ff53a817740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a3730a2f0 0x560a376de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.504+0000 7ff53a817740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a376deac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.504+0000 7ff53a817740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.504+0000 7ff531702640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a3730a2f0 0x560a376de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.504+0000 7ff530f01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a376deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.504+0000 7ff530f01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a376deac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:42864/0 (socket says 172.21.15.39:42864) 2024-05-04T03:28:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.504+0000 7ff531702640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a3730a2f0 0x560a376de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55670/0 (socket says 172.21.15.39:55670) 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.504+0000 7ff530f01640 1 -- 172.21.15.39:0/1303018816 learned_addr learned my addr 172.21.15.39:0/1303018816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560a376dc930 con 0x560a3730a2f0 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x560a375f6370 con 0x560a373331b0 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560a376dcab0 con 0x560a373124d0 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff531f03640 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff53a817740 10 client.0 _ll_register_callbacks cb 0x560a3734aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff531f03640 1 -- 172.21.15.39:0/1303018816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 msgr2=0x560a376deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff531f03640 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a376deac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff531f03640 1 -- 172.21.15.39:0/1303018816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a3730a2f0 msgr2=0x560a376de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff531f03640 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a3730a2f0 0x560a376de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff531f03640 1 -- 172.21.15.39:0/1303018816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a37417620 con 0x560a373331b0 2024-05-04T03:28:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.505+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5280043c0 con 0x560a373331b0 2024-05-04T03:28:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.506+0000 7ff531f03640 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376e5a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff52402acc0 tx=0x7ff524005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdcb907387c5abc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.506+0000 7ff514ff9640 10 client.26729 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:28:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.506+0000 7ff514ff9640 1 -- 172.21.15.39:0/1303018816 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524046040 con 0x560a373331b0 2024-05-04T03:28:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.506+0000 7ff514ff9640 1 -- 172.21.15.39:0/1303018816 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff524004230 con 0x560a373331b0 2024-05-04T03:28:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.508+0000 7ff514ff9640 1 -- 172.21.15.39:0/1303018816 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff524036050 con 0x560a373331b0 2024-05-04T03:28:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.508+0000 7ff53a817740 10 client.26729 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.508+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({fsmap=32}) -- 0x560a376e86b0 con 0x560a373331b0 2024-05-04T03:28:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.509+0000 7ff514ff9640 1 -- 172.21.15.39:0/1303018816 <== mon.2 v2:172.21.15.66:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff524052030 con 0x560a373331b0 2024-05-04T03:28:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.509+0000 7ff53a817740 10 client.26729 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.509+0000 7ff53a817740 10 client.26729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.509+0000 7ff53a817740 20 client.26729 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.509+0000 7ff53a817740 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560a372f1d80 0x560a376ebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.509+0000 7ff53a817740 4 client.26729 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:28:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.509+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a376471e0 con 0x560a372f1d80 2024-05-04T03:28:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.509+0000 7ff531702640 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560a372f1d80 0x560a376ebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.510+0000 7ff514ff9640 10 client.26729 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.510+0000 7ff531702640 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560a372f1d80 0x560a376ebcb0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba07190368040dc server_cookie=b9295d2730a2ef0a in_seq=0 out_seq=0 2024-05-04T03:28:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.511+0000 7ff514ff9640 1 -- 172.21.15.39:0/1303018816 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a376471e0 con 0x560a372f1d80 2024-05-04T03:28:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.511+0000 7ff514ff9640 10 client.26729 handle_command_reply: tid=0 2024-05-04T03:28:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.511+0000 7ff53a817740 1 client.26729 shutdown 2024-05-04T03:28:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.511+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 msgr2=0x560a376e5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.511+0000 7ff53a817740 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376e5a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff52402acc0 tx=0x7ff524005fd0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.512+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 shutdown_connections 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.512+0000 7ff53a817740 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560a372f1d80 0x560a376ebcb0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.512+0000 7ff53a817740 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376e5a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.512+0000 7ff53a817740 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a376deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.512+0000 7ff53a817740 1 --2- 172.21.15.39:0/1303018816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a3730a2f0 0x560a376de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.512+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 >> 172.21.15.39:0/1303018816 conn(0x560a37327300 msgr2=0x560a376cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.512+0000 7ff514ff9640 0 client.26729 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.513+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 shutdown_connections 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.513+0000 7ff53a817740 1 -- 172.21.15.39:0/1303018816 wait complete. 2024-05-04T03:28:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.513+0000 7ff53a817740 20 client.26729 trim_cache size 0 max 16384 2024-05-04T03:28:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.546+0000 7ff53a817740 1 Processor -- start 2024-05-04T03:28:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.546+0000 7ff53a817740 1 -- start start 2024-05-04T03:28:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.546+0000 7ff53a817740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff53a817740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff53a817740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff530f01640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff531702640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff530f01640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376e5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55680/0 (socket says 172.21.15.39:55680) 2024-05-04T03:28:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff530f01640 1 -- 172.21.15.39:0/49509709 learned_addr learned my addr 172.21.15.39:0/49509709 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff531f03640 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff53a817740 1 -- 172.21.15.39:0/49509709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b17b0 con 0x560a373124d0 2024-05-04T03:28:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff53a817740 1 -- 172.21.15.39:0/49509709 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b32f0 con 0x560a373331b0 2024-05-04T03:28:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff53a817740 1 -- 172.21.15.39:0/49509709 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b1e80 con 0x560a3730a2f0 2024-05-04T03:28:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff530f01640 1 -- 172.21.15.39:0/49509709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 msgr2=0x560a376cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.547+0000 7ff530f01640 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff530f01640 1 -- 172.21.15.39:0/49509709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 msgr2=0x560a376d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff530f01640 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff530f01640 1 -- 172.21.15.39:0/49509709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a376d0d00 con 0x560a373124d0 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff531f03640 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff531702640 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff530f01640 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376e5940 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff51c005fd0 tx=0x7ff51c00e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a801c07d7a412 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff516ffd640 1 -- 172.21.15.39:0/49509709 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0aa0f0 con 0x560a373124d0 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff516ffd640 1 -- 172.21.15.39:0/49509709 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff51c0a99f0 con 0x560a373124d0 2024-05-04T03:28:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.548+0000 7ff516ffd640 1 -- 172.21.15.39:0/49509709 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0cc0b0 con 0x560a373124d0 2024-05-04T03:28:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.550+0000 7ff53a817740 1 -- 172.21.15.39:0/49509709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 msgr2=0x560a376e5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.550+0000 7ff53a817740 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376e5940 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff51c005fd0 tx=0x7ff51c00e710 comp rx=0 tx=0).stop 2024-05-04T03:28:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.550+0000 7ff53a817740 1 -- 172.21.15.39:0/49509709 shutdown_connections 2024-05-04T03:28:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.550+0000 7ff53a817740 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a373331b0 0x560a376d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.550+0000 7ff53a817740 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373124d0 0x560a376e5940 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.550+0000 7ff53a817740 1 --2- 172.21.15.39:0/49509709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a3730a2f0 0x560a376cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.550+0000 7ff53a817740 1 -- 172.21.15.39:0/49509709 >> 172.21.15.39:0/49509709 conn(0x560a37327300 msgr2=0x560a376d89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.551+0000 7ff53a817740 1 -- 172.21.15.39:0/49509709 shutdown_connections 2024-05-04T03:28:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.551+0000 7ff53a817740 1 -- 172.21.15.39:0/49509709 wait complete. 2024-05-04T03:28:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.551+0000 7ff53a817740 1 -- start start 2024-05-04T03:28:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.551+0000 7ff53a817740 1 Processor -- start 2024-05-04T03:28:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff53a817740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a3730a2f0 0x560a376e3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff53a817740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a37750010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff53a817740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373331b0 0x560a37752560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff53a817740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b2c20 con 0x560a373331b0 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff53a817740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b2550 con 0x560a3730a2f0 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff53a817740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1af790 con 0x560a373124d0 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff53a817740 10 client.0 _ll_register_callbacks cb 0x7ff51c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff530f01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a37750010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff531702640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a3730a2f0 0x560a376e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff530f01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a37750010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:42884/0 (socket says 172.21.15.39:42884) 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff530f01640 1 -- 172.21.15.39:0/2286370260 learned_addr learned my addr 172.21.15.39:0/2286370260 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.552+0000 7ff531f03640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373331b0 0x560a37752560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.553+0000 7ff530f01640 1 -- 172.21.15.39:0/2286370260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a3730a2f0 msgr2=0x560a376e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.553+0000 7ff530f01640 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a3730a2f0 0x560a376e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.553+0000 7ff530f01640 1 -- 172.21.15.39:0/2286370260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373331b0 msgr2=0x560a37752560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.553+0000 7ff530f01640 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373331b0 0x560a37752560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.553+0000 7ff530f01640 1 -- 172.21.15.39:0/2286370260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a376e5d10 con 0x560a373124d0 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.553+0000 7ff530f01640 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a37750010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0a9980 tx=0x7ff51c092b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a250fbcd5c4563f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.553+0000 7ff514ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.553+0000 7ff514ff9640 1 -- 172.21.15.39:0/2286370260 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0aa0f0 con 0x560a373124d0 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.554+0000 7ff514ff9640 1 -- 172.21.15.39:0/2286370260 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7ff51c0a5fe0 con 0x560a373124d0 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.554+0000 7ff53a817740 1 -- 172.21.15.39:0/2286370260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51c0a99f0 con 0x560a373124d0 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.554+0000 7ff514ff9640 1 -- 172.21.15.39:0/2286370260 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff51c0adda0 con 0x560a373124d0 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff53a817740 10 client.26764 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff53a817740 1 -- 172.21.15.39:0/2286370260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a37753b40 con 0x560a373124d0 2024-05-04T03:28:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff514ff9640 1 -- 172.21.15.39:0/2286370260 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff51c086070 con 0x560a373124d0 2024-05-04T03:28:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff514ff9640 1 -- 172.21.15.39:0/2286370260 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff51c0a5360 con 0x560a373124d0 2024-05-04T03:28:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff53a817740 10 client.26764 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff53a817740 10 client.26764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff53a817740 20 client.26764 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff53a817740 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560a372c71c0 0x560a377563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff53a817740 4 client.26764 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:28:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff53a817740 1 -- 172.21.15.39:0/2286370260 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff51c0a99f0 con 0x560a372c71c0 2024-05-04T03:28:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.555+0000 7ff531702640 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560a372c71c0 0x560a377563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.556+0000 7ff514ff9640 10 client.26764 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.557+0000 7ff531702640 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560a372c71c0 0x560a377563f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416e5ef664809ed2 server_cookie=b4aa6ce86f013b37 in_seq=0 out_seq=0 2024-05-04T03:28:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff514ff9640 1 -- 172.21.15.39:0/2286370260 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff51c0a99f0 con 0x560a372c71c0 2024-05-04T03:28:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff514ff9640 10 client.26764 handle_command_reply: tid=0 2024-05-04T03:28:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 client.26764 shutdown 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 -- 172.21.15.39:0/2286370260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 msgr2=0x560a37750010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a37750010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0a9980 tx=0x7ff51c092b10 comp rx=0 tx=0).stop 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 -- 172.21.15.39:0/2286370260 shutdown_connections 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x560a372c71c0 0x560a377563f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560a373331b0 0x560a37752560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560a373124d0 0x560a37750010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 --2- 172.21.15.39:0/2286370260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560a3730a2f0 0x560a376e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff53a817740 1 -- 172.21.15.39:0/2286370260 >> 172.21.15.39:0/2286370260 conn(0x560a37327300 msgr2=0x560a376cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.566+0000 7ff514ff9640 0 client.26764 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.567+0000 7ff53a817740 1 -- 172.21.15.39:0/2286370260 shutdown_connections 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.567+0000 7ff53a817740 1 -- 172.21.15.39:0/2286370260 wait complete. 2024-05-04T03:28:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.567+0000 7ff53a817740 20 client.26764 trim_cache size 0 max 16384 2024-05-04T03:28:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:28:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:28:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:28:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:28:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 -- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5180b28b0 msgr2=0x7ff5180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5180b28b0 0x7ff5180b4dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff524009420 tx=0x7ff5240304c0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 -- 172.21.15.39:0/2860233154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c104eb0 msgr2=0x7ff52c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c104eb0 0x7ff52c198400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00e790 tx=0x7ff51c00ec50 comp rx=0 tx=0).stop 2024-05-04T03:28:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 -- 172.21.15.39:0/2860233154 shutdown_connections 2024-05-04T03:28:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7ff5180b28b0 0x7ff5180b4dc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff52c109bb0 0x7ff52c1a7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff52c104eb0 0x7ff52c198400 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00e790 tx=0x7ff51c00ec50 comp rx=0 tx=0).stop 2024-05-04T03:28:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 --2- 172.21.15.39:0/2860233154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff52c103c70 0x7ff52c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.568+0000 7ff533966640 1 -- 172.21.15.39:0/2860233154 >> 172.21.15.39:0/2860233154 conn(0x7ff52c0fee60 msgr2=0x7ff52c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.569+0000 7ff533966640 1 -- 172.21.15.39:0/2860233154 shutdown_connections 2024-05-04T03:28:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:03.569+0000 7ff533966640 1 -- 172.21.15.39:0/2860233154 wait complete. 2024-05-04T03:28:04.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:04 smithi066 ceph-mon[33264]: pgmap v904: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-05-04T03:28:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:04 smithi167 ceph-mon[33614]: pgmap v904: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-05-04T03:28:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:04 smithi039 ceph-mon[29715]: pgmap v904: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 1.15k op/s 2024-05-04T03:28:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 855.0 s, 1375.2 tps, lat 72.688 ms stddev 20.648 2024-05-04T03:28:06.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:06 smithi066 ceph-mon[33264]: pgmap v905: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-05-04T03:28:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:06 smithi167 ceph-mon[33614]: pgmap v905: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-05-04T03:28:06.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:06 smithi039 ceph-mon[29715]: pgmap v905: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 1.18k op/s 2024-05-04T03:28:08.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:08 smithi066 ceph-mon[33264]: pgmap v906: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-05-04T03:28:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:08 smithi167 ceph-mon[33614]: pgmap v906: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-05-04T03:28:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:08 smithi039 ceph-mon[29715]: pgmap v906: 97 pgs: 97 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-05-04T03:28:09.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:09 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:28:09.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:09 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:28:09.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:09 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:28:10.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:10 smithi066 ceph-mon[33264]: pgmap v907: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-05-04T03:28:10.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:28:10.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:28:10.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:28:10.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:10 smithi167 ceph-mon[33614]: pgmap v907: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-05-04T03:28:10.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:28:10.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:28:10.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:28:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:10 smithi039 ceph-mon[29715]: pgmap v907: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-05-04T03:28:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:28:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:28:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:28:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 860.0 s, 1350.8 tps, lat 74.149 ms stddev 20.774 2024-05-04T03:28:12.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:12 smithi066 ceph-mon[33264]: pgmap v908: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-05-04T03:28:12.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:12 smithi167 ceph-mon[33614]: pgmap v908: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-05-04T03:28:12.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:12 smithi039 ceph-mon[29715]: pgmap v908: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 14 MiB/s wr, 1.26k op/s 2024-05-04T03:28:14.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:14 smithi066 ceph-mon[33264]: pgmap v909: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-05-04T03:28:14.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:14 smithi167 ceph-mon[33614]: pgmap v909: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-05-04T03:28:14.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:14 smithi039 ceph-mon[29715]: pgmap v909: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-05-04T03:28:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 -- 172.21.15.39:0/3731365325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa0101bd0 msgr2=0x7effa0101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 --2- 172.21.15.39:0/3731365325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa0101bd0 0x7effa0101fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff94009860 tx=0x7eff9402a340 comp rx=0 tx=0).stop 2024-05-04T03:28:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 -- 172.21.15.39:0/3731365325 shutdown_connections 2024-05-04T03:28:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 --2- 172.21.15.39:0/3731365325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effa01034c0 0x7effa0103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 --2- 172.21.15.39:0/3731365325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa0102b70 0x7effa0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 --2- 172.21.15.39:0/3731365325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa0101bd0 0x7effa0101fe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 -- 172.21.15.39:0/3731365325 >> 172.21.15.39:0/3731365325 conn(0x7effa00fd080 msgr2=0x7effa00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 -- 172.21.15.39:0/3731365325 shutdown_connections 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.734+0000 7effa51a8640 1 -- 172.21.15.39:0/3731365325 wait complete. 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.735+0000 7effa51a8640 1 Processor -- start 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7effa51a8640 1 -- start start 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7effa51a8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa0101bd0 0x7effa0071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7effa51a8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effa0102b70 0x7effa00722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7effa51a8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa01034c0 0x7effa0073200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7effa51a8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7effa0111830 con 0x7effa0102b70 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7effa51a8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effa01116b0 con 0x7effa01034c0 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7effa51a8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effa01119b0 con 0x7effa0101bd0 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7eff9ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa0101bd0 0x7effa0071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7eff9ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa0101bd0 0x7effa0071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:48430/0 (socket says 172.21.15.39:48430) 2024-05-04T03:28:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.736+0000 7eff9ed9d640 1 -- 172.21.15.39:0/3294915773 learned_addr learned my addr 172.21.15.39:0/3294915773 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9f59e640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa01034c0 0x7effa0073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9e59c640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effa0102b70 0x7effa00722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9f59e640 1 -- 172.21.15.39:0/3294915773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa0101bd0 msgr2=0x7effa0071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9f59e640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa0101bd0 0x7effa0071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9f59e640 1 -- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effa0102b70 msgr2=0x7effa00722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9f59e640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effa0102b70 0x7effa00722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9f59e640 1 -- 172.21.15.39:0/3294915773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa01111e0 con 0x7effa01034c0 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9e59c640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effa0102b70 0x7effa00722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.737+0000 7eff9f59e640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa01034c0 0x7effa0073200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7eff90009fd0 tx=0x7eff9000eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.738+0000 7eff87fff640 1 -- 172.21.15.39:0/3294915773 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90004220 con 0x7effa01034c0 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.738+0000 7eff87fff640 1 -- 172.21.15.39:0/3294915773 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff900043c0 con 0x7effa01034c0 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.738+0000 7eff87fff640 1 -- 172.21.15.39:0/3294915773 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff900047a0 con 0x7effa01034c0 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.739+0000 7effa51a8640 1 -- 172.21.15.39:0/3294915773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa0073740 con 0x7effa01034c0 2024-05-04T03:28:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.739+0000 7effa51a8640 1 -- 172.21.15.39:0/3294915773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa0073c10 con 0x7effa01034c0 2024-05-04T03:28:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.743+0000 7effac05a740 1 Processor -- start 2024-05-04T03:28:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7eff87fff640 1 -- 172.21.15.39:0/3294915773 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7eff9000c8b0 con 0x7effa01034c0 2024-05-04T03:28:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7effac05a740 1 -- start start 2024-05-04T03:28:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7effac05a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fce1b0 0x559a9a36fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7effac05a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a36ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7effac05a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a376ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7effac05a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559a9a2d45d0 con 0x559a99fad4d0 2024-05-04T03:28:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7effac05a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559a9a28d890 con 0x559a99fce1b0 2024-05-04T03:28:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7effac05a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a9a2d0590 con 0x559a99fa52f0 2024-05-04T03:28:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7eff9e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fce1b0 0x559a9a36fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7eff9e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fce1b0 0x559a9a36fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:36814/0 (socket says 172.21.15.39:36814) 2024-05-04T03:28:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7eff9e59c640 1 -- 172.21.15.39:0/1198009675 learned_addr learned my addr 172.21.15.39:0/1198009675 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.745+0000 7eff9e59c640 1 --2- 172.21.15.39:0/1198009675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a376ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.746+0000 7eff87fff640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7eff880b2930 0x7eff880b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.746+0000 7eff9e59c640 1 -- 172.21.15.39:0/1198009675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 msgr2=0x559a9a36ffe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:28:16.749 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.746+0000 7eff9e59c640 1 --2- 172.21.15.39:0/1198009675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a36ffe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.746+0000 7eff9e59c640 1 -- 172.21.15.39:0/1198009675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 msgr2=0x559a9a376ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.746+0000 7eff9e59c640 1 --2- 172.21.15.39:0/1198009675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a376ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.746+0000 7eff9e59c640 1 -- 172.21.15.39:0/1198009675 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a99fe6210 con 0x559a99fce1b0 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.746+0000 7eff87fff640 1 -- 172.21.15.39:0/3294915773 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7eff900e2690 con 0x7effa01034c0 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.747+0000 7eff9e59c640 1 --2- 172.21.15.39:0/1198009675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fce1b0 0x559a9a36fa60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff98012b90 tx=0x7eff98009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd52a84c060c80f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.747+0000 7eff9f59e640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7eff880b2930 0x7eff880b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.747+0000 7eff857fa640 1 -- 172.21.15.39:0/1198009675 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98012c80 con 0x559a99fce1b0 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.747+0000 7eff857fa640 1 -- 172.21.15.39:0/1198009675 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff98012e20 con 0x559a99fce1b0 2024-05-04T03:28:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.747+0000 7eff857fa640 1 -- 172.21.15.39:0/1198009675 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff980044c0 con 0x559a99fce1b0 2024-05-04T03:28:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.748+0000 7effac05a740 1 -- 172.21.15.39:0/1198009675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fce1b0 msgr2=0x559a9a36fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.748+0000 7effac05a740 1 --2- 172.21.15.39:0/1198009675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fce1b0 0x559a9a36fa60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff98012b90 tx=0x7eff98009ac0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.749+0000 7effac05a740 1 -- 172.21.15.39:0/1198009675 shutdown_connections 2024-05-04T03:28:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.749+0000 7effac05a740 1 --2- 172.21.15.39:0/1198009675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a376ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.749+0000 7effac05a740 1 --2- 172.21.15.39:0/1198009675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a36ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.749+0000 7effac05a740 1 --2- 172.21.15.39:0/1198009675 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fce1b0 0x559a9a36fa60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.749+0000 7effac05a740 1 -- 172.21.15.39:0/1198009675 >> 172.21.15.39:0/1198009675 conn(0x559a99fc2300 msgr2=0x559a9a36b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.749+0000 7effac05a740 1 -- 172.21.15.39:0/1198009675 shutdown_connections 2024-05-04T03:28:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.749+0000 7eff9f59e640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7eff880b2930 0x7eff880b4e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7eff900a6ed0 tx=0x7eff90009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.749+0000 7effac05a740 1 -- 172.21.15.39:0/1198009675 wait complete. 2024-05-04T03:28:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.750+0000 7effac05a740 1 -- start start 2024-05-04T03:28:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.750+0000 7effac05a740 1 Processor -- start 2024-05-04T03:28:16.753 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.751+0000 7effac05a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a379730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.751+0000 7effac05a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a379c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.751+0000 7eff9e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a379730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.751+0000 7eff9e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a379730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:48438/0 (socket says 172.21.15.39:48438) 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.751+0000 7eff9e59c640 1 -- 172.21.15.39:0/2013287637 learned_addr learned my addr 172.21.15.39:0/2013287637 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.751+0000 7effac05a740 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fce1b0 0x559a9a380bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.751+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559a9a377930 con 0x559a99fce1b0 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.751+0000 7eff9ed9d640 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a379c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.752+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x559a9a291370 con 0x559a99fad4d0 2024-05-04T03:28:16.754 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.752+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a9a377ab0 con 0x559a99fa52f0 2024-05-04T03:28:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.752+0000 7effac05a740 10 client.0 _ll_register_callbacks cb 0x559a99fe5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.752+0000 7eff9e59c640 1 -- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 msgr2=0x559a9a379c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:16.755 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.752+0000 7eff9e59c640 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a379c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.752+0000 7eff9e59c640 1 -- 172.21.15.39:0/2013287637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fce1b0 msgr2=0x559a9a380bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.752+0000 7eff9e59c640 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fce1b0 0x559a9a380bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.752+0000 7eff9e59c640 1 -- 172.21.15.39:0/2013287637 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9a0b2620 con 0x559a99fa52f0 2024-05-04T03:28:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.753+0000 7eff9ed9d640 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a379c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:28:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.753+0000 7eff9e59c640 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a379730 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff98011000 tx=0x7eff98009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc2c09496d7f208c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.753+0000 7eff82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:28:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.753+0000 7eff82ff5640 1 -- 172.21.15.39:0/2013287637 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98017040 con 0x559a99fa52f0 2024-05-04T03:28:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.753+0000 7eff82ff5640 1 -- 172.21.15.39:0/2013287637 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7eff98003c30 con 0x559a99fa52f0 2024-05-04T03:28:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.754+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff98012e20 con 0x559a99fa52f0 2024-05-04T03:28:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.754+0000 7eff82ff5640 1 -- 172.21.15.39:0/2013287637 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff98003e30 con 0x559a99fa52f0 2024-05-04T03:28:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.754+0000 7effac05a740 10 client.26773 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.754+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a9a383ac0 con 0x559a99fa52f0 2024-05-04T03:28:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.755+0000 7eff82ff5640 1 -- 172.21.15.39:0/2013287637 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff98012e20 con 0x559a99fa52f0 2024-05-04T03:28:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.755+0000 7eff82ff5640 1 -- 172.21.15.39:0/2013287637 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7eff98005420 con 0x559a99fa52f0 2024-05-04T03:28:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.755+0000 7effac05a740 10 client.26773 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.755+0000 7effac05a740 10 client.26773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.755+0000 7effac05a740 20 client.26773 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.755+0000 7effac05a740 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559a99f8cd80 0x559a9a3874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.755+0000 7effac05a740 4 client.26773 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:28:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.756+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a9a2e21e0 con 0x559a99f8cd80 2024-05-04T03:28:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.756+0000 7eff9ed9d640 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559a99f8cd80 0x559a9a3874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.757+0000 7eff9ed9d640 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559a99f8cd80 0x559a9a3874a0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6764b6866c8548ae server_cookie=bb37cbc42b694188 in_seq=0 out_seq=0 2024-05-04T03:28:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.757+0000 7eff82ff5640 10 client.26773 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.757+0000 7eff82ff5640 1 -- 172.21.15.39:0/2013287637 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a9a2e21e0 con 0x559a99f8cd80 2024-05-04T03:28:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.757+0000 7eff82ff5640 10 client.26773 handle_command_reply: tid=0 2024-05-04T03:28:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.758+0000 7effac05a740 1 client.26773 shutdown 2024-05-04T03:28:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.758+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 msgr2=0x559a9a379730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.758+0000 7effac05a740 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a379730 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff98011000 tx=0x7eff98009fa0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 shutdown_connections 2024-05-04T03:28:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559a99f8cd80 0x559a9a3874a0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fce1b0 0x559a9a380bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7eff82ff5640 0 client.26773 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a379c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 1 --2- 172.21.15.39:0/2013287637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a379730 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 >> 172.21.15.39:0/2013287637 conn(0x559a99fc2300 msgr2=0x559a9a36a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 shutdown_connections 2024-05-04T03:28:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 1 -- 172.21.15.39:0/2013287637 wait complete. 2024-05-04T03:28:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.759+0000 7effac05a740 20 client.26773 trim_cache size 0 max 16384 2024-05-04T03:28:16.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 865.0 s, 1294.6 tps, lat 77.033 ms stddev 19.816 2024-05-04T03:28:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.792+0000 7effac05a740 1 Processor -- start 2024-05-04T03:28:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.792+0000 7effac05a740 1 -- start start 2024-05-04T03:28:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.792+0000 7effac05a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99f8cd80 0x559a9a36b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.792+0000 7effac05a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a36e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.792+0000 7effac05a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a36e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7effac05a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7effa01ba230 con 0x559a99fad4d0 2024-05-04T03:28:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7effac05a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effa01bbd70 con 0x559a99f8cd80 2024-05-04T03:28:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7effac05a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effa01ba900 con 0x559a99fa52f0 2024-05-04T03:28:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a36e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a36e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a36e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33672/0 (socket says 172.21.15.39:33672) 2024-05-04T03:28:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 -- 172.21.15.39:0/724614340 learned_addr learned my addr 172.21.15.39:0/724614340 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 --2- 172.21.15.39:0/724614340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99f8cd80 0x559a9a36b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 -- 172.21.15.39:0/724614340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 msgr2=0x559a9a36e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 --2- 172.21.15.39:0/724614340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a36e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 -- 172.21.15.39:0/724614340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 msgr2=0x559a9a36e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 --2- 172.21.15.39:0/724614340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a36e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.793+0000 7eff9e59c640 1 -- 172.21.15.39:0/724614340 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9a37a870 con 0x559a99f8cd80 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.794+0000 7eff9e59c640 1 --2- 172.21.15.39:0/724614340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99f8cd80 0x559a9a36b470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7eff98004dc0 tx=0x7eff98000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac2bcea52f6ae87d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.794+0000 7eff84ff9640 1 -- 172.21.15.39:0/724614340 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98038070 con 0x559a99f8cd80 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.794+0000 7eff84ff9640 1 -- 172.21.15.39:0/724614340 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff98039040 con 0x559a99f8cd80 2024-05-04T03:28:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.794+0000 7eff84ff9640 1 -- 172.21.15.39:0/724614340 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff980054d0 con 0x559a99f8cd80 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.795+0000 7effac05a740 1 -- 172.21.15.39:0/724614340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99f8cd80 msgr2=0x559a9a36b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.795+0000 7effac05a740 1 --2- 172.21.15.39:0/724614340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99f8cd80 0x559a9a36b470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7eff98004dc0 tx=0x7eff98000f40 comp rx=0 tx=0).stop 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.795+0000 7effac05a740 1 -- 172.21.15.39:0/724614340 shutdown_connections 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.795+0000 7effac05a740 1 --2- 172.21.15.39:0/724614340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99fad4d0 0x559a9a36e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.795+0000 7effac05a740 1 --2- 172.21.15.39:0/724614340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a36e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.795+0000 7effac05a740 1 --2- 172.21.15.39:0/724614340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99f8cd80 0x559a9a36b470 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.795+0000 7effac05a740 1 -- 172.21.15.39:0/724614340 >> 172.21.15.39:0/724614340 conn(0x559a99fc2300 msgr2=0x559a9a36ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.796+0000 7effac05a740 1 -- 172.21.15.39:0/724614340 shutdown_connections 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.796+0000 7effac05a740 1 -- 172.21.15.39:0/724614340 wait complete. 2024-05-04T03:28:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.796+0000 7effac05a740 1 -- start start 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.796+0000 7effac05a740 1 Processor -- start 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7effac05a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99f8cd80 0x559a9a37d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7effac05a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a37d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7effac05a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a3ed720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7effac05a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7effa01bb6a0 con 0x559a99f8cd80 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7effac05a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effa01bafd0 con 0x559a99fad4d0 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7effac05a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effa01b8210 con 0x559a99fa52f0 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7effac05a740 10 client.0 _ll_register_callbacks cb 0x7eff98012680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a37d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99f8cd80 0x559a9a37d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a37d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:48450/0 (socket says 172.21.15.39:48450) 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9ed9d640 1 -- 172.21.15.39:0/2811505827 learned_addr learned my addr 172.21.15.39:0/2811505827 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9e59c640 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a3ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9ed9d640 1 -- 172.21.15.39:0/2811505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 msgr2=0x559a9a3ed720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9ed9d640 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a3ed720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9ed9d640 1 -- 172.21.15.39:0/2811505827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99f8cd80 msgr2=0x559a9a37d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9ed9d640 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99f8cd80 0x559a9a37d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.797+0000 7eff9ed9d640 1 -- 172.21.15.39:0/2811505827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9a3edc60 con 0x559a99fa52f0 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.798+0000 7eff9ed9d640 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a37d750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7eff94005cd0 tx=0x7eff94030670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09a87c44027e0b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.798+0000 7eff82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.798+0000 7eff82ff5640 1 -- 172.21.15.39:0/2811505827 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94037040 con 0x559a99fa52f0 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.798+0000 7eff82ff5640 1 -- 172.21.15.39:0/2811505827 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7eff94030cb0 con 0x559a99fa52f0 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.798+0000 7effac05a740 1 -- 172.21.15.39:0/2811505827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff98039040 con 0x559a99fa52f0 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.798+0000 7eff82ff5640 1 -- 172.21.15.39:0/2811505827 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff94030820 con 0x559a99fa52f0 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.798+0000 7effac05a740 10 client.26779 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.798+0000 7effac05a740 1 -- 172.21.15.39:0/2811505827 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a9a3eeed0 con 0x559a99fa52f0 2024-05-04T03:28:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.799+0000 7eff82ff5640 1 -- 172.21.15.39:0/2811505827 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff98039040 con 0x559a99fa52f0 2024-05-04T03:28:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.800+0000 7eff82ff5640 1 -- 172.21.15.39:0/2811505827 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7eff9404b030 con 0x559a99fa52f0 2024-05-04T03:28:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.800+0000 7effac05a740 10 client.26779 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.800+0000 7effac05a740 10 client.26779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.800+0000 7effac05a740 20 client.26779 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.800+0000 7effac05a740 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559a99fce1b0 0x559a9a3f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.800+0000 7effac05a740 4 client.26779 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:28:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.800+0000 7effac05a740 1 -- 172.21.15.39:0/2811505827 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a9a2dc050 con 0x559a99fce1b0 2024-05-04T03:28:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.800+0000 7eff9e59c640 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559a99fce1b0 0x559a9a3f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.801+0000 7eff9e59c640 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559a99fce1b0 0x559a9a3f18f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a44363c7f78ff95 server_cookie=9900725cf4a90e5e in_seq=0 out_seq=0 2024-05-04T03:28:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.801+0000 7eff82ff5640 10 client.26779 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7eff82ff5640 1 -- 172.21.15.39:0/2811505827 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a9a2dc050 con 0x559a99fce1b0 2024-05-04T03:28:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7eff82ff5640 10 client.26779 handle_command_reply: tid=0 2024-05-04T03:28:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 client.26779 shutdown 2024-05-04T03:28:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 -- 172.21.15.39:0/2811505827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 msgr2=0x559a9a37d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a37d750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7eff94005cd0 tx=0x7eff94030670 comp rx=0 tx=0).stop 2024-05-04T03:28:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 -- 172.21.15.39:0/2811505827 shutdown_connections 2024-05-04T03:28:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x559a99fce1b0 0x559a9a3f18f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a99fad4d0 0x559a9a3ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a99fa52f0 0x559a9a37d750 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7eff94005cd0 tx=0x7eff94030670 comp rx=0 tx=0).stop 2024-05-04T03:28:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 --2- 172.21.15.39:0/2811505827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559a99f8cd80 0x559a9a37d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7effac05a740 1 -- 172.21.15.39:0/2811505827 >> 172.21.15.39:0/2811505827 conn(0x559a99fc2300 msgr2=0x559a9a369970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.812+0000 7eff82ff5640 0 client.26779 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.813+0000 7effac05a740 1 -- 172.21.15.39:0/2811505827 shutdown_connections 2024-05-04T03:28:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.813+0000 7effac05a740 1 -- 172.21.15.39:0/2811505827 wait complete. 2024-05-04T03:28:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.813+0000 7effac05a740 20 client.26779 trim_cache size 0 max 16384 2024-05-04T03:28:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:28:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:28:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:28:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:28:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 -- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7eff880b2930 msgr2=0x7eff880b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7eff880b2930 0x7eff880b4e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7eff900a6ed0 tx=0x7eff90009600 comp rx=0 tx=0).stop 2024-05-04T03:28:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 -- 172.21.15.39:0/3294915773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa01034c0 msgr2=0x7effa0073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa01034c0 0x7effa0073200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7eff90009fd0 tx=0x7eff9000eba0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 -- 172.21.15.39:0/3294915773 shutdown_connections 2024-05-04T03:28:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7eff880b2930 0x7eff880b4e40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effa01034c0 0x7effa0073200 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effa0102b70 0x7effa00722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 --2- 172.21.15.39:0/3294915773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa0101bd0 0x7effa0071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 -- 172.21.15.39:0/3294915773 >> 172.21.15.39:0/3294915773 conn(0x7effa00fd080 msgr2=0x7effa01090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 -- 172.21.15.39:0/3294915773 shutdown_connections 2024-05-04T03:28:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:16.814+0000 7effa51a8640 1 -- 172.21.15.39:0/3294915773 wait complete. 2024-05-04T03:28:16.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:16 smithi066 ceph-mon[33264]: pgmap v910: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-05-04T03:28:16.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:16 smithi167 ceph-mon[33614]: pgmap v910: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-05-04T03:28:16.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:16 smithi039 ceph-mon[29715]: pgmap v910: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 1.31k op/s 2024-05-04T03:28:18.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:18 smithi066 ceph-mon[33264]: pgmap v911: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-05-04T03:28:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:18 smithi167 ceph-mon[33614]: pgmap v911: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-05-04T03:28:18.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:18 smithi039 ceph-mon[29715]: pgmap v911: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 1.44k op/s 2024-05-04T03:28:20.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:20 smithi066 ceph-mon[33264]: pgmap v912: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:28:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:20 smithi167 ceph-mon[33614]: pgmap v912: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:28:20.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:20 smithi039 ceph-mon[29715]: pgmap v912: 97 pgs: 97 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 1.36k op/s 2024-05-04T03:28:21.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 870.0 s, 1300.2 tps, lat 77.071 ms stddev 24.444 2024-05-04T03:28:22.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:22 smithi066 ceph-mon[33264]: pgmap v913: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.29k op/s 2024-05-04T03:28:22.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:22 smithi167 ceph-mon[33614]: pgmap v913: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.29k op/s 2024-05-04T03:28:22.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:22 smithi039 ceph-mon[29715]: pgmap v913: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 12 MiB/s wr, 1.29k op/s 2024-05-04T03:28:24.632 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-04T03:28:24.632 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.9 2024-05-04T03:28:24.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:28:24.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:24 smithi066 ceph-mon[33264]: pgmap v914: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.43k op/s 2024-05-04T03:28:24.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:24 smithi167 ceph-mon[33614]: pgmap v914: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.43k op/s 2024-05-04T03:28:24.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:24 smithi039 ceph-mon[29715]: pgmap v914: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.43k op/s 2024-05-04T03:28:25.062 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:28:26.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 875.0 s, 1348.8 tps, lat 74.137 ms stddev 22.441 2024-05-04T03:28:26.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:26 smithi066 ceph-mon[33264]: pgmap v915: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 1.29k op/s 2024-05-04T03:28:26.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:26 smithi039 ceph-mon[29715]: pgmap v915: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 1.29k op/s 2024-05-04T03:28:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:26 smithi167 ceph-mon[33614]: pgmap v915: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 MiB/s wr, 1.29k op/s 2024-05-04T03:28:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.915+0000 7f1fa4602700 1 -- 172.21.15.39:0/1800589062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c110970 msgr2=0x7f1f9c110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.915+0000 7f1fa4602700 1 --2- 172.21.15.39:0/1800589062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c110970 0x7f1f9c110d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b0a0 tx=0x7f1f9400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:28:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.915+0000 7f1fa4602700 1 -- 172.21.15.39:0/1800589062 shutdown_connections 2024-05-04T03:28:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.915+0000 7f1fa4602700 1 --2- 172.21.15.39:0/1800589062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c10b2e0 0x7f1f9c117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.915+0000 7f1fa4602700 1 --2- 172.21.15.39:0/1800589062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c110970 0x7f1f9c110d80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.915+0000 7f1fa4602700 1 --2- 172.21.15.39:0/1800589062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c10a990 0x7f1f9c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.915+0000 7f1fa4602700 1 -- 172.21.15.39:0/1800589062 >> 172.21.15.39:0/1800589062 conn(0x7f1f9c077c80 msgr2=0x7f1f9c0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.915+0000 7f1fa4602700 1 -- 172.21.15.39:0/1800589062 shutdown_connections 2024-05-04T03:28:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.916+0000 7f1fa4602700 1 -- 172.21.15.39:0/1800589062 wait complete. 2024-05-04T03:28:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.917+0000 7f1fa4602700 1 Processor -- start 2024-05-04T03:28:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.917+0000 7f1fa4602700 1 -- start start 2024-05-04T03:28:27.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa4602700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c10a990 0x7f1f9c1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:27.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa4602700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c10b2e0 0x7f1f9c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:27.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa4602700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c110970 0x7f1f9c1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:27.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa4602700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c11a9d0 con 0x7f1f9c110970 2024-05-04T03:28:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa4602700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c11a850 con 0x7f1f9c10b2e0 2024-05-04T03:28:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa4602700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c11ab50 con 0x7f1f9c10a990 2024-05-04T03:28:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa1b9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c10b2e0 0x7f1f9c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa239e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c10a990 0x7f1f9c1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa1b9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c10b2e0 0x7f1f9c1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37816/0 (socket says 172.21.15.39:37816) 2024-05-04T03:28:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa1b9d700 1 -- 172.21.15.39:0/3768593606 learned_addr learned my addr 172.21.15.39:0/3768593606 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.918+0000 7f1fa2b9f700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c110970 0x7f1f9c1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.919+0000 7f1fa2b9f700 1 -- 172.21.15.39:0/3768593606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c10a990 msgr2=0x7f1f9c1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.919+0000 7f1fa2b9f700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c10a990 0x7f1f9c1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.919+0000 7f1fa2b9f700 1 -- 172.21.15.39:0/3768593606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c10b2e0 msgr2=0x7f1f9c1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.919+0000 7f1fa2b9f700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c10b2e0 0x7f1f9c1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.919+0000 7f1fa2b9f700 1 -- 172.21.15.39:0/3768593606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c1a60b0 con 0x7f1f9c110970 2024-05-04T03:28:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.919+0000 7f1fa2b9f700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c110970 0x7f1f9c1a5a10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c7a0 tx=0x7f1f8c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.920+0000 7f1f937fe700 1 -- 172.21.15.39:0/3768593606 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c004a30 con 0x7f1f9c110970 2024-05-04T03:28:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.920+0000 7f1f937fe700 1 -- 172.21.15.39:0/3768593606 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f8c0054b0 con 0x7f1f9c110970 2024-05-04T03:28:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.920+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c1a6340 con 0x7f1f9c110970 2024-05-04T03:28:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.920+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1adcc0 con 0x7f1f9c110970 2024-05-04T03:28:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.920+0000 7f1f937fe700 1 -- 172.21.15.39:0/3768593606 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00eb10 con 0x7f1f9c110970 2024-05-04T03:28:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.921+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f80005330 con 0x7f1f9c110970 2024-05-04T03:28:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.924+0000 7f1f937fe700 1 -- 172.21.15.39:0/3768593606 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f1f8c004bd0 con 0x7f1f9c110970 2024-05-04T03:28:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.925+0000 7f1f937fe700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b29c0 0x7f1f880b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:27.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.926+0000 7f1f937fe700 1 -- 172.21.15.39:0/3768593606 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f1f8c0e1f20 con 0x7f1f9c110970 2024-05-04T03:28:27.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.931+0000 7f1fa239e700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b29c0 0x7f1f880b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:27.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.932+0000 7f1fa239e700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b29c0 0x7f1f880b4ed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009e60 tx=0x7f1f940099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:27.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:27.933+0000 7f1f937fe700 1 -- 172.21.15.39:0/3768593606 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f8c005660 con 0x7f1f9c110970 2024-05-04T03:28:28.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.417+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f80005620 con 0x7f1f9c110970 2024-05-04T03:28:28.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.420+0000 7f1f937fe700 1 -- 172.21.15.39:0/3768593606 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f1f8c0a9d10 con 0x7f1f9c110970 2024-05-04T03:28:28.423 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:28:28.423 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:28:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.423+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b29c0 msgr2=0x7f1f880b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.423+0000 7f1fa4602700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b29c0 0x7f1f880b4ed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009e60 tx=0x7f1f940099e0 comp rx=0 tx=0).stop 2024-05-04T03:28:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.423+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c110970 msgr2=0x7f1f9c1a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.423+0000 7f1fa4602700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c110970 0x7f1f9c1a5a10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c7a0 tx=0x7f1f8c00cab0 comp rx=0 tx=0).stop 2024-05-04T03:28:28.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.424+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 shutdown_connections 2024-05-04T03:28:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.424+0000 7f1fa4602700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f1f880b29c0 0x7f1f880b4ed0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.424+0000 7f1fa4602700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9c110970 0x7f1f9c1a5a10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.424+0000 7f1fa4602700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f9c10a990 0x7f1f9c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.424+0000 7f1fa4602700 1 --2- 172.21.15.39:0/3768593606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f9c10b2e0 0x7f1f9c1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.424+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 >> 172.21.15.39:0/3768593606 conn(0x7f1f9c077c80 msgr2=0x7f1f9c07b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.425+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 shutdown_connections 2024-05-04T03:28:28.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:28.425+0000 7f1fa4602700 1 -- 172.21.15.39:0/3768593606 wait complete. 2024-05-04T03:28:28.429 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:28:28.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:28 smithi039 ceph-mon[29715]: pgmap v916: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-05-04T03:28:28.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:28 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3768593606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:28:28.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:28 smithi066 ceph-mon[33264]: pgmap v916: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-05-04T03:28:28.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:28 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3768593606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:28:28.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:28 smithi167 ceph-mon[33614]: pgmap v916: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 1.37k op/s 2024-05-04T03:28:28.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:28 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3768593606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:28:29.172 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 80.9 --await-for 42.0' 2024-05-04T03:28:29.172 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 80.9 --await-for 42.0 2024-05-04T03:28:29.587 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:28:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.994+0000 7f6174d52640 1 -- 172.21.15.39:0/3057815163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616809e200 msgr2=0x7f616809e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.994+0000 7f6174d52640 1 --2- 172.21.15.39:0/3057815163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616809e200 0x7f616809e610 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f615c009860 tx=0x7f615c02a340 comp rx=0 tx=0).stop 2024-05-04T03:28:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.994+0000 7f6174d52640 1 -- 172.21.15.39:0/3057815163 shutdown_connections 2024-05-04T03:28:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.994+0000 7f6174d52640 1 --2- 172.21.15.39:0/3057815163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61680a2f00 0x7f61680a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.994+0000 7f6174d52640 1 --2- 172.21.15.39:0/3057815163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616809e200 0x7f616809e610 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.994+0000 7f6174d52640 1 --2- 172.21.15.39:0/3057815163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616809d360 0x7f616809dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.994+0000 7f6174d52640 1 -- 172.21.15.39:0/3057815163 >> 172.21.15.39:0/3057815163 conn(0x7f616800bbe0 msgr2=0x7f616800c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.995+0000 7f6174d52640 1 -- 172.21.15.39:0/3057815163 shutdown_connections 2024-05-04T03:28:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.995+0000 7f6174d52640 1 -- 172.21.15.39:0/3057815163 wait complete. 2024-05-04T03:28:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.997+0000 7f6174d52640 1 Processor -- start 2024-05-04T03:28:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.997+0000 7f6174d52640 1 -- start start 2024-05-04T03:28:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.997+0000 7f6174d52640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f616809d360 0x7f61681335e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.997+0000 7f6174d52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616809e200 0x7f6168133b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f6174d52640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61680a2f00 0x7f616813aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f616809d360 0x7f61681335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f616f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f616809d360 0x7f61681335e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53868/0 (socket says 172.21.15.39:53868) 2024-05-04T03:28:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f616f7fe640 1 -- 172.21.15.39:0/2544031715 learned_addr learned my addr 172.21.15.39:0/2544031715 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f616effd640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616809e200 0x7f6168133b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f616ffff640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61680a2f00 0x7f616813aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61680a89b0 con 0x7f61680a2f00 2024-05-04T03:28:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61680a8830 con 0x7f616809e200 2024-05-04T03:28:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.998+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61680a8b30 con 0x7f616809d360 2024-05-04T03:28:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.999+0000 7f616f7fe640 1 -- 172.21.15.39:0/2544031715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616809e200 msgr2=0x7f6168133b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.999+0000 7f616f7fe640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616809e200 0x7f6168133b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.999+0000 7f616f7fe640 1 -- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61680a2f00 msgr2=0x7f616813aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.999+0000 7f616f7fe640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61680a2f00 0x7f616813aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:29.999+0000 7f616f7fe640 1 -- 172.21.15.39:0/2544031715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616813b070 con 0x7f616809d360 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.000+0000 7f616ffff640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61680a2f00 0x7f616813aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.000+0000 7f616f7fe640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f616809d360 0x7f61681335e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f61640058c0 tx=0x7f616400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.000+0000 7f616cff9640 1 -- 172.21.15.39:0/2544031715 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164017020 con 0x7f616809d360 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.001+0000 7f616cff9640 1 -- 172.21.15.39:0/2544031715 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6164009c30 con 0x7f616809d360 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.001+0000 7f616cff9640 1 -- 172.21.15.39:0/2544031715 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616400f620 con 0x7f616809d360 2024-05-04T03:28:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.001+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616813b360 con 0x7f616809d360 2024-05-04T03:28:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.001+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616813b830 con 0x7f616809d360 2024-05-04T03:28:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.004+0000 7f617ce62740 1 Processor -- start 2024-05-04T03:28:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.004+0000 7f617ce62740 1 -- start start 2024-05-04T03:28:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f617ce62740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440cc21b0 0x55c441063c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f617ce62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c4410641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f617ce62740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440ca14d0 0x55c44106b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f617ce62740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c440fc85d0 con 0x55c440ca14d0 2024-05-04T03:28:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f617ce62740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c440f81890 con 0x55c440cc21b0 2024-05-04T03:28:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f617ce62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c440fc4590 con 0x55c440c992f0 2024-05-04T03:28:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f616ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440cc21b0 0x55c441063c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f616ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440cc21b0 0x55c441063c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37832/0 (socket says 172.21.15.39:37832) 2024-05-04T03:28:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f616ffff640 1 -- 172.21.15.39:0/3231649901 learned_addr learned my addr 172.21.15.39:0/3231649901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f616cff9640 1 -- 172.21.15.39:0/2544031715 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f616401a020 con 0x7f616809d360 2024-05-04T03:28:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f616f7fe640 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c4410641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.005+0000 7f616effd640 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440ca14d0 0x55c44106b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f616ffff640 1 -- 172.21.15.39:0/3231649901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 msgr2=0x55c4410641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f616ffff640 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c4410641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f616ffff640 1 -- 172.21.15.39:0/3231649901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440ca14d0 msgr2=0x55c44106b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f616ffff640 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440ca14d0 0x55c44106b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f616ffff640 1 -- 172.21.15.39:0/3231649901 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c440cda210 con 0x55c440cc21b0 2024-05-04T03:28:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f616effd640 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440ca14d0 0x55c44106b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:28:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f616ffff640 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440cc21b0 0x55c441063c30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6170051dc0 tx=0x7f617006c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f15b6fc1918a4c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f6159ffb640 1 -- 172.21.15.39:0/3231649901 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170082070 con 0x55c440cc21b0 2024-05-04T03:28:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f6159ffb640 1 -- 172.21.15.39:0/3231649901 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f617007ecc0 con 0x55c440cc21b0 2024-05-04T03:28:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f616cff9640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f61600b2940 0x7f61600b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.006+0000 7f6159ffb640 1 -- 172.21.15.39:0/3231649901 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617007db10 con 0x55c440cc21b0 2024-05-04T03:28:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.007+0000 7f616cff9640 1 -- 172.21.15.39:0/2544031715 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f61640e20a0 con 0x7f616809d360 2024-05-04T03:28:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.007+0000 7f616f7fe640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f61600b2940 0x7f61600b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 -- 172.21.15.39:0/3231649901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440cc21b0 msgr2=0x55c441063c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440cc21b0 0x55c441063c30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6170051dc0 tx=0x7f617006c4f0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 -- 172.21.15.39:0/3231649901 shutdown_connections 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440ca14d0 0x55c44106b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c4410641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 --2- 172.21.15.39:0/3231649901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440cc21b0 0x55c441063c30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 -- 172.21.15.39:0/3231649901 >> 172.21.15.39:0/3231649901 conn(0x55c440cb6300 msgr2=0x55c44105f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f616f7fe640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f61600b2940 0x7f61600b4e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6164000c00 tx=0x7f616400bf60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 -- 172.21.15.39:0/3231649901 shutdown_connections 2024-05-04T03:28:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.008+0000 7f617ce62740 1 -- 172.21.15.39:0/3231649901 wait complete. 2024-05-04T03:28:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.009+0000 7f617ce62740 1 -- start start 2024-05-04T03:28:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f617ce62740 1 Processor -- start 2024-05-04T03:28:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f617ce62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c44106d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f617ce62740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f617ce62740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440cc21b0 0x55c441074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f617ce62740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c44106bb00 con 0x55c440cc21b0 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f617ce62740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55c440f85370 con 0x55c440ca14d0 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f617ce62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c44106bc80 con 0x55c440c992f0 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f617ce62740 10 client.0 _ll_register_callbacks cb 0x55c440cd9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f616ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c44106d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f616ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c44106d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53880/0 (socket says 172.21.15.39:53880) 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f616ffff640 1 -- 172.21.15.39:0/3006125493 learned_addr learned my addr 172.21.15.39:0/3006125493 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f616ffff640 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440cc21b0 0x55c441074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.010+0000 7f616effd640 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.011+0000 7f616effd640 1 -- 172.21.15.39:0/3006125493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 msgr2=0x55c44106d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.011+0000 7f616effd640 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c44106d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.011+0000 7f616effd640 1 -- 172.21.15.39:0/3006125493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440cc21b0 msgr2=0x55c441074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.011+0000 7f616effd640 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440cc21b0 0x55c441074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.011+0000 7f616effd640 1 -- 172.21.15.39:0/3006125493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c440da6620 con 0x55c440ca14d0 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.011+0000 7f616ffff640 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c44106d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.012+0000 7f614effd640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.012+0000 7f616effd640 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106dd90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f615c02a860 tx=0x7f615c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e095212abb180e49 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.012+0000 7f614effd640 1 -- 172.21.15.39:0/3006125493 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c037040 con 0x55c440ca14d0 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.012+0000 7f614effd640 1 -- 172.21.15.39:0/3006125493 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f615c005230 con 0x55c440ca14d0 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.012+0000 7f617ce62740 1 -- 172.21.15.39:0/3006125493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f617007ecc0 con 0x55c440ca14d0 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.012+0000 7f614effd640 1 -- 172.21.15.39:0/3006125493 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f615c005430 con 0x55c440ca14d0 2024-05-04T03:28:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.013+0000 7f617ce62740 10 client.26768 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.013+0000 7f617ce62740 1 -- 172.21.15.39:0/3006125493 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4410779d0 con 0x55c440ca14d0 2024-05-04T03:28:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.013+0000 7f614effd640 1 -- 172.21.15.39:0/3006125493 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f617007ecc0 con 0x55c440ca14d0 2024-05-04T03:28:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.013+0000 7f614effd640 1 -- 172.21.15.39:0/3006125493 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f615c0047e0 con 0x55c440ca14d0 2024-05-04T03:28:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.014+0000 7f617ce62740 10 client.26768 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.014+0000 7f617ce62740 10 client.26768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.014+0000 7f617ce62740 20 client.26768 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.014+0000 7f617ce62740 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c440c80d80 0x55c44107b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.014+0000 7f617ce62740 4 client.26768 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:28:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.014+0000 7f617ce62740 1 -- 172.21.15.39:0/3006125493 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c440fd61e0 con 0x55c440c80d80 2024-05-04T03:28:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.014+0000 7f616ffff640 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c440c80d80 0x55c44107b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.015+0000 7f616ffff640 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c440c80d80 0x55c44107b140 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce2650593256025 server_cookie=a4d12cd02e681a60 in_seq=0 out_seq=0 2024-05-04T03:28:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.015+0000 7f614effd640 10 client.26768 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.018+0000 7f614effd640 1 -- 172.21.15.39:0/3006125493 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c440fd61e0 con 0x55c440c80d80 2024-05-04T03:28:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.018+0000 7f614effd640 10 client.26768 handle_command_reply: tid=0 2024-05-04T03:28:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.018+0000 7f617ce62740 1 client.26768 shutdown 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.021+0000 7f617ce62740 1 -- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 msgr2=0x55c44106dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.021+0000 7f617ce62740 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106dd90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f615c02a860 tx=0x7f615c009510 comp rx=0 tx=0).stop 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.021+0000 7f617ce62740 1 -- 172.21.15.39:0/3006125493 shutdown_connections 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.021+0000 7f617ce62740 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c440c80d80 0x55c44107b140 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.021+0000 7f614effd640 0 client.26768 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.021+0000 7f617ce62740 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440cc21b0 0x55c441074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.021+0000 7f617ce62740 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106dd90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.022+0000 7f617ce62740 1 --2- 172.21.15.39:0/3006125493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c44106d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.022+0000 7f617ce62740 1 -- 172.21.15.39:0/3006125493 >> 172.21.15.39:0/3006125493 conn(0x55c440cb6300 msgr2=0x55c4410624d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.022+0000 7f617ce62740 1 -- 172.21.15.39:0/3006125493 shutdown_connections 2024-05-04T03:28:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.022+0000 7f617ce62740 1 -- 172.21.15.39:0/3006125493 wait complete. 2024-05-04T03:28:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.023+0000 7f617ce62740 20 client.26768 trim_cache size 0 max 16384 2024-05-04T03:28:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.054+0000 7f617ce62740 1 Processor -- start 2024-05-04T03:28:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.054+0000 7f617ce62740 1 -- start start 2024-05-04T03:28:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.055+0000 7f617ce62740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c44106cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.055+0000 7f617ce62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441074c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.055+0000 7f617ce62740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.055+0000 7f616ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c44106cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.055+0000 7f616ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c44106cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48612/0 (socket says 172.21.15.39:48612) 2024-05-04T03:28:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.055+0000 7f616effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441074c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.055+0000 7f616effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441074c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53888/0 (socket says 172.21.15.39:53888) 2024-05-04T03:28:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f617ce62740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6168144560 con 0x55c440c80d80 2024-05-04T03:28:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f617ce62740 1 -- 172.21.15.39:0/350761757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61681460a0 con 0x55c440ca14d0 2024-05-04T03:28:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f616ffff640 1 -- 172.21.15.39:0/350761757 learned_addr learned my addr 172.21.15.39:0/350761757 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f617ce62740 1 -- 172.21.15.39:0/350761757 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6168144c30 con 0x55c440c992f0 2024-05-04T03:28:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f616ffff640 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f616ffff640 1 -- 172.21.15.39:0/350761757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 msgr2=0x55c441074c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f616ffff640 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441074c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f616ffff640 1 -- 172.21.15.39:0/350761757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 msgr2=0x55c44106d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f616ffff640 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f616ffff640 1 -- 172.21.15.39:0/350761757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44105ed60 con 0x55c440c80d80 2024-05-04T03:28:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.056+0000 7f616effd640 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441074c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:28:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.057+0000 7f616ffff640 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c44106cb00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6170052170 tx=0x7f617006c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f7aa8601dd46a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.057+0000 7f61597fa640 1 -- 172.21.15.39:0/350761757 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61700a1070 con 0x55c440c80d80 2024-05-04T03:28:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.057+0000 7f61597fa640 1 -- 172.21.15.39:0/350761757 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6170083040 con 0x55c440c80d80 2024-05-04T03:28:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.058+0000 7f61597fa640 1 -- 172.21.15.39:0/350761757 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170082040 con 0x55c440c80d80 2024-05-04T03:28:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 -- 172.21.15.39:0/350761757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 msgr2=0x55c44106cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c44106cb00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6170052170 tx=0x7f617006c680 comp rx=0 tx=0).stop 2024-05-04T03:28:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 -- 172.21.15.39:0/350761757 shutdown_connections 2024-05-04T03:28:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c44106d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441074c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 --2- 172.21.15.39:0/350761757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c44106cb00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 -- 172.21.15.39:0/350761757 >> 172.21.15.39:0/350761757 conn(0x55c440cb6300 msgr2=0x55c441067c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 -- 172.21.15.39:0/350761757 shutdown_connections 2024-05-04T03:28:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.059+0000 7f617ce62740 1 -- 172.21.15.39:0/350761757 wait complete. 2024-05-04T03:28:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.060+0000 7f617ce62740 1 -- start start 2024-05-04T03:28:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.060+0000 7f617ce62740 1 Processor -- start 2024-05-04T03:28:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f617ce62740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c441074a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f617ce62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f617ce62740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c441071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f617ce62740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61681459d0 con 0x55c440c80d80 2024-05-04T03:28:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f617ce62740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6168145300 con 0x55c440ca14d0 2024-05-04T03:28:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f617ce62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6168142540 con 0x55c440c992f0 2024-05-04T03:28:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f617ce62740 10 client.0 _ll_register_callbacks cb 0x7f615c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f616effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f616effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441071680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53894/0 (socket says 172.21.15.39:53894) 2024-05-04T03:28:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f616effd640 1 -- 172.21.15.39:0/1360762348 learned_addr learned my addr 172.21.15.39:0/1360762348 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f616ffff640 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c441071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.061+0000 7f616ffff640 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c441074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.062+0000 7f616effd640 1 -- 172.21.15.39:0/1360762348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 msgr2=0x55c441071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.062+0000 7f616effd640 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c441071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.062+0000 7f616effd640 1 -- 172.21.15.39:0/1360762348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 msgr2=0x55c441074a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.062+0000 7f616effd640 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c441074a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.062+0000 7f616effd640 1 -- 172.21.15.39:0/1360762348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4410e14e0 con 0x55c440c992f0 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.062+0000 7f616effd640 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441071680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f615c02d020 tx=0x7f615c02aac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=480f09248608679 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.062+0000 7f614effd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.063+0000 7f614effd640 1 -- 172.21.15.39:0/1360762348 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c055060 con 0x55c440c992f0 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.063+0000 7f614effd640 1 -- 172.21.15.39:0/1360762348 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f615c005060 con 0x55c440c992f0 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.063+0000 7f617ce62740 1 -- 172.21.15.39:0/1360762348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6170083040 con 0x55c440c992f0 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.063+0000 7f614effd640 1 -- 172.21.15.39:0/1360762348 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f615c037030 con 0x55c440c992f0 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.063+0000 7f617ce62740 10 client.26800 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.063+0000 7f617ce62740 1 -- 172.21.15.39:0/1360762348 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4410e2710 con 0x55c440c992f0 2024-05-04T03:28:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.064+0000 7f614effd640 1 -- 172.21.15.39:0/1360762348 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6170083040 con 0x55c440c992f0 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.064+0000 7f614effd640 1 -- 172.21.15.39:0/1360762348 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f615c030710 con 0x55c440c992f0 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.064+0000 7f617ce62740 10 client.26800 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.064+0000 7f617ce62740 10 client.26800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.064+0000 7f617ce62740 20 client.26800 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.064+0000 7f617ce62740 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c440cc21b0 0x55c4410e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.064+0000 7f617ce62740 4 client.26800 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.064+0000 7f617ce62740 1 -- 172.21.15.39:0/1360762348 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c440fd0050 con 0x55c440cc21b0 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.065+0000 7f616ffff640 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c440cc21b0 0x55c4410e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.065+0000 7f614effd640 10 client.26800 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.066+0000 7f616ffff640 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c440cc21b0 0x55c4410e5090 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560f4466306b9801 server_cookie=8782d2ee1dc31957 in_seq=0 out_seq=0 2024-05-04T03:28:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.080+0000 7f614effd640 1 -- 172.21.15.39:0/1360762348 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c440fd0050 con 0x55c440cc21b0 2024-05-04T03:28:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.080+0000 7f614effd640 10 client.26800 handle_command_reply: tid=0 2024-05-04T03:28:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.080+0000 7f617ce62740 1 client.26800 shutdown 2024-05-04T03:28:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.081+0000 7f617ce62740 1 -- 172.21.15.39:0/1360762348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 msgr2=0x55c441071680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.081+0000 7f617ce62740 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441071680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f615c02d020 tx=0x7f615c02aac0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.081+0000 7f617ce62740 1 -- 172.21.15.39:0/1360762348 shutdown_connections 2024-05-04T03:28:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.081+0000 7f617ce62740 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55c440cc21b0 0x55c4410e5090 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.081+0000 7f614effd640 0 client.26800 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.081+0000 7f617ce62740 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c440ca14d0 0x55c441071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.082+0000 7f617ce62740 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c440c992f0 0x55c441071680 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.082+0000 7f617ce62740 1 --2- 172.21.15.39:0/1360762348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c440c80d80 0x55c441074a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.082+0000 7f617ce62740 1 -- 172.21.15.39:0/1360762348 >> 172.21.15.39:0/1360762348 conn(0x55c440cb6300 msgr2=0x55c44106cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.082+0000 7f617ce62740 1 -- 172.21.15.39:0/1360762348 shutdown_connections 2024-05-04T03:28:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.082+0000 7f617ce62740 1 -- 172.21.15.39:0/1360762348 wait complete. 2024-05-04T03:28:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.083+0000 7f617ce62740 20 client.26800 trim_cache size 0 max 16384 2024-05-04T03:28:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:28:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:28:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:28:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:28:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f61600b2940 msgr2=0x7f61600b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f61600b2940 0x7f61600b4e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6164000c00 tx=0x7f616400bf60 comp rx=0 tx=0).stop 2024-05-04T03:28:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f616809d360 msgr2=0x7f61681335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f616809d360 0x7f61681335e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f61640058c0 tx=0x7f616400dde0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 shutdown_connections 2024-05-04T03:28:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f61600b2940 0x7f61600b4e50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61680a2f00 0x7f616813aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f616809e200 0x7f6168133b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 --2- 172.21.15.39:0/2544031715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f616809d360 0x7f61681335e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.085+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 >> 172.21.15.39:0/2544031715 conn(0x7f616800bbe0 msgr2=0x7f616809eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.087+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 shutdown_connections 2024-05-04T03:28:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:30.087+0000 7f6174d52640 1 -- 172.21.15.39:0/2544031715 wait complete. 2024-05-04T03:28:30.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:30 smithi066 ceph-mon[33264]: pgmap v917: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-04T03:28:30.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:30 smithi039 ceph-mon[29715]: pgmap v917: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-04T03:28:30.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:30 smithi167 ceph-mon[33614]: pgmap v917: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-05-04T03:28:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 880.0 s, 1121.8 tps, lat 89.069 ms stddev 32.773 2024-05-04T03:28:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 -- 172.21.15.39:0/548786006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 msgr2=0x7fce6410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 --2- 172.21.15.39:0/548786006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 0x7fce6410ad40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00b0a0 tx=0x7fce5c00b3b0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 -- 172.21.15.39:0/548786006 shutdown_connections 2024-05-04T03:28:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 --2- 172.21.15.39:0/548786006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce6411d930 0x7fce6411fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 --2- 172.21.15.39:0/548786006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 0x7fce6410ad40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 --2- 172.21.15.39:0/548786006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6410b280 0x7fce6411d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 -- 172.21.15.39:0/548786006 >> 172.21.15.39:0/548786006 conn(0x7fce6406f4d0 msgr2=0x7fce6406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:31.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 -- 172.21.15.39:0/548786006 shutdown_connections 2024-05-04T03:28:31.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.933+0000 7fce6bf03700 1 -- 172.21.15.39:0/548786006 wait complete. 2024-05-04T03:28:31.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.934+0000 7fce6bf03700 1 Processor -- start 2024-05-04T03:28:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.935+0000 7fce6bf03700 1 -- start start 2024-05-04T03:28:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce6bf03700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 0x7fce641af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce6bf03700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce6410b280 0x7fce641afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce6bf03700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6411d930 0x7fce641a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce6bf03700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce641231e0 con 0x7fce6410b280 2024-05-04T03:28:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce6bf03700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce64123060 con 0x7fce6410a930 2024-05-04T03:28:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce6bf03700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce64123360 con 0x7fce6411d930 2024-05-04T03:28:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce69c9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 0x7fce641af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce69c9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 0x7fce641af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37880/0 (socket says 172.21.15.39:37880) 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce69c9f700 1 -- 172.21.15.39:0/2842628877 learned_addr learned my addr 172.21.15.39:0/2842628877 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce6a4a0700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6411d930 0x7fce641a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce6949e700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce6410b280 0x7fce641afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce69c9f700 1 -- 172.21.15.39:0/2842628877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6411d930 msgr2=0x7fce641a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce69c9f700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6411d930 0x7fce641a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce69c9f700 1 -- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce6410b280 msgr2=0x7fce641afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce69c9f700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce6410b280 0x7fce641afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.936+0000 7fce69c9f700 1 -- 172.21.15.39:0/2842628877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce641aa070 con 0x7fce6410a930 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.937+0000 7fce69c9f700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 0x7fce641af7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0062a0 tx=0x7fce5c004da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.937+0000 7fce5affd700 1 -- 172.21.15.39:0/2842628877 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c021070 con 0x7fce6410a930 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.937+0000 7fce6bf03700 1 -- 172.21.15.39:0/2842628877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce641aa300 con 0x7fce6410a930 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.937+0000 7fce6bf03700 1 -- 172.21.15.39:0/2842628877 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce641b6630 con 0x7fce6410a930 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.937+0000 7fce5affd700 1 -- 172.21.15.39:0/2842628877 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce5c007c00 con 0x7fce6410a930 2024-05-04T03:28:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.937+0000 7fce5affd700 1 -- 172.21.15.39:0/2842628877 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c004760 con 0x7fce6410a930 2024-05-04T03:28:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.941+0000 7fce5affd700 1 -- 172.21.15.39:0/2842628877 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fce5c01d450 con 0x7fce6410a930 2024-05-04T03:28:31.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.942+0000 7fce5affd700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce500b29c0 0x7fce500b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.942+0000 7fce5affd700 1 -- 172.21.15.39:0/2842628877 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fce5c00f020 con 0x7fce6410a930 2024-05-04T03:28:31.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.943+0000 7fce6949e700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce500b29c0 0x7fce500b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.955+0000 7fce6949e700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce500b29c0 0x7fce500b4ed0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fce54005950 tx=0x7fce540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce7a917b80 1 Processor -- start 2024-05-04T03:28:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce7a917b80 1 -- start start 2024-05-04T03:28:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc7aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc85430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 0x55739dc87d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce7a917b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55739db9d9f0 con 0x55739dc85970 2024-05-04T03:28:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce7a917b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55739db3c950 con 0x55739dc7a660 2024-05-04T03:28:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce7a917b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55739db999b0 con 0x55739dc7afb0 2024-05-04T03:28:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce6a4a0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 0x55739dc87d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce6949e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc85430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce6a4a0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 0x55739dc87d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48662/0 (socket says 172.21.15.39:48662) 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce6a4a0700 1 -- 172.21.15.39:0/1806040958 learned_addr learned my addr 172.21.15.39:0/1806040958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.958+0000 7fce69c9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc7aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.959+0000 7fce6949e700 1 -- 172.21.15.39:0/1806040958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 msgr2=0x55739dc7aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.959+0000 7fce6949e700 1 --2- 172.21.15.39:0/1806040958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc7aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.959+0000 7fce6949e700 1 -- 172.21.15.39:0/1806040958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 msgr2=0x55739dc87d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.959+0000 7fce6949e700 1 --2- 172.21.15.39:0/1806040958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 0x55739dc87d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.959+0000 7fce6949e700 1 -- 172.21.15.39:0/1806040958 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55739daeaa60 con 0x55739dc7afb0 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.959+0000 7fce6949e700 1 --2- 172.21.15.39:0/1806040958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc85430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fce641200c0 tx=0x7fce54012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea5f016bdd346fc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.960+0000 7fce4ffff700 1 -- 172.21.15.39:0/1806040958 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54022070 con 0x55739dc7afb0 2024-05-04T03:28:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.960+0000 7fce4ffff700 1 -- 172.21.15.39:0/1806040958 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce5401e450 con 0x55739dc7afb0 2024-05-04T03:28:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.960+0000 7fce4ffff700 1 -- 172.21.15.39:0/1806040958 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5401d610 con 0x55739dc7afb0 2024-05-04T03:28:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.961+0000 7fce7a917b80 1 -- 172.21.15.39:0/1806040958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 msgr2=0x55739dc85430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.961+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1806040958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc85430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fce641200c0 tx=0x7fce54012000 comp rx=0 tx=0).stop 2024-05-04T03:28:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.961+0000 7fce7a917b80 1 -- 172.21.15.39:0/1806040958 shutdown_connections 2024-05-04T03:28:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.961+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1806040958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 0x55739dc87d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.961+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1806040958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc7aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.961+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1806040958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc85430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.961+0000 7fce7a917b80 1 -- 172.21.15.39:0/1806040958 >> 172.21.15.39:0/1806040958 conn(0x55739dafdfd0 msgr2=0x55739dc76690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.962+0000 7fce7a917b80 1 -- 172.21.15.39:0/1806040958 shutdown_connections 2024-05-04T03:28:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.962+0000 7fce7a917b80 1 -- 172.21.15.39:0/1806040958 wait complete. 2024-05-04T03:28:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.963+0000 7fce7a917b80 1 -- start start 2024-05-04T03:28:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.963+0000 7fce7a917b80 1 Processor -- start 2024-05-04T03:28:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.963+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7a660 0x55739dc828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.963+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc82e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc85970 0x55739dc83360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6949e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc82e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6949e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc82e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:53938/0 (socket says 172.21.15.39:53938) 2024-05-04T03:28:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6949e700 1 -- 172.21.15.39:0/2399849232 learned_addr learned my addr 172.21.15.39:0/2399849232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55739dc88700 con 0x55739dc7a660 2024-05-04T03:28:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55739db03dc0 con 0x55739dc85970 2024-05-04T03:28:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55739dc88880 con 0x55739dc7afb0 2024-05-04T03:28:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6a4a0700 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc85970 0x55739dc83360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce7a917b80 10 client.0 _ll_register_callbacks cb 0x55739db35ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6949e700 1 -- 172.21.15.39:0/2399849232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc85970 msgr2=0x55739dc83360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6949e700 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc85970 0x55739dc83360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6949e700 1 -- 172.21.15.39:0/2399849232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7a660 msgr2=0x55739dc828e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6949e700 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7a660 0x55739dc828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.964+0000 7fce6949e700 1 -- 172.21.15.39:0/2399849232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55739db39820 con 0x55739dc7afb0 2024-05-04T03:28:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.965+0000 7fce6949e700 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc82e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fce5401e880 tx=0x7fce5401e8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=676e1d063eb02812 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.965+0000 7fce4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:28:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.966+0000 7fce4d7fa700 1 -- 172.21.15.39:0/2399849232 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54036060 con 0x55739dc7afb0 2024-05-04T03:28:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.966+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5401e450 con 0x55739dc7afb0 2024-05-04T03:28:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.966+0000 7fce4d7fa700 1 -- 172.21.15.39:0/2399849232 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fce54011ab0 con 0x55739dc7afb0 2024-05-04T03:28:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.966+0000 7fce4d7fa700 1 -- 172.21.15.39:0/2399849232 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fce54022050 con 0x55739dc7afb0 2024-05-04T03:28:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.966+0000 7fce7a917b80 10 client.26809 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.966+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55739dc932f0 con 0x55739dc7afb0 2024-05-04T03:28:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce4d7fa700 1 -- 172.21.15.39:0/2399849232 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce5401e450 con 0x55739dc7afb0 2024-05-04T03:28:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce4d7fa700 1 -- 172.21.15.39:0/2399849232 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fce54027ce0 con 0x55739dc7afb0 2024-05-04T03:28:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce7a917b80 10 client.26809 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce7a917b80 10 client.26809 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:28:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce7a917b80 20 client.26809 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce7a917b80 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55739dc95c80 0x55739dc98170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce7a917b80 4 client.26809 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:28:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55739dbab3d0 con 0x55739dc95c80 2024-05-04T03:28:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.967+0000 7fce69c9f700 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55739dc95c80 0x55739dc98170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.968+0000 7fce4d7fa700 10 client.26809 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:31.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.968+0000 7fce69c9f700 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55739dc95c80 0x55739dc98170 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c11101c182dfea7 server_cookie=f125a425f7e4ae7a in_seq=0 out_seq=0 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce4d7fa700 1 -- 172.21.15.39:0/2399849232 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55739dbab3d0 con 0x55739dc95c80 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce4d7fa700 10 client.26809 handle_command_reply: tid=0 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 client.26809 shutdown 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 msgr2=0x55739dc82e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc82e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fce5401e880 tx=0x7fce5401e8b0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 shutdown_connections 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55739dc95c80 0x55739dc98170 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7a660 0x55739dc828e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc85970 0x55739dc83360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 --2- 172.21.15.39:0/2399849232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc82e20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 >> 172.21.15.39:0/2399849232 conn(0x55739dafdfd0 msgr2=0x55739dc7c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce4d7fa700 0 client.26809 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 shutdown_connections 2024-05-04T03:28:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.969+0000 7fce7a917b80 1 -- 172.21.15.39:0/2399849232 wait complete. 2024-05-04T03:28:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:31.970+0000 7fce7a917b80 20 client.26809 trim_cache size 0 max 16384 2024-05-04T03:28:32.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.007+0000 7fce7a917b80 1 Processor -- start 2024-05-04T03:28:32.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.007+0000 7fce7a917b80 1 -- start start 2024-05-04T03:28:32.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.008+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:32.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.008+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7afb0 0x55739dc763e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.008+0000 7fce69c9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.008+0000 7fce69c9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc917f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37924/0 (socket says 172.21.15.39:37924) 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.008+0000 7fce69c9f700 1 -- 172.21.15.39:0/1832242722 learned_addr learned my addr 172.21.15.39:0/1832242722 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.008+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc85970 0x55739dc82e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce7a917b80 1 -- 172.21.15.39:0/1832242722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce4800d430 con 0x55739dc7afb0 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce7a917b80 1 -- 172.21.15.39:0/1832242722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce4800ef70 con 0x55739dc7a660 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce7a917b80 1 -- 172.21.15.39:0/1832242722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce4800db00 con 0x55739dc85970 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce6a4a0700 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc85970 0x55739dc82e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:32.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce6949e700 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7afb0 0x55739dc763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce69c9f700 1 -- 172.21.15.39:0/1832242722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc85970 msgr2=0x55739dc82e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce69c9f700 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc85970 0x55739dc82e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce69c9f700 1 -- 172.21.15.39:0/1832242722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7afb0 msgr2=0x55739dc763e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce69c9f700 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7afb0 0x55739dc763e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.009+0000 7fce69c9f700 1 -- 172.21.15.39:0/1832242722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55739dc80e50 con 0x55739dc7a660 2024-05-04T03:28:32.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.010+0000 7fce69c9f700 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc917f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009fd0 tx=0x7fce5c0e1fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea978c935cf8c0b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:32.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.010+0000 7fce4f7fe700 1 -- 172.21.15.39:0/1832242722 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c0b43a0 con 0x55739dc7a660 2024-05-04T03:28:32.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.011+0000 7fce4f7fe700 1 -- 172.21.15.39:0/1832242722 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce5c0ab7a0 con 0x55739dc7a660 2024-05-04T03:28:32.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.011+0000 7fce4f7fe700 1 -- 172.21.15.39:0/1832242722 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c009c60 con 0x55739dc7a660 2024-05-04T03:28:32.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.012+0000 7fce7a917b80 1 -- 172.21.15.39:0/1832242722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 msgr2=0x55739dc917f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:32.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.012+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc917f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009fd0 tx=0x7fce5c0e1fb0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.014+0000 7fce7a917b80 1 -- 172.21.15.39:0/1832242722 shutdown_connections 2024-05-04T03:28:32.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.014+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc85970 0x55739dc82e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.014+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc917f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.014+0000 7fce7a917b80 1 --2- 172.21.15.39:0/1832242722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc7afb0 0x55739dc763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.014+0000 7fce7a917b80 1 -- 172.21.15.39:0/1832242722 >> 172.21.15.39:0/1832242722 conn(0x55739dafdfd0 msgr2=0x55739dc97c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:32.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.014+0000 7fce7a917b80 1 -- 172.21.15.39:0/1832242722 shutdown_connections 2024-05-04T03:28:32.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.015+0000 7fce7a917b80 1 -- 172.21.15.39:0/1832242722 wait complete. 2024-05-04T03:28:32.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.015+0000 7fce7a917b80 1 -- start start 2024-05-04T03:28:32.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.015+0000 7fce7a917b80 1 Processor -- start 2024-05-04T03:28:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.016+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc8bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.016+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc8c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.016+0000 7fce7a917b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 0x55739dcff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.016+0000 7fce69c9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc8bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.016+0000 7fce7a917b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce4800e8a0 con 0x55739dc85970 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.016+0000 7fce7a917b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fce4800e1d0 con 0x55739dc7a660 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce69c9f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc8bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:37928/0 (socket says 172.21.15.39:37928) 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce69c9f700 1 -- 172.21.15.39:0/3737183067 learned_addr learned my addr 172.21.15.39:0/3737183067 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce4800b3d0 con 0x55739dc7afb0 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce6949e700 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc8c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce7a917b80 10 client.0 _ll_register_callbacks cb 0x55739db35ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce69c9f700 1 -- 172.21.15.39:0/3737183067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 msgr2=0x55739dc8c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce69c9f700 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc8c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce69c9f700 1 -- 172.21.15.39:0/3737183067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 msgr2=0x55739dcff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce69c9f700 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 0x55739dcff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce69c9f700 1 -- 172.21.15.39:0/3737183067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55739dcffa30 con 0x55739dc7a660 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce69c9f700 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc8bea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0abc20 tx=0x7fce5c004900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=645f149dab0a3d3a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.017+0000 7fce4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.018+0000 7fce4d7fa700 1 -- 172.21.15.39:0/3737183067 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c08c060 con 0x55739dc7a660 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.018+0000 7fce4d7fa700 1 -- 172.21.15.39:0/3737183067 <== mon.2 v2:172.21.15.66:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fce5c0a8280 con 0x55739dc7a660 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.018+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5c0ab7a0 con 0x55739dc7a660 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.018+0000 7fce4d7fa700 1 -- 172.21.15.39:0/3737183067 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fce5c0abc90 con 0x55739dc7a660 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.018+0000 7fce7a917b80 10 client.26792 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.018+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55739dd00be0 con 0x55739dc7a660 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce4d7fa700 1 -- 172.21.15.39:0/3737183067 <== mon.2 v2:172.21.15.66:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce5c0b0040 con 0x55739dc7a660 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce4d7fa700 1 -- 172.21.15.39:0/3737183067 <== mon.2 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fce5c011070 con 0x55739dc7a660 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce7a917b80 10 client.26792 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce7a917b80 10 client.26792 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce7a917b80 20 client.26792 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce7a917b80 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55739dd01220 0x55739dd03640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce7a917b80 4 client.26792 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.9, "await_for": 42.0}] 2024-05-04T03:28:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.9, "await_for": 42.0}) -- 0x7fce5c0ab7a0 con 0x55739dd01220 2024-05-04T03:28:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.019+0000 7fce6949e700 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55739dd01220 0x55739dd03640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.020+0000 7fce4d7fa700 10 client.26792 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.020+0000 7fce6949e700 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55739dd01220 0x55739dd03640 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797a25739e7af462 server_cookie=948427710e649d8c in_seq=0 out_seq=0 2024-05-04T03:28:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce4d7fa700 1 -- 172.21.15.39:0/3737183067 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7fce5c0ab7a0 con 0x55739dd01220 2024-05-04T03:28:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce4d7fa700 10 client.26792 handle_command_reply: tid=0 2024-05-04T03:28:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce7a917b80 1 client.26792 shutdown 2024-05-04T03:28:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 msgr2=0x55739dc8bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce7a917b80 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc8bea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0abc20 tx=0x7fce5c004900 comp rx=0 tx=0).stop 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 shutdown_connections 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce7a917b80 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55739dd01220 0x55739dd03640 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce7a917b80 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55739dc85970 0x55739dcff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce4d7fa700 0 client.26792 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce7a917b80 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55739dc7a660 0x55739dc8bea0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.374+0000 7fce7a917b80 1 --2- 172.21.15.39:0/3737183067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55739dc7afb0 0x55739dc8c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 >> 172.21.15.39:0/3737183067 conn(0x55739dafdfd0 msgr2=0x55739dc97c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 shutdown_connections 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce7a917b80 1 -- 172.21.15.39:0/3737183067 wait complete. 2024-05-04T03:28:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce7a917b80 20 client.26792 trim_cache size 0 max 16384 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 46, 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "6525321e": { 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 46, 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:28:32.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 80.9, 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:32.381 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:28:32.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 -- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce500b29c0 msgr2=0x7fce500b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:32.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce500b29c0 0x7fce500b4ed0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fce54005950 tx=0x7fce540058e0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 -- 172.21.15.39:0/2842628877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 msgr2=0x7fce641af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:32.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 0x7fce641af7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0062a0 tx=0x7fce5c004da0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 -- 172.21.15.39:0/2842628877 shutdown_connections 2024-05-04T03:28:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fce500b29c0 0x7fce500b4ed0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce6411d930 0x7fce641a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fce6410a930 0x7fce641af7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 --2- 172.21.15.39:0/2842628877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce6410b280 0x7fce641afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 -- 172.21.15.39:0/2842628877 >> 172.21.15.39:0/2842628877 conn(0x7fce6406f4d0 msgr2=0x7fce6411b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.375+0000 7fce6bf03700 1 -- 172.21.15.39:0/2842628877 shutdown_connections 2024-05-04T03:28:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:32.376+0000 7fce6bf03700 1 -- 172.21.15.39:0/2842628877 wait complete. 2024-05-04T03:28:32.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:32 smithi039 ceph-mon[29715]: pgmap v918: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-04T03:28:32.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:32 smithi066 ceph-mon[33264]: pgmap v918: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-04T03:28:32.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:32 smithi167 ceph-mon[33614]: pgmap v918: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 1.27k op/s 2024-05-04T03:28:33.099 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6525321e', quiesce took 8.5 seconds. Will release after: 20.9 2024-05-04T03:28:34.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:34 smithi167 ceph-mon[33614]: pgmap v919: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-05-04T03:28:34.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:34 smithi039 ceph-mon[29715]: pgmap v919: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-05-04T03:28:35.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:34 smithi066 ceph-mon[33264]: pgmap v919: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 1.34k op/s 2024-05-04T03:28:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 885.0 s, 65.7 tps, lat 89.533 ms stddev 22.457 2024-05-04T03:28:36.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:36 smithi167 ceph-mon[33614]: pgmap v920: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-04T03:28:36.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:36 smithi039 ceph-mon[29715]: pgmap v920: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-04T03:28:37.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:36 smithi066 ceph-mon[33264]: pgmap v920: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 1.12k op/s 2024-05-04T03:28:38.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:38 smithi167 ceph-mon[33614]: pgmap v921: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2024-05-04T03:28:38.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:38 smithi039 ceph-mon[29715]: pgmap v921: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2024-05-04T03:28:39.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:38 smithi066 ceph-mon[33264]: pgmap v921: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.2 MiB/s wr, 1.00k op/s 2024-05-04T03:28:40.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:40 smithi167 ceph-mon[33614]: pgmap v922: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-05-04T03:28:40.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:40 smithi039 ceph-mon[29715]: pgmap v922: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-05-04T03:28:41.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:40 smithi066 ceph-mon[33264]: pgmap v922: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-05-04T03:28:41.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 890.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:28:42.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:42 smithi167 ceph-mon[33614]: pgmap v923: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-05-04T03:28:42.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:42 smithi039 ceph-mon[29715]: pgmap v923: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-05-04T03:28:43.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:42 smithi066 ceph-mon[33264]: pgmap v923: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-05-04T03:28:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3935670143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 msgr2=0x7f7ab8101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3935670143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 0x7f7ab8101d40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009900 tx=0x7f7aa802a410 comp rx=0 tx=0).stop 2024-05-04T03:28:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3935670143 shutdown_connections 2024-05-04T03:28:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3935670143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ab81023d0 0x7f7ab810be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3935670143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 0x7f7ab8101d40 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3935670143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8105220 0x7f7ab8105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3935670143 >> 172.21.15.39:0/3935670143 conn(0x7f7ab80fd4f0 msgr2=0x7f7ab80ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3935670143 shutdown_connections 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.212+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3935670143 wait complete. 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7ac07b7640 1 Processor -- start 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7ac07b7640 1 -- start start 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7ac07b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 0x7f7ab819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7ac07b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ab81023d0 0x7f7ab819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7ac07b7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8105220 0x7f7ab81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7ac07b7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab810f640 con 0x7f7ab81018d0 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7ac07b7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab810f4c0 con 0x7f7ab8105220 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7ac07b7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab810f7c0 con 0x7f7ab81023d0 2024-05-04T03:28:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abe553640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 0x7f7ab819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abe553640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 0x7f7ab819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33738/0 (socket says 172.21.15.39:33738) 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abe553640 1 -- 172.21.15.39:0/3028048408 learned_addr learned my addr 172.21.15.39:0/3028048408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abdd52640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ab81023d0 0x7f7ab819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abed54640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8105220 0x7f7ab81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abe553640 1 -- 172.21.15.39:0/3028048408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ab81023d0 msgr2=0x7f7ab819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abe553640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ab81023d0 0x7f7ab819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abe553640 1 -- 172.21.15.39:0/3028048408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8105220 msgr2=0x7f7ab81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abe553640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8105220 0x7f7ab81a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.213+0000 7f7abe553640 1 -- 172.21.15.39:0/3028048408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab81a1d00 con 0x7f7ab81018d0 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.214+0000 7f7abe553640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 0x7f7ab819a270 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c970 tx=0x7f7ab400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.214+0000 7f7aaf7fe640 1 -- 172.21.15.39:0/3028048408 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4004220 con 0x7f7ab81018d0 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.214+0000 7f7aaf7fe640 1 -- 172.21.15.39:0/3028048408 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ab4007e40 con 0x7f7ab81018d0 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.214+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3028048408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab81a1ff0 con 0x7f7ab81018d0 2024-05-04T03:28:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.214+0000 7f7aaf7fe640 1 -- 172.21.15.39:0/3028048408 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab400f450 con 0x7f7ab81018d0 2024-05-04T03:28:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.214+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3028048408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab81a24b0 con 0x7f7ab81018d0 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.217+0000 7f7ac766e740 1 Processor -- start 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.217+0000 7f7ac766e740 1 -- start start 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.217+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.217+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 0x5599365d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7ac766e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5599365395d0 con 0x5599362331b0 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7ac766e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599364f2890 con 0x55993620a2f0 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7ac766e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559936535590 con 0x5599362124d0 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7aaf7fe640 1 -- 172.21.15.39:0/3028048408 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7ab40079b0 con 0x7f7ab81018d0 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365d4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33750/0 (socket says 172.21.15.39:33750) 2024-05-04T03:28:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 -- 172.21.15.39:0/981340074 learned_addr learned my addr 172.21.15.39:0/981340074 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abdd52640 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abe553640 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 0x5599365d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 -- 172.21.15.39:0/981340074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 msgr2=0x5599365dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365dbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 -- 172.21.15.39:0/981340074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 msgr2=0x5599365d4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 0x5599365d4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 -- 172.21.15.39:0/981340074 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993624b210 con 0x5599362331b0 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7abed54640 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365d4a60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007e50 tx=0x7f7ab000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1904381ae3ee641e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.218+0000 7f7aaf7fe640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aa40b2860 0x7f7aa40b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.219+0000 7f7aacff9640 1 -- 172.21.15.39:0/981340074 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0014070 con 0x5599362331b0 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.219+0000 7f7abe553640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aa40b2860 0x7f7aa40b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.219+0000 7f7aacff9640 1 -- 172.21.15.39:0/981340074 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ab000ee90 con 0x5599362331b0 2024-05-04T03:28:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.219+0000 7f7aacff9640 1 -- 172.21.15.39:0/981340074 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0005310 con 0x5599362331b0 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.219+0000 7f7aaf7fe640 1 -- 172.21.15.39:0/3028048408 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7ab40e1e50 con 0x7f7ab81018d0 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 -- 172.21.15.39:0/981340074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 msgr2=0x5599365d4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365d4a60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007e50 tx=0x7f7ab000e720 comp rx=0 tx=0).stop 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7abe553640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aa40b2860 0x7f7aa40b4d70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x5599365d5e40 tx=0x7f7ab4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 -- 172.21.15.39:0/981340074 shutdown_connections 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 0x5599365d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 --2- 172.21.15.39:0/981340074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365d4a60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 -- 172.21.15.39:0/981340074 >> 172.21.15.39:0/981340074 conn(0x559936227300 msgr2=0x5599365d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 -- 172.21.15.39:0/981340074 shutdown_connections 2024-05-04T03:28:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 -- 172.21.15.39:0/981340074 wait complete. 2024-05-04T03:28:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 -- start start 2024-05-04T03:28:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 Processor -- start 2024-05-04T03:28:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.220+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 0x5599365de700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365dec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7ac766e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5599365dc930 con 0x5599362331b0 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7ac766e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5599364f6370 con 0x55993620a2f0 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7ac766e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5599365dcab0 con 0x5599362124d0 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7ac766e740 10 client.0 _ll_register_callbacks cb 0x55993624aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365e5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33764/0 (socket says 172.21.15.39:33764) 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 -- 172.21.15.39:0/2666346901 learned_addr learned my addr 172.21.15.39:0/2666346901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 0x5599365de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abdd52640 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365dec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 -- 172.21.15.39:0/2666346901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 msgr2=0x5599365dec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365dec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 -- 172.21.15.39:0/2666346901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 msgr2=0x5599365de700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 0x5599365de700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 -- 172.21.15.39:0/2666346901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559936317620 con 0x5599362331b0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7abed54640 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365e5b30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005390 tx=0x7f7ab0011fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb46ac534ca05e85 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.221+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/2666346901 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0014040 con 0x5599362331b0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/2666346901 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7ab0005970 con 0x5599362331b0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 1 -- 172.21.15.39:0/2666346901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab000ee90 con 0x5599362331b0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/2666346901 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ab0003970 con 0x5599362331b0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 10 client.17235 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 1 -- 172.21.15.39:0/2666346901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5599365e8a60 con 0x5599362331b0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/2666346901 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ab000ee90 con 0x5599362331b0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/2666346901 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7ab0031080 con 0x5599362331b0 2024-05-04T03:28:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 10 client.17235 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 20 client.17235 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599361f1d80 0x5599365ebee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 4 client.17235 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7ac766e740 1 -- 172.21.15.39:0/2666346901 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599365471e0 con 0x5599361f1d80 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.222+0000 7f7abdd52640 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599361f1d80 0x5599365ebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.223+0000 7f7aa1ffb640 10 client.17235 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.223+0000 7f7abdd52640 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599361f1d80 0x5599365ebee0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f07961f3a05a037 server_cookie=430772d42feaf8a2 in_seq=0 out_seq=0 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/2666346901 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599365471e0 con 0x5599361f1d80 2024-05-04T03:28:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7aa1ffb640 10 client.17235 handle_command_reply: tid=0 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 client.17235 shutdown 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 -- 172.21.15.39:0/2666346901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 msgr2=0x5599365e5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365e5b30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005390 tx=0x7f7ab0011fc0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 -- 172.21.15.39:0/2666346901 shutdown_connections 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599361f1d80 0x5599365ebee0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599362331b0 0x5599365e5b30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365dec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7aa1ffb640 0 client.17235 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 --2- 172.21.15.39:0/2666346901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55993620a2f0 0x5599365de700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 -- 172.21.15.39:0/2666346901 >> 172.21.15.39:0/2666346901 conn(0x559936227300 msgr2=0x5599365cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 -- 172.21.15.39:0/2666346901 shutdown_connections 2024-05-04T03:28:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 1 -- 172.21.15.39:0/2666346901 wait complete. 2024-05-04T03:28:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.224+0000 7f7ac766e740 20 client.17235 trim_cache size 0 max 16384 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7ac766e740 1 Processor -- start 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7ac766e740 1 -- start start 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x5599365d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362331b0 0x5599365d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7ac766e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab81ab2e0 con 0x55993620a2f0 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7ac766e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab81ace20 con 0x5599362331b0 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7ac766e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab81ab9b0 con 0x5599362124d0 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x5599365d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x5599365d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33776/0 (socket says 172.21.15.39:33776) 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 -- 172.21.15.39:0/1985555035 learned_addr learned my addr 172.21.15.39:0/1985555035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abdd52640 1 --2- 172.21.15.39:0/1985555035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 -- 172.21.15.39:0/1985555035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 msgr2=0x5599365e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 --2- 172.21.15.39:0/1985555035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365e5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 -- 172.21.15.39:0/1985555035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362331b0 msgr2=0x5599365d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 --2- 172.21.15.39:0/1985555035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362331b0 0x5599365d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 -- 172.21.15.39:0/1985555035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599365cba80 con 0x55993620a2f0 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.240+0000 7f7abed54640 1 --2- 172.21.15.39:0/1985555035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x5599365d0400 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007be0 tx=0x7f7ab0004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b155a07813e9c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7aa3fff640 1 -- 172.21.15.39:0/1985555035 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0014040 con 0x55993620a2f0 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7aa3fff640 1 -- 172.21.15.39:0/1985555035 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ab002e070 con 0x55993620a2f0 2024-05-04T03:28:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7aa3fff640 1 -- 172.21.15.39:0/1985555035 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00041f0 con 0x55993620a2f0 2024-05-04T03:28:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 -- 172.21.15.39:0/1985555035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 msgr2=0x5599365d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 --2- 172.21.15.39:0/1985555035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x5599365d0400 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0007be0 tx=0x7f7ab0004e70 comp rx=0 tx=0).stop 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 -- 172.21.15.39:0/1985555035 shutdown_connections 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 --2- 172.21.15.39:0/1985555035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362331b0 0x5599365d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 --2- 172.21.15.39:0/1985555035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362124d0 0x5599365e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 --2- 172.21.15.39:0/1985555035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x5599365d0400 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 -- 172.21.15.39:0/1985555035 >> 172.21.15.39:0/1985555035 conn(0x559936227300 msgr2=0x5599365cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 -- 172.21.15.39:0/1985555035 shutdown_connections 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.241+0000 7f7ac766e740 1 -- 172.21.15.39:0/1985555035 wait complete. 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 1 -- start start 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 1 Processor -- start 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x55993664fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362124d0 0x559936650170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362331b0 0x5599366570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab81ac750 con 0x55993620a2f0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab81ac080 con 0x5599362124d0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab81a92c0 con 0x5599362331b0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7ac766e740 10 client.0 _ll_register_callbacks cb 0x7f7aa8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x55993664fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x55993664fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33790/0 (socket says 172.21.15.39:33790) 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 -- 172.21.15.39:0/813189711 learned_addr learned my addr 172.21.15.39:0/813189711 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abdd52640 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362124d0 0x559936650170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362331b0 0x5599366570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 -- 172.21.15.39:0/813189711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362331b0 msgr2=0x5599366570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362331b0 0x5599366570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 -- 172.21.15.39:0/813189711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362124d0 msgr2=0x559936650170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362124d0 0x559936650170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.242+0000 7f7abed54640 1 -- 172.21.15.39:0/813189711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599365d0940 con 0x55993620a2f0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7abed54640 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x55993664fc30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0004dd0 tx=0x7f7ab0007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7c694d99dbd912 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/813189711 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0040030 con 0x55993620a2f0 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/813189711 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7ab0020040 con 0x55993620a2f0 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 1 -- 172.21.15.39:0/813189711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab002e070 con 0x55993620a2f0 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/813189711 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ab0014070 con 0x55993620a2f0 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 10 client.17241 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 1 -- 172.21.15.39:0/813189711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559936658750 con 0x55993620a2f0 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/813189711 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ab002e070 con 0x55993620a2f0 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/813189711 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7ab0031070 con 0x55993620a2f0 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 10 client.17241 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 10 client.17241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 20 client.17241 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599361c71c0 0x55993665af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 4 client.17241 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:28:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.243+0000 7f7ac766e740 1 -- 172.21.15.39:0/813189711 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559936541050 con 0x5599361c71c0 2024-05-04T03:28:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.244+0000 7f7abdd52640 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599361c71c0 0x55993665af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.244+0000 7f7aa1ffb640 10 client.17241 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.244+0000 7f7abdd52640 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599361c71c0 0x55993665af40 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8509dfd202657cb6 server_cookie=dc17e4a9d8ee8964 in_seq=0 out_seq=0 2024-05-04T03:28:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.253+0000 7f7aa1ffb640 1 -- 172.21.15.39:0/813189711 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559936541050 con 0x5599361c71c0 2024-05-04T03:28:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.253+0000 7f7aa1ffb640 10 client.17241 handle_command_reply: tid=0 2024-05-04T03:28:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 client.17241 shutdown 2024-05-04T03:28:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 -- 172.21.15.39:0/813189711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 msgr2=0x55993664fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x55993664fc30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0004dd0 tx=0x7f7ab0007b00 comp rx=0 tx=0).stop 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 -- 172.21.15.39:0/813189711 shutdown_connections 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5599361c71c0 0x55993665af40 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5599362331b0 0x5599366570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599362124d0 0x559936650170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7aa1ffb640 0 client.17241 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 --2- 172.21.15.39:0/813189711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55993620a2f0 0x55993664fc30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 -- 172.21.15.39:0/813189711 >> 172.21.15.39:0/813189711 conn(0x559936227300 msgr2=0x5599365cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 -- 172.21.15.39:0/813189711 shutdown_connections 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 1 -- 172.21.15.39:0/813189711 wait complete. 2024-05-04T03:28:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac766e740 20 client.17241 trim_cache size 0 max 16384 2024-05-04T03:28:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:28:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:28:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:28:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:28:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aa40b2860 msgr2=0x7f7aa40b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aa40b2860 0x7f7aa40b4d70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x5599365d5e40 tx=0x7f7ab4000bc0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 msgr2=0x7f7ab819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.254+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 0x7f7ab819a270 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c970 tx=0x7f7ab400ce30 comp rx=0 tx=0).stop 2024-05-04T03:28:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.255+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3028048408 shutdown_connections 2024-05-04T03:28:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.255+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7aa40b2860 0x7f7aa40b4d70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.255+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ab8105220 0x7f7ab81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.255+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ab81023d0 0x7f7ab819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.255+0000 7f7ac07b7640 1 --2- 172.21.15.39:0/3028048408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ab81018d0 0x7f7ab819a270 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.255+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3028048408 >> 172.21.15.39:0/3028048408 conn(0x7f7ab80fd4f0 msgr2=0x7f7ab8105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.255+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3028048408 shutdown_connections 2024-05-04T03:28:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:43.255+0000 7f7ac07b7640 1 -- 172.21.15.39:0/3028048408 wait complete. 2024-05-04T03:28:44.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:44 smithi167 ceph-mon[33614]: pgmap v924: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-05-04T03:28:44.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:44 smithi039 ceph-mon[29715]: pgmap v924: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-05-04T03:28:45.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:44 smithi066 ceph-mon[33264]: pgmap v924: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-05-04T03:28:46.778 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 895.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:28:46.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:46 smithi167 ceph-mon[33614]: pgmap v925: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-05-04T03:28:46.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:46 smithi039 ceph-mon[29715]: pgmap v925: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-05-04T03:28:47.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:46 smithi066 ceph-mon[33264]: pgmap v925: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-05-04T03:28:48.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:48 smithi167 ceph-mon[33614]: pgmap v926: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 412 KiB/s wr, 64 op/s 2024-05-04T03:28:48.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:48 smithi039 ceph-mon[29715]: pgmap v926: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 412 KiB/s wr, 64 op/s 2024-05-04T03:28:49.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:48 smithi066 ceph-mon[33264]: pgmap v926: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 412 KiB/s wr, 64 op/s 2024-05-04T03:28:50.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:50 smithi167 ceph-mon[33614]: pgmap v927: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 189 KiB/s wr, 34 op/s 2024-05-04T03:28:50.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:50 smithi039 ceph-mon[29715]: pgmap v927: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 189 KiB/s wr, 34 op/s 2024-05-04T03:28:51.071 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:50 smithi066 ceph-mon[33264]: pgmap v927: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 189 KiB/s wr, 34 op/s 2024-05-04T03:28:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 900.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:28:52.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:52 smithi167 ceph-mon[33614]: pgmap v928: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 203 KiB/s wr, 37 op/s 2024-05-04T03:28:52.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:52 smithi039 ceph-mon[29715]: pgmap v928: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 203 KiB/s wr, 37 op/s 2024-05-04T03:28:53.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:52 smithi066 ceph-mon[33264]: pgmap v928: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 203 KiB/s wr, 37 op/s 2024-05-04T03:28:54.000 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:28:54.263 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:28:54.679 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:54 smithi039 ceph-mon[29715]: pgmap v929: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 218 KiB/s wr, 38 op/s 2024-05-04T03:28:54.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:54 smithi167 ceph-mon[33614]: pgmap v929: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 218 KiB/s wr, 38 op/s 2024-05-04T03:28:55.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:54 smithi066 ceph-mon[33264]: pgmap v929: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 218 KiB/s wr, 38 op/s 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.146+0000 7fa756167700 1 -- 172.21.15.39:0/3987527574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa750108740 msgr2=0x7fa7501100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.146+0000 7fa756167700 1 --2- 172.21.15.39:0/3987527574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa750108740 0x7fa7501100a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa7480099c0 tx=0x7fa748009cd0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.146+0000 7fa756167700 1 -- 172.21.15.39:0/3987527574 shutdown_connections 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.146+0000 7fa756167700 1 --2- 172.21.15.39:0/3987527574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa750108740 0x7fa7501100a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.146+0000 7fa756167700 1 --2- 172.21.15.39:0/3987527574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7501034c0 0x7fa7501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.146+0000 7fa756167700 1 --2- 172.21.15.39:0/3987527574 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750107df0 0x7fa750108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.147+0000 7fa756167700 1 -- 172.21.15.39:0/3987527574 >> 172.21.15.39:0/3987527574 conn(0x7fa7500fedf0 msgr2=0x7fa750101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.147+0000 7fa756167700 1 -- 172.21.15.39:0/3987527574 shutdown_connections 2024-05-04T03:28:56.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.147+0000 7fa756167700 1 -- 172.21.15.39:0/3987527574 wait complete. 2024-05-04T03:28:56.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.147+0000 7fa756167700 1 Processor -- start 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa756167700 1 -- start start 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa756167700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7501034c0 0x7fa75019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa756167700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750107df0 0x7fa75019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa756167700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750108740 0x7fa7501a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa756167700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa750113220 con 0x7fa7501034c0 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa756167700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501130a0 con 0x7fa750107df0 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa756167700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501133a0 con 0x7fa750108740 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7501034c0 0x7fa75019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7501034c0 0x7fa75019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43580/0 (socket says 172.21.15.39:43580) 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 -- 172.21.15.39:0/1813312780 learned_addr learned my addr 172.21.15.39:0/1813312780 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:56.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74ffff700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750108740 0x7fa7501a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74effd700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750107df0 0x7fa75019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 -- 172.21.15.39:0/1813312780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750108740 msgr2=0x7fa7501a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750108740 0x7fa7501a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 -- 172.21.15.39:0/1813312780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750107df0 msgr2=0x7fa75019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750107df0 0x7fa75019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 -- 172.21.15.39:0/1813312780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7501a1df0 con 0x7fa7501034c0 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74ffff700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750108740 0x7fa7501a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.148+0000 7fa74f7fe700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7501034c0 0x7fa75019cf20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa740007f10 tx=0x7fa74000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.149+0000 7fa74cff9700 1 -- 172.21.15.39:0/1813312780 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740003a00 con 0x7fa7501034c0 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.149+0000 7fa74cff9700 1 -- 172.21.15.39:0/1813312780 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa740003ba0 con 0x7fa7501034c0 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.149+0000 7fa74cff9700 1 -- 172.21.15.39:0/1813312780 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740015660 con 0x7fa7501034c0 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.149+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7501a20e0 con 0x7fa7501034c0 2024-05-04T03:28:56.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.149+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7501a9a00 con 0x7fa7501034c0 2024-05-04T03:28:56.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.150+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7500512c0 con 0x7fa7501034c0 2024-05-04T03:28:56.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.151+0000 7fa74cff9700 1 -- 172.21.15.39:0/1813312780 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa740004000 con 0x7fa7501034c0 2024-05-04T03:28:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.152+0000 7fa74cff9700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa73c0b2900 0x7fa73c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.152+0000 7fa74effd700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa73c0b2900 0x7fa73c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.153+0000 7fa74cff9700 1 -- 172.21.15.39:0/1813312780 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa7400e1f80 con 0x7fa7501034c0 2024-05-04T03:28:56.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.153+0000 7fa74effd700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa73c0b2900 0x7fa73c0b4e10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa750107540 tx=0x7fa738009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:56.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.157+0000 7fa74cff9700 1 -- 172.21.15.39:0/1813312780 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa740051df0 con 0x7fa7501034c0 2024-05-04T03:28:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 -- 172.21.15.39:0/2116722339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 msgr2=0x7f9a00103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 --2- 172.21.15.39:0/2116722339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 0x7f9a00103850 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009900 tx=0x7f99fc02a410 comp rx=0 tx=0).stop 2024-05-04T03:28:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 -- 172.21.15.39:0/2116722339 shutdown_connections 2024-05-04T03:28:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 --2- 172.21.15.39:0/2116722339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a00108a90 0x7f9a0010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 --2- 172.21.15.39:0/2116722339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a00103d90 0x7f9a001041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 --2- 172.21.15.39:0/2116722339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 0x7f9a00103850 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 -- 172.21.15.39:0/2116722339 >> 172.21.15.39:0/2116722339 conn(0x7f9a000fdd40 msgr2=0x7f9a001001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 -- 172.21.15.39:0/2116722339 shutdown_connections 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 -- 172.21.15.39:0/2116722339 wait complete. 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 Processor -- start 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.376+0000 7f9a076e0640 1 -- start start 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a076e0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 0x7f9a0006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a076e0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a00103d90 0x7f9a0019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a076e0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a00108a90 0x7f9a001a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a076e0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010e550 con 0x7f9a00102b50 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a076e0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010e3d0 con 0x7f9a00103d90 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a076e0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0010e6d0 con 0x7f9a00108a90 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 0x7f9a0006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 0x7f9a0006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43590/0 (socket says 172.21.15.39:43590) 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 -- 172.21.15.39:0/3057774943 learned_addr learned my addr 172.21.15.39:0/3057774943 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a00108a90 0x7f9a001a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a00103d90 0x7f9a0019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 -- 172.21.15.39:0/3057774943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a00108a90 msgr2=0x7f9a001a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a00108a90 0x7f9a001a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 -- 172.21.15.39:0/3057774943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a00103d90 msgr2=0x7f9a0019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a00103d90 0x7f9a0019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 -- 172.21.15.39:0/3057774943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a001a1e00 con 0x7f9a00102b50 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a00103d90 0x7f9a0019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f9a0547c640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 0x7f9a0006afc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005760 tx=0x7f99fc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.377+0000 7f99f67fc640 1 -- 172.21.15.39:0/3057774943 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc03f070 con 0x7f9a00102b50 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.378+0000 7f99f67fc640 1 -- 172.21.15.39:0/3057774943 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99fc031da0 con 0x7f9a00102b50 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.378+0000 7f9a076e0640 1 -- 172.21.15.39:0/3057774943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a001a2090 con 0x7f9a00102b50 2024-05-04T03:28:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.378+0000 7f99f67fc640 1 -- 172.21.15.39:0/3057774943 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc035030 con 0x7f9a00102b50 2024-05-04T03:28:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.378+0000 7f9a076e0640 1 -- 172.21.15.39:0/3057774943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001a25d0 con 0x7f9a00102b50 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.380+0000 7f9a0e58a740 1 Processor -- start 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.380+0000 7f9a0e58a740 1 -- start start 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcddd1b0 0x5634dd17ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcdb42f0 0x5634dd17efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd185ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5634dd0e35d0 con 0x5634dcdb42f0 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5634dd09c890 con 0x5634dcdbc4d0 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5634dd0df590 con 0x5634dcddd1b0 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a05c7d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcddd1b0 0x5634dd17ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a05c7d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcddd1b0 0x5634dd17ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:44038/0 (socket says 172.21.15.39:44038) 2024-05-04T03:28:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a05c7d640 1 -- 172.21.15.39:0/4046041305 learned_addr learned my addr 172.21.15.39:0/4046041305 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/4046041305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd185ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a05c7d640 1 -- 172.21.15.39:0/4046041305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 msgr2=0x5634dd185ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/4046041305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd185ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a05c7d640 1 -- 172.21.15.39:0/4046041305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcdb42f0 msgr2=0x5634dd17efe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/4046041305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcdb42f0 0x5634dd17efe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.381+0000 7f9a05c7d640 1 -- 172.21.15.39:0/4046041305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634dcdf5210 con 0x5634dcddd1b0 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.382+0000 7f99f67fc640 1 -- 172.21.15.39:0/3057774943 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f99fc042070 con 0x7f9a00102b50 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.382+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/4046041305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcddd1b0 0x5634dd17ea60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009ee0 tx=0x7f99f8010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6badfd0c28ca002a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.382+0000 7f99eb7fe640 1 -- 172.21.15.39:0/4046041305 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8016070 con 0x5634dcddd1b0 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.382+0000 7f99eb7fe640 1 -- 172.21.15.39:0/4046041305 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f8004d20 con 0x5634dcddd1b0 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.382+0000 7f99eb7fe640 1 -- 172.21.15.39:0/4046041305 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8012650 con 0x5634dcddd1b0 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.382+0000 7f99f67fc640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99ec0b2860 0x7f99ec0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.382+0000 7f9a0547c640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99ec0b2860 0x7f99ec0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f99f67fc640 1 -- 172.21.15.39:0/3057774943 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f99fc102af0 con 0x7f9a00102b50 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0547c640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99ec0b2860 0x7f99ec0b4d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0c9e20 tx=0x7f99fc08fb00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 -- 172.21.15.39:0/4046041305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcddd1b0 msgr2=0x5634dd17ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/4046041305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcddd1b0 0x5634dd17ea60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009ee0 tx=0x7f99f8010740 comp rx=0 tx=0).stop 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 -- 172.21.15.39:0/4046041305 shutdown_connections 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/4046041305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd185ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/4046041305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcdb42f0 0x5634dd17efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/4046041305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcddd1b0 0x5634dd17ea60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 -- 172.21.15.39:0/4046041305 >> 172.21.15.39:0/4046041305 conn(0x5634dcdd1300 msgr2=0x5634dd17a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 -- 172.21.15.39:0/4046041305 shutdown_connections 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.383+0000 7f9a0e58a740 1 -- 172.21.15.39:0/4046041305 wait complete. 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.384+0000 7f9a0e58a740 1 -- start start 2024-05-04T03:28:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.384+0000 7f9a0e58a740 1 Processor -- start 2024-05-04T03:28:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.384+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd188790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.384+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd188cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.384+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcddd1b0 0x5634dd18fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.384+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5634dd186930 con 0x5634dcddd1b0 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.384+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5634dd0a0370 con 0x5634dcdbc4d0 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.384+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5634dd186ab0 con 0x5634dcdb42f0 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a0e58a740 10 client.0 _ll_register_callbacks cb 0x5634dcdf4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcddd1b0 0x5634dd18fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcddd1b0 0x5634dd18fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43598/0 (socket says 172.21.15.39:43598) 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 -- 172.21.15.39:0/1170073368 learned_addr learned my addr 172.21.15.39:0/1170073368 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd188790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd188cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 -- 172.21.15.39:0/1170073368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 msgr2=0x5634dd188790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd188790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 -- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 msgr2=0x5634dd188cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd188cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 -- 172.21.15.39:0/1170073368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634dcec1620 con 0x5634dcddd1b0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd188cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcddd1b0 0x5634dd18fc30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f99f8005b40 tx=0x7f99f8009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6556dfe0bfae4896 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f99e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f99e8ff9640 1 -- 172.21.15.39:0/1170073368 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f801e030 con 0x5634dcddd1b0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f99e8ff9640 1 -- 172.21.15.39:0/1170073368 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f99f8012830 con 0x5634dcddd1b0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1170073368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f8004d20 con 0x5634dcddd1b0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.385+0000 7f99e8ff9640 1 -- 172.21.15.39:0/1170073368 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f99f8030050 con 0x5634dcddd1b0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a0e58a740 10 client.17256 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1170073368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634dd192b90 con 0x5634dcddd1b0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f99e8ff9640 1 -- 172.21.15.39:0/1170073368 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f8004d20 con 0x5634dcddd1b0 2024-05-04T03:28:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f99e8ff9640 1 -- 172.21.15.39:0/1170073368 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f99f800f040 con 0x5634dcddd1b0 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a0e58a740 10 client.17256 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a0e58a740 10 client.17256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a0e58a740 20 client.17256 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5634dcd9bd80 0x5634dd195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a0e58a740 4 client.17256 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1170073368 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634dd0f11e0 con 0x5634dcd9bd80 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.386+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5634dcd9bd80 0x5634dd195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.387+0000 7f99e8ff9640 10 client.17256 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.387+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5634dcd9bd80 0x5634dd195fb0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f300f26eba58ac7c server_cookie=5d4d2b36f56d0933 in_seq=0 out_seq=0 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.387+0000 7f99e8ff9640 1 -- 172.21.15.39:0/1170073368 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634dd0f11e0 con 0x5634dcd9bd80 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.387+0000 7f99e8ff9640 10 client.17256 handle_command_reply: tid=0 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 client.17256 shutdown 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1170073368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcddd1b0 msgr2=0x5634dd18fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcddd1b0 0x5634dd18fc30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f99f8005b40 tx=0x7f99f8009a40 comp rx=0 tx=0).stop 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1170073368 shutdown_connections 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5634dcd9bd80 0x5634dd195fb0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcddd1b0 0x5634dd18fc30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdbc4d0 0x5634dd188cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f99e8ff9640 0 client.17256 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1170073368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd188790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1170073368 >> 172.21.15.39:0/1170073368 conn(0x5634dcdd1300 msgr2=0x5634dd1793a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1170073368 shutdown_connections 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1170073368 wait complete. 2024-05-04T03:28:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.388+0000 7f9a0e58a740 20 client.17256 trim_cache size 0 max 16384 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.415+0000 7f9a0e58a740 1 Processor -- start 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.415+0000 7f9a0e58a740 1 -- start start 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.415+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 0x5634dd17a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.415+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.415+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd190100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.415+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ab2d0 con 0x5634dcd9bd80 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.415+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ace10 con 0x5634dcdd1300 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.415+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ab9a0 con 0x5634dcdb42f0 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43602/0 (socket says 172.21.15.39:43602) 2024-05-04T03:28:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 -- 172.21.15.39:0/1520970419 learned_addr learned my addr 172.21.15.39:0/1520970419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 0x5634dd17a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd190100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 -- 172.21.15.39:0/1520970419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 msgr2=0x5634dd190100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd190100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 -- 172.21.15.39:0/1520970419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 msgr2=0x5634dd17a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 0x5634dd17a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 -- 172.21.15.39:0/1520970419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634dd17a940 con 0x5634dcd9bd80 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd190100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18fbc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x5634dd17b180 tx=0x7f99f000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c924d86bff693208 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f99eaffd640 1 -- 172.21.15.39:0/1520970419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f001f020 con 0x5634dcd9bd80 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f99eaffd640 1 -- 172.21.15.39:0/1520970419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f001ba30 con 0x5634dcd9bd80 2024-05-04T03:28:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.416+0000 7f99eaffd640 1 -- 172.21.15.39:0/1520970419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0011d90 con 0x5634dcd9bd80 2024-05-04T03:28:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1520970419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 msgr2=0x5634dd18fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18fbc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x5634dd17b180 tx=0x7f99f000c330 comp rx=0 tx=0).stop 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1520970419 shutdown_connections 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd190100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18fbc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/1520970419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 0x5634dd17a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1520970419 >> 172.21.15.39:0/1520970419 conn(0x5634dd175520 msgr2=0x5634dd179cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1520970419 shutdown_connections 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.417+0000 7f9a0e58a740 1 -- 172.21.15.39:0/1520970419 wait complete. 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 1 -- start start 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 1 Processor -- start 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd18c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 0x5634dd18f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ac740 con 0x5634dcd9bd80 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001ac070 con 0x5634dcdd1300 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a001a92b0 con 0x5634dcdb42f0 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a0e58a740 10 client.0 _ll_register_callbacks cb 0x7f99f8009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a05c7d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a05c7d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43616/0 (socket says 172.21.15.39:43616) 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a05c7d640 1 -- 172.21.15.39:0/2498661349 learned_addr learned my addr 172.21.15.39:0/2498661349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 0x5634dd18f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.418+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd18c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a05c7d640 1 -- 172.21.15.39:0/2498661349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 msgr2=0x5634dd18c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd18c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a05c7d640 1 -- 172.21.15.39:0/2498661349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 msgr2=0x5634dd18f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 0x5634dd18f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a05c7d640 1 -- 172.21.15.39:0/2498661349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634dd18f8e0 con 0x5634dcd9bd80 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a05c7d640 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18bea0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f99f8005d70 tx=0x7f99f8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e2dc347e45f29c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f99e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f99e8ff9640 1 -- 172.21.15.39:0/2498661349 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f803b020 con 0x5634dcd9bd80 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f99e8ff9640 1 -- 172.21.15.39:0/2498661349 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f99f801cb60 con 0x5634dcd9bd80 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a0e58a740 1 -- 172.21.15.39:0/2498661349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f001ba30 con 0x5634dcd9bd80 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f99e8ff9640 1 -- 172.21.15.39:0/2498661349 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f99f801e040 con 0x5634dcd9bd80 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a0e58a740 10 client.17262 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f9a0e58a740 1 -- 172.21.15.39:0/2498661349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5634dd1fe560 con 0x5634dcd9bd80 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f99e8ff9640 1 -- 172.21.15.39:0/2498661349 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f001ba30 con 0x5634dcd9bd80 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.419+0000 7f99e8ff9640 1 -- 172.21.15.39:0/2498661349 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f99f801c180 con 0x5634dcd9bd80 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f9a0e58a740 10 client.17262 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f9a0e58a740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:28:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f9a0e58a740 20 client.17262 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5634dcdbc4d0 0x5634dd200e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f9a0e58a740 4 client.17262 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:28:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f9a0e58a740 1 -- 172.21.15.39:0/2498661349 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634dd0eb050 con 0x5634dcdbc4d0 2024-05-04T03:28:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5634dcdbc4d0 0x5634dd200e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f99e8ff9640 10 client.17262 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.420+0000 7f9a04c7b640 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5634dcdbc4d0 0x5634dd200e30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726aa867a7036206 server_cookie=7bb2b20a32c49daa in_seq=0 out_seq=0 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f99e8ff9640 1 -- 172.21.15.39:0/2498661349 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634dd0eb050 con 0x5634dcdbc4d0 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f99e8ff9640 10 client.17262 handle_command_reply: tid=0 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 client.17262 shutdown 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 -- 172.21.15.39:0/2498661349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 msgr2=0x5634dd18bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18bea0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f99f8005d70 tx=0x7f99f8009c20 comp rx=0 tx=0).stop 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 -- 172.21.15.39:0/2498661349 shutdown_connections 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5634dcdbc4d0 0x5634dd200e30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5634dcdd1300 0x5634dd18f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5634dcdb42f0 0x5634dd18c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f99e8ff9640 0 client.17262 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 --2- 172.21.15.39:0/2498661349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5634dcd9bd80 0x5634dd18bea0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.430+0000 7f9a0e58a740 1 -- 172.21.15.39:0/2498661349 >> 172.21.15.39:0/2498661349 conn(0x5634dd175520 msgr2=0x5634dd178970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a0e58a740 1 -- 172.21.15.39:0/2498661349 shutdown_connections 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a0e58a740 1 -- 172.21.15.39:0/2498661349 wait complete. 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a0e58a740 20 client.17262 trim_cache size 0 max 16384 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:28:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:28:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 -- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99ec0b2860 msgr2=0x7f99ec0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99ec0b2860 0x7f99ec0b4d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0c9e20 tx=0x7f99fc08fb00 comp rx=0 tx=0).stop 2024-05-04T03:28:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 -- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 msgr2=0x7f9a0006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 0x7f9a0006afc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005760 tx=0x7f99fc003d50 comp rx=0 tx=0).stop 2024-05-04T03:28:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 -- 172.21.15.39:0/3057774943 shutdown_connections 2024-05-04T03:28:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f99ec0b2860 0x7f99ec0b4d70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a00108a90 0x7f9a001a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a00103d90 0x7f9a0019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 --2- 172.21.15.39:0/3057774943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00102b50 0x7f9a0006afc0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 -- 172.21.15.39:0/3057774943 >> 172.21.15.39:0/3057774943 conn(0x7f9a000fdd40 msgr2=0x7f9a00104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 -- 172.21.15.39:0/3057774943 shutdown_connections 2024-05-04T03:28:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:28:56.431+0000 7f9a076e0640 1 -- 172.21.15.39:0/3057774943 wait complete. 2024-05-04T03:28:56.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.454+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7501a9bd0 con 0x7fa7501034c0 2024-05-04T03:28:56.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.455+0000 7fa74cff9700 1 -- 172.21.15.39:0/1813312780 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7fa740010070 con 0x7fa7501034c0 2024-05-04T03:28:56.457 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:28:56.458 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa73c0b2900 msgr2=0x7fa73c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa73c0b2900 0x7fa73c0b4e10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa750107540 tx=0x7fa738009260 comp rx=0 tx=0).stop 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7501034c0 msgr2=0x7fa75019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7501034c0 0x7fa75019cf20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa740007f10 tx=0x7fa74000d3b0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 shutdown_connections 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa73c0b2900 0x7fa73c0b4e10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7501034c0 0x7fa75019cf20 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa750108740 0x7fa7501a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 --2- 172.21.15.39:0/1813312780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa750107df0 0x7fa75019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:56.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 >> 172.21.15.39:0/1813312780 conn(0x7fa7500fedf0 msgr2=0x7fa750107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:56.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 shutdown_connections 2024-05-04T03:28:56.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:56.458+0000 7fa756167700 1 -- 172.21.15.39:0/1813312780 wait complete. 2024-05-04T03:28:56.461 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:28:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 905.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-04T03:28:56.785 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:56 smithi039 ceph-mon[29715]: pgmap v930: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 183 KiB/s wr, 33 op/s 2024-05-04T03:28:56.785 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:56 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1813312780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:28:56.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:56 smithi167 ceph-mon[33614]: pgmap v930: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 183 KiB/s wr, 33 op/s 2024-05-04T03:28:57.005 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:56 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1813312780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:28:57.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:56 smithi066 ceph-mon[33264]: pgmap v930: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 183 KiB/s wr, 33 op/s 2024-05-04T03:28:57.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:56 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1813312780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:28:57.133 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --set-id 6525321e --release --await' 2024-05-04T03:28:57.133 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --set-id 6525321e --release --await 2024-05-04T03:28:57.387 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:28:58.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:28:58 smithi039 ceph-mon[29715]: pgmap v931: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 192 KiB/s wr, 36 op/s 2024-05-04T03:28:58.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:28:58 smithi167 ceph-mon[33614]: pgmap v931: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 192 KiB/s wr, 36 op/s 2024-05-04T03:28:59.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:28:58 smithi066 ceph-mon[33264]: pgmap v931: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 192 KiB/s wr, 36 op/s 2024-05-04T03:28:59.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 -- 172.21.15.39:0/2861269767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c8109850 msgr2=0x7fe6c81111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/2861269767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c8109850 0x7fe6c81111b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009fc0 tx=0x7fe6c000b330 comp rx=0 tx=0).stop 2024-05-04T03:28:59.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 -- 172.21.15.39:0/2861269767 shutdown_connections 2024-05-04T03:28:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/2861269767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c8109850 0x7fe6c81111b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/2861269767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c81045d0 0x7fe6c81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/2861269767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c8108f00 0x7fe6c8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 -- 172.21.15.39:0/2861269767 >> 172.21.15.39:0/2861269767 conn(0x7fe6c80ffec0 msgr2=0x7fe6c8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 -- 172.21.15.39:0/2861269767 shutdown_connections 2024-05-04T03:28:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.083+0000 7fe6cfd49700 1 -- 172.21.15.39:0/2861269767 wait complete. 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cfd49700 1 Processor -- start 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cfd49700 1 -- start start 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cfd49700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c81045d0 0x7fe6c81a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cfd49700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c8108f00 0x7fe6c81a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cfd49700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c8109850 0x7fe6c819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cfd49700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8114330 con 0x7fe6c81045d0 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cfd49700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81141b0 con 0x7fe6c8109850 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cfd49700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81144b0 con 0x7fe6c8108f00 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cdae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c81045d0 0x7fe6c81a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cdae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c81045d0 0x7fe6c81a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43638/0 (socket says 172.21.15.39:43638) 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cdae5700 1 -- 172.21.15.39:0/3441274131 learned_addr learned my addr 172.21.15.39:0/3441274131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.084+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c8108f00 0x7fe6c81a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6ce2e6700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c8109850 0x7fe6c819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6cdae5700 1 -- 172.21.15.39:0/3441274131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c8108f00 msgr2=0x7fe6c81a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c8108f00 0x7fe6c81a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6cdae5700 1 -- 172.21.15.39:0/3441274131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c8109850 msgr2=0x7fe6c819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c8109850 0x7fe6c819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6cdae5700 1 -- 172.21.15.39:0/3441274131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c819b1d0 con 0x7fe6c81045d0 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c81045d0 0x7fe6c81a08b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d950 tx=0x7fe6c400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6beffd700 1 -- 172.21.15.39:0/3441274131 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4016070 con 0x7fe6c81045d0 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6beffd700 1 -- 172.21.15.39:0/3441274131 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c4009dc0 con 0x7fe6c81045d0 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6beffd700 1 -- 172.21.15.39:0/3441274131 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400f3c0 con 0x7fe6c81045d0 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6cfd49700 1 -- 172.21.15.39:0/3441274131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c819b4c0 con 0x7fe6c81045d0 2024-05-04T03:28:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.085+0000 7fe6cfd49700 1 -- 172.21.15.39:0/3441274131 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c8076b00 con 0x7fe6c81045d0 2024-05-04T03:28:59.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.088+0000 7fe6beffd700 1 -- 172.21.15.39:0/3441274131 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe6c4019070 con 0x7fe6c81045d0 2024-05-04T03:28:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.089+0000 7fe6beffd700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6b40b2880 0x7fe6b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.089+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6b40b2880 0x7fe6b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.089+0000 7fe6beffd700 1 -- 172.21.15.39:0/3441274131 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe6c40e1e60 con 0x7fe6c81045d0 2024-05-04T03:28:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.089+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6b40b2880 0x7fe6b40b4d90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8103460 tx=0x7fe6b8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:59.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6de75db80 1 Processor -- start 2024-05-04T03:28:59.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6de75db80 1 -- start start 2024-05-04T03:28:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ab8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ac3c40 0x558612ac6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6de75db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5586129dba40 con 0x558612ab8ad0 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6de75db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558612940ea0 con 0x558612ab9420 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6de75db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5586129d7a00 con 0x558612ac3c40 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6cdae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ab8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6cdae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ab8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43644/0 (socket says 172.21.15.39:43644) 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6cdae5700 1 -- 172.21.15.39:0/3183754464 learned_addr learned my addr 172.21.15.39:0/3183754464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.090+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6ce2e6700 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ac3c40 0x558612ac6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6cdae5700 1 -- 172.21.15.39:0/3183754464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ac3c40 msgr2=0x558612ac6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ac3c40 0x558612ac6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6cdae5700 1 -- 172.21.15.39:0/3183754464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 msgr2=0x558612ac3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6cdae5700 1 -- 172.21.15.39:0/3183754464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586129289d0 con 0x558612ab8ad0 2024-05-04T03:28:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ab8ee0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40df850 tx=0x7fe6c40a8fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bb15fe705900bd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6b3fff700 1 -- 172.21.15.39:0/3183754464 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4020070 con 0x558612ab8ad0 2024-05-04T03:28:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6b3fff700 1 -- 172.21.15.39:0/3183754464 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c40a9430 con 0x558612ab8ad0 2024-05-04T03:28:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.091+0000 7fe6b3fff700 1 -- 172.21.15.39:0/3183754464 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40b1f80 con 0x558612ab8ad0 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 -- 172.21.15.39:0/3183754464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 msgr2=0x558612ab8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ab8ee0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40df850 tx=0x7fe6c40a8fc0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 -- 172.21.15.39:0/3183754464 shutdown_connections 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ab8ee0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ac3c40 0x558612ac6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 --2- 172.21.15.39:0/3183754464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 -- 172.21.15.39:0/3183754464 >> 172.21.15.39:0/3183754464 conn(0x55861293bf40 msgr2=0x558612ab49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 -- 172.21.15.39:0/3183754464 shutdown_connections 2024-05-04T03:28:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.092+0000 7fe6de75db80 1 -- 172.21.15.39:0/3183754464 wait complete. 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 1 -- start start 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 1 Processor -- start 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ac0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ac11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ac3c40 0x558612ac16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558612ac6950 con 0x558612ab8ad0 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558612994fd0 con 0x558612ac3c40 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558612ac6ad0 con 0x558612ab9420 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6de75db80 10 client.0 _ll_register_callbacks cb 0x558612973a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6cdae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ac0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6cdae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ac0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43646/0 (socket says 172.21.15.39:43646) 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.093+0000 7fe6cdae5700 1 -- 172.21.15.39:0/1405680318 learned_addr learned my addr 172.21.15.39:0/1405680318 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ac11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6ce2e6700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ac3c40 0x558612ac16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6cdae5700 1 -- 172.21.15.39:0/1405680318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 msgr2=0x558612ac11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ac11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6cdae5700 1 -- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ac3c40 msgr2=0x558612ac16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ac3c40 0x558612ac16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6cdae5700 1 -- 172.21.15.39:0/1405680318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558612992860 con 0x558612ab8ad0 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ac11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6ce2e6700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ac3c40 0x558612ac16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ac0c60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40ad9a0 tx=0x7fe6c40108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f59713534ca304f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1405680318 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4020070 con 0x558612ab8ad0 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1405680318 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe6c40a5a20 con 0x558612ab8ad0 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6de75db80 1 -- 172.21.15.39:0/1405680318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c40a9430 con 0x558612ab8ad0 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1405680318 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe6c40b2290 con 0x558612ab8ad0 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6de75db80 10 client.17274 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.094+0000 7fe6de75db80 1 -- 172.21.15.39:0/1405680318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558612ad15d0 con 0x558612ab8ad0 2024-05-04T03:28:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1405680318 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c40a9430 con 0x558612ab8ad0 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1405680318 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe6c40a3cc0 con 0x558612ab8ad0 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6de75db80 10 client.17274 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6de75db80 10 client.17274 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6de75db80 20 client.17274 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558612ad3aa0 0x558612ad5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6de75db80 4 client.17274 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6de75db80 1 -- 172.21.15.39:0/1405680318 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586129e9420 con 0x558612ad3aa0 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.095+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558612ad3aa0 0x558612ad5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.096+0000 7fe6b17fa700 10 client.17274 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.096+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558612ad3aa0 0x558612ad5ec0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921dee3f0d937ca2 server_cookie=eedc2f0cb7bcc642 in_seq=0 out_seq=0 2024-05-04T03:28:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.096+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1405680318 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586129e9420 con 0x558612ad3aa0 2024-05-04T03:28:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.096+0000 7fe6b17fa700 10 client.17274 handle_command_reply: tid=0 2024-05-04T03:28:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 client.17274 shutdown 2024-05-04T03:28:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 -- 172.21.15.39:0/1405680318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 msgr2=0x558612ac0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ac0c60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40ad9a0 tx=0x7fe6c40108b0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 -- 172.21.15.39:0/1405680318 shutdown_connections 2024-05-04T03:28:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558612ad3aa0 0x558612ad5ec0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ac0c60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ac3c40 0x558612ac16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1405680318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ac11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6b17fa700 0 client.17274 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 -- 172.21.15.39:0/1405680318 >> 172.21.15.39:0/1405680318 conn(0x55861293bf40 msgr2=0x558612aba4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 -- 172.21.15.39:0/1405680318 shutdown_connections 2024-05-04T03:28:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 1 -- 172.21.15.39:0/1405680318 wait complete. 2024-05-04T03:28:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.097+0000 7fe6de75db80 20 client.17274 trim_cache size 0 max 16384 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.117+0000 7fe6de75db80 1 Processor -- start 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6de75db80 1 -- start start 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55861293bf40 0x558612ab4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab8ad0 0x558612acfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6de75db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81b0e10 con 0x55861293bf40 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6de75db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81b2950 con 0x558612ab9420 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6de75db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81b14e0 con 0x558612ab8ad0 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55861293bf40 0x558612ab4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55861293bf40 0x558612ab4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43654/0 (socket says 172.21.15.39:43654) 2024-05-04T03:28:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 -- 172.21.15.39:0/63465752 learned_addr learned my addr 172.21.15.39:0/63465752 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab8ad0 0x558612acfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6ce2e6700 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 -- 172.21.15.39:0/63465752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab8ad0 msgr2=0x558612acfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab8ad0 0x558612acfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 -- 172.21.15.39:0/63465752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 msgr2=0x558612ac2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 -- 172.21.15.39:0/63465752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558612ac6ac0 con 0x55861293bf40 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6ce2e6700 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.118+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55861293bf40 0x558612ab4840 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40b2450 tx=0x7fe6c40a8fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f319869458f15c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.119+0000 7fe6b37fe700 1 -- 172.21.15.39:0/63465752 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4020070 con 0x55861293bf40 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.119+0000 7fe6b37fe700 1 -- 172.21.15.39:0/63465752 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c406e020 con 0x55861293bf40 2024-05-04T03:28:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.119+0000 7fe6b37fe700 1 -- 172.21.15.39:0/63465752 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40b2010 con 0x55861293bf40 2024-05-04T03:28:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 -- 172.21.15.39:0/63465752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55861293bf40 msgr2=0x558612ab4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55861293bf40 0x558612ab4840 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40b2450 tx=0x7fe6c40a8fc0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 -- 172.21.15.39:0/63465752 shutdown_connections 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55861293bf40 0x558612ab4840 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558612ab9420 0x558612ac2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 --2- 172.21.15.39:0/63465752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab8ad0 0x558612acfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 -- 172.21.15.39:0/63465752 >> 172.21.15.39:0/63465752 conn(0x558612aadb60 msgr2=0x558612ac2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 -- 172.21.15.39:0/63465752 shutdown_connections 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 -- 172.21.15.39:0/63465752 wait complete. 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 -- start start 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.120+0000 7fe6de75db80 1 Processor -- start 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55861293bf40 0x558612ad91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ad7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6de75db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ad7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6de75db80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81b2280 con 0x558612ab8ad0 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6de75db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81b1bb0 con 0x55861293bf40 2024-05-04T03:28:59.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6de75db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81aedd0 con 0x558612ab9420 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6de75db80 10 client.0 _ll_register_callbacks cb 0x558612973a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ad7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ad7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43668/0 (socket says 172.21.15.39:43668) 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 -- 172.21.15.39:0/1024563229 learned_addr learned my addr 172.21.15.39:0/1024563229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6ce2e6700 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ad7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55861293bf40 0x558612ad91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 -- 172.21.15.39:0/1024563229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 msgr2=0x558612ad7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ad7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 -- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55861293bf40 msgr2=0x558612ad91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55861293bf40 0x558612ad91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 -- 172.21.15.39:0/1024563229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558612ad7fd0 con 0x558612ab8ad0 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55861293bf40 0x558612ad91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6cd2e4700 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ad7550 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009a00 tx=0x7fe6b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b2819489ead578 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.121+0000 7fe6b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:28:59.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1024563229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8003cf0 con 0x558612ab8ad0 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 1 -- 172.21.15.39:0/1024563229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c406e020 con 0x558612ab8ad0 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1024563229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe6b8004830 con 0x558612ab8ad0 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1024563229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe6b8003e90 con 0x558612ab8ad0 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 10 client.17280 fetch_fsmap learned FSMap version 32 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 1 -- 172.21.15.39:0/1024563229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558612b47550 con 0x558612ab8ad0 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1024563229 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c406e020 con 0x558612ab8ad0 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1024563229 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe6b8028ce0 con 0x558612ab8ad0 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 10 client.17280 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 10 client.17280 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 20 client.17280 populate_metadata read hostname 'smithi039' 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558612ac3c40 0x558612b49cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 4 client.17280 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6525321e", "release": true, "await": true}] 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.122+0000 7fe6de75db80 1 -- 172.21.15.39:0/1024563229 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6525321e", "release": true, "await": true}) -- 0x5586129e34c0 con 0x558612ac3c40 2024-05-04T03:28:59.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.123+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558612ac3c40 0x558612b49cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:28:59.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.123+0000 7fe6cdae5700 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558612ac3c40 0x558612b49cd0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574d938ceeef2de8 server_cookie=429de2e68cc0d50f in_seq=0 out_seq=0 2024-05-04T03:28:59.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.123+0000 7fe6b17fa700 10 client.17280 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:59.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.128+0000 7fe6b17fa700 1 -- 172.21.15.39:0/1024563229 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5586129e34c0 con 0x558612ac3c40 2024-05-04T03:28:59.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.128+0000 7fe6b17fa700 10 client.17280 handle_command_reply: tid=0 2024-05-04T03:28:59.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.128+0000 7fe6de75db80 1 client.17280 shutdown 2024-05-04T03:28:59.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 -- 172.21.15.39:0/1024563229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 msgr2=0x558612ad7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ad7550 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009a00 tx=0x7fe6b80047c0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 -- 172.21.15.39:0/1024563229 shutdown_connections 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558612ac3c40 0x558612b49cd0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558612ab9420 0x558612ad7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55861293bf40 0x558612ad91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 --2- 172.21.15.39:0/1024563229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558612ab8ad0 0x558612ad7550 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6b17fa700 0 client.17280 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 -- 172.21.15.39:0/1024563229 >> 172.21.15.39:0/1024563229 conn(0x558612aadb60 msgr2=0x558612ab45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 -- 172.21.15.39:0/1024563229 shutdown_connections 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 1 -- 172.21.15.39:0/1024563229 wait complete. 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.129+0000 7fe6de75db80 20 client.17280 trim_cache size 0 max 16384 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:28:59.132 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 48, 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "6525321e": { 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 48, 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 80.9, 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:28:59.133 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:28:59.134 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 -- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6b40b2880 msgr2=0x7fe6b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6b40b2880 0x7fe6b40b4d90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8103460 tx=0x7fe6b8009310 comp rx=0 tx=0).stop 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 -- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c81045d0 msgr2=0x7fe6c81a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c81045d0 0x7fe6c81a08b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d950 tx=0x7fe6c400dd10 comp rx=0 tx=0).stop 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 -- 172.21.15.39:0/3441274131 shutdown_connections 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe6b40b2880 0x7fe6b40b4d90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c81045d0 0x7fe6c81a08b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6c8109850 0x7fe6c819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 --2- 172.21.15.39:0/3441274131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c8108f00 0x7fe6c81a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 -- 172.21.15.39:0/3441274131 >> 172.21.15.39:0/3441274131 conn(0x7fe6c80ffec0 msgr2=0x7fe6c81086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 -- 172.21.15.39:0/3441274131 shutdown_connections 2024-05-04T03:28:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:28:59.130+0000 7fe6cfd49700 1 -- 172.21.15.39:0/3441274131 wait complete. 2024-05-04T03:28:59.375 INFO:teuthology.orchestra.run.smithi039.stdout:transaction type: 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:scaling factor: 32 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:query mode: simple 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:number of clients: 100 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:number of threads: 4 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:duration: 900 s 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:number of transactions actually processed: 759959 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:latency average = 119.360 ms 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:latency stddev = 1168.482 ms 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:tps = 837.329411 (including connections establishing) 2024-05-04T03:28:59.376 INFO:teuthology.orchestra.run.smithi039.stdout:tps = 837.340557 (excluding connections establishing) 2024-05-04T03:28:59.381 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-04T03:28:59.985 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6525321e', seconds elapsed: 35.4 2024-05-04T03:28:59.985 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.9 2024-05-04T03:28:59.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:29:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:00.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:00 smithi039 ceph-mon[29715]: pgmap v932: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-05-04T03:29:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:00 smithi167 ceph-mon[33614]: pgmap v932: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-05-04T03:29:01.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:00 smithi066 ceph-mon[33264]: pgmap v932: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-05-04T03:29:02.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.659+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/2705115382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a3811d930 msgr2=0x7f4a3811fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:02.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.659+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/2705115382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a3811d930 0x7f4a3811fd50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009f90 tx=0x7f4a3000b330 comp rx=0 tx=0).stop 2024-05-04T03:29:02.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.660+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/2705115382 shutdown_connections 2024-05-04T03:29:02.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.660+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/2705115382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a3811d930 0x7f4a3811fd50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:02.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.660+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/2705115382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a3810a930 0x7f4a3810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:02.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.660+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/2705115382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3810b280 0x7f4a3811d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:02.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.660+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/2705115382 >> 172.21.15.39:0/2705115382 conn(0x7f4a3806f4d0 msgr2=0x7f4a3806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:02.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.664+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/2705115382 shutdown_connections 2024-05-04T03:29:02.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.664+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/2705115382 wait complete. 2024-05-04T03:29:02.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.665+0000 7f4a3e1e3700 1 Processor -- start 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.665+0000 7f4a3e1e3700 1 -- start start 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a3e1e3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3810a930 0x7f4a3811bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a3e1e3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a3810b280 0x7f4a38114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a3e1e3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a3811d930 0x7f4a38115560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a3e1e3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38123170 con 0x7f4a3810a930 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a3e1e3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38122ff0 con 0x7f4a3810b280 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a3e1e3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a381232f0 con 0x7f4a3811d930 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a3811d930 0x7f4a38115560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a3811d930 0x7f4a38115560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:44132/0 (socket says 172.21.15.39:44132) 2024-05-04T03:29:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a37fff700 1 -- 172.21.15.39:0/1029056659 learned_addr learned my addr 172.21.15.39:0/1029056659 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a377fe700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3810a930 0x7f4a3811bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a37fff700 1 -- 172.21.15.39:0/1029056659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a3810b280 msgr2=0x7f4a38114f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:29:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a37fff700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a3810b280 0x7f4a38114f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a37fff700 1 -- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3810a930 msgr2=0x7f4a3811bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a37fff700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3810a930 0x7f4a3811bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.666+0000 7f4a37fff700 1 -- 172.21.15.39:0/1029056659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a38115d30 con 0x7f4a3811d930 2024-05-04T03:29:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.667+0000 7f4a37fff700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a3811d930 0x7f4a38115560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009f60 tx=0x7f4a3000ce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.667+0000 7f4a34ff9700 1 -- 172.21.15.39:0/1029056659 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30021070 con 0x7f4a3811d930 2024-05-04T03:29:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.667+0000 7f4a34ff9700 1 -- 172.21.15.39:0/1029056659 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a30007c00 con 0x7f4a3811d930 2024-05-04T03:29:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.668+0000 7f4a34ff9700 1 -- 172.21.15.39:0/1029056659 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a300052a0 con 0x7f4a3811d930 2024-05-04T03:29:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.668+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a381199d0 con 0x7f4a3811d930 2024-05-04T03:29:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.669+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a38119e90 con 0x7f4a3811d930 2024-05-04T03:29:02.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.673+0000 7f4a34ff9700 1 -- 172.21.15.39:0/1029056659 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f4a30024070 con 0x7f4a3811d930 2024-05-04T03:29:02.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.679+0000 7f4a34ff9700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a200b2830 0x7f4a200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:02.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.679+0000 7f4a34ff9700 1 -- 172.21.15.39:0/1029056659 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f4a3000f070 con 0x7f4a3811d930 2024-05-04T03:29:02.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.680+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3810c070 con 0x7f4a3811d930 2024-05-04T03:29:02.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.685+0000 7f4a377fe700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a200b2830 0x7f4a200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:02.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.689+0000 7f4a34ff9700 1 -- 172.21.15.39:0/1029056659 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a3002b020 con 0x7f4a3811d930 2024-05-04T03:29:02.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:02.692+0000 7f4a377fe700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a200b2830 0x7f4a200b4d40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4a28005ea0 tx=0x7f4a28005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:02.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:02 smithi167 ceph-mon[33614]: pgmap v933: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-05-04T03:29:02.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:02 smithi039 ceph-mon[29715]: pgmap v933: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-05-04T03:29:03.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:02 smithi066 ceph-mon[33264]: pgmap v933: 97 pgs: 97 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-05-04T03:29:03.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.233+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a380512c0 con 0x7f4a3811d930 2024-05-04T03:29:03.239 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:03.239 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:29:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.237+0000 7f4a34ff9700 1 -- 172.21.15.39:0/1029056659 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f4a300aac00 con 0x7f4a3811d930 2024-05-04T03:29:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.252+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a200b2830 msgr2=0x7f4a200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.252+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a200b2830 0x7f4a200b4d40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4a28005ea0 tx=0x7f4a28005e30 comp rx=0 tx=0).stop 2024-05-04T03:29:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.252+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a3811d930 msgr2=0x7f4a38115560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.252+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a3811d930 0x7f4a38115560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009f60 tx=0x7f4a3000ce20 comp rx=0 tx=0).stop 2024-05-04T03:29:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.254+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 shutdown_connections 2024-05-04T03:29:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.254+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f4a200b2830 0x7f4a200b4d40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.254+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3810a930 0x7f4a3811bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.254+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a3811d930 0x7f4a38115560 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.254+0000 7f4a3e1e3700 1 --2- 172.21.15.39:0/1029056659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4a3810b280 0x7f4a38114f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.254+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 >> 172.21.15.39:0/1029056659 conn(0x7f4a3806f4d0 msgr2=0x7f4a38072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:03.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.256+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 shutdown_connections 2024-05-04T03:29:03.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:03.258+0000 7f4a3e1e3700 1 -- 172.21.15.39:0/1029056659 wait complete. 2024-05-04T03:29:03.270 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:29:03.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:03 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1029056659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:29:03.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:03 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1029056659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:29:03.978 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db / --timeout 60 --expiration 80.9 --await-for 42.0' 2024-05-04T03:29:03.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db / --timeout 60 --expiration 80.9 --await-for 42.0 2024-05-04T03:29:04.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:03 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1029056659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:29:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:04 smithi167 ceph-mon[33614]: pgmap v934: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.8 MiB/s wr, 321 op/s 2024-05-04T03:29:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:04 smithi039 ceph-mon[29715]: pgmap v934: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.8 MiB/s wr, 321 op/s 2024-05-04T03:29:05.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:04 smithi066 ceph-mon[33264]: pgmap v934: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.8 MiB/s wr, 321 op/s 2024-05-04T03:29:06.230 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/2632477318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34108740 msgr2=0x7f9d341100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/2632477318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34108740 0x7f9d341100a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00b0d0 tx=0x7f9d2c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/2632477318 shutdown_connections 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/2632477318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34108740 0x7f9d341100a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/2632477318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d341034c0 0x7f9d341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/2632477318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d34107df0 0x7f9d34108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/2632477318 >> 172.21.15.39:0/2632477318 conn(0x7f9d340fedc0 msgr2=0x7f9d341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/2632477318 shutdown_connections 2024-05-04T03:29:06.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.440+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/2632477318 wait complete. 2024-05-04T03:29:06.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.441+0000 7f9d3bcd5700 1 Processor -- start 2024-05-04T03:29:06.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.441+0000 7f9d3bcd5700 1 -- start start 2024-05-04T03:29:06.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d3bcd5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d341034c0 0x7f9d341a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d3bcd5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34107df0 0x7f9d341a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d3bcd5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d34108740 0x7f9d3419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d3bcd5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34113220 con 0x7f9d34107df0 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d3bcd5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341130a0 con 0x7f9d34108740 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d3bcd5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341133a0 con 0x7f9d341034c0 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34107df0 0x7f9d341a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34107df0 0x7f9d341a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44900/0 (socket says 172.21.15.39:44900) 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 -- 172.21.15.39:0/1657193767 learned_addr learned my addr 172.21.15.39:0/1657193767 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39a71700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d341034c0 0x7f9d341a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d3a272700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d34108740 0x7f9d3419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 -- 172.21.15.39:0/1657193767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d341034c0 msgr2=0x7f9d341a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d341034c0 0x7f9d341a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 -- 172.21.15.39:0/1657193767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d34108740 msgr2=0x7f9d3419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d34108740 0x7f9d3419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 -- 172.21.15.39:0/1657193767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3419b260 con 0x7f9d34107df0 2024-05-04T03:29:06.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d3a272700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d34108740 0x7f9d3419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:29:06.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39a71700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d341034c0 0x7f9d341a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:29:06.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.442+0000 7f9d39270700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34107df0 0x7f9d341a0ee0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400d980 tx=0x7f9d2400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:06.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.443+0000 7f9d2affd700 1 -- 172.21.15.39:0/1657193767 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24017070 con 0x7f9d34107df0 2024-05-04T03:29:06.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.443+0000 7f9d2affd700 1 -- 172.21.15.39:0/1657193767 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d24010460 con 0x7f9d34107df0 2024-05-04T03:29:06.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.443+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/1657193767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3419b550 con 0x7f9d34107df0 2024-05-04T03:29:06.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.443+0000 7f9d2affd700 1 -- 172.21.15.39:0/1657193767 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400de60 con 0x7f9d34107df0 2024-05-04T03:29:06.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.443+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/1657193767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d341a7750 con 0x7f9d34107df0 2024-05-04T03:29:06.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.446+0000 7f9d2affd700 1 -- 172.21.15.39:0/1657193767 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f9d24010610 con 0x7f9d34107df0 2024-05-04T03:29:06.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.447+0000 7f9d2affd700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d200b2880 0x7f9d200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.447+0000 7f9d39a71700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d200b2880 0x7f9d200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.447+0000 7f9d2affd700 1 -- 172.21.15.39:0/1657193767 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9d240e1ce0 con 0x7f9d34107df0 2024-05-04T03:29:06.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.447+0000 7f9d39a71700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d200b2880 0x7f9d200b4d90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009cb0 tx=0x7f9d30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:06.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.447+0000 7f9d4a6e9b80 1 Processor -- start 2024-05-04T03:29:06.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d4a6e9b80 1 -- start start 2024-05-04T03:29:06.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506ca30 0x558ec506ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 0x558ec507bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec507c0e0 0x558ec507e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558ec4f8fad0 con 0x558ec507c0e0 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558ec4ef4f30 con 0x558ec506ca30 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558ec4f8ba90 con 0x558ec506d380 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d39a71700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506ca30 0x558ec506ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d39a71700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506ca30 0x558ec506ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:41848/0 (socket says 172.21.15.39:41848) 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d39a71700 1 -- 172.21.15.39:0/647440701 learned_addr learned my addr 172.21.15.39:0/647440701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d3a272700 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec507c0e0 0x558ec507e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.448+0000 7f9d39270700 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 0x558ec507bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d39a71700 1 -- 172.21.15.39:0/647440701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 msgr2=0x558ec507bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d39a71700 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 0x558ec507bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d39a71700 1 -- 172.21.15.39:0/647440701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec507c0e0 msgr2=0x558ec507e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:06.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d39a71700 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec507c0e0 0x558ec507e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d39a71700 1 -- 172.21.15.39:0/647440701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ec4edca60 con 0x558ec506ca30 2024-05-04T03:29:06.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d39a71700 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506ca30 0x558ec506ce40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d30011fb0 tx=0x7f9d30014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60cffe8beb62a25f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:06.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d1ffff700 1 -- 172.21.15.39:0/647440701 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30025070 con 0x558ec506ca30 2024-05-04T03:29:06.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d1ffff700 1 -- 172.21.15.39:0/647440701 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d30016070 con 0x558ec506ca30 2024-05-04T03:29:06.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.449+0000 7f9d1ffff700 1 -- 172.21.15.39:0/647440701 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d300129f0 con 0x558ec506ca30 2024-05-04T03:29:06.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.450+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/647440701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506ca30 msgr2=0x558ec506ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.450+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506ca30 0x558ec506ce40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d30011fb0 tx=0x7f9d30014000 comp rx=0 tx=0).stop 2024-05-04T03:29:06.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.450+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/647440701 shutdown_connections 2024-05-04T03:29:06.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.450+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec507c0e0 0x558ec507e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.450+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506ca30 0x558ec506ce40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.450+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/647440701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 0x558ec507bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.450+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/647440701 >> 172.21.15.39:0/647440701 conn(0x558ec4eeffd0 msgr2=0x558ec5068a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.451+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/647440701 shutdown_connections 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.451+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/647440701 wait complete. 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.451+0000 7f9d4a6e9b80 1 -- start start 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.451+0000 7f9d4a6e9b80 1 Processor -- start 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec50790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 0x558ec50795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec507c0e0 0x558ec5079b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558ec507eeb0 con 0x558ec506ca30 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x558ec4f49060 con 0x558ec507c0e0 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558ec507f030 con 0x558ec506d380 2024-05-04T03:29:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d4a6e9b80 10 client.0 _ll_register_callbacks cb 0x558ec4f27ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39a71700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec50790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39a71700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec50790b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44908/0 (socket says 172.21.15.39:44908) 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39a71700 1 -- 172.21.15.39:0/3561416910 learned_addr learned my addr 172.21.15.39:0/3561416910 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39270700 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 0x558ec50795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d3a272700 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec507c0e0 0x558ec5079b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39a71700 1 -- 172.21.15.39:0/3561416910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 msgr2=0x558ec50795f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39a71700 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 0x558ec50795f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39a71700 1 -- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec507c0e0 msgr2=0x558ec5079b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39a71700 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec507c0e0 0x558ec5079b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d39a71700 1 -- 172.21.15.39:0/3561416910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ec4f468f0 con 0x558ec506ca30 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.452+0000 7f9d3a272700 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec507c0e0 0x558ec5079b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.453+0000 7f9d39a71700 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec50790b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009b10 tx=0x7f9d30012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dff351923f3816c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.453+0000 7f9d1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.453+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/3561416910 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30030020 con 0x558ec506ca30 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.453+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/3561416910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d30016070 con 0x558ec506ca30 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.453+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/3561416910 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9d30003680 con 0x558ec506ca30 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.453+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/3561416910 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d30025030 con 0x558ec506ca30 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.453+0000 7f9d4a6e9b80 10 client.17298 fetch_fsmap learned FSMap version 32 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.453+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/3561416910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ec5089ae0 con 0x558ec506ca30 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/3561416910 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d30016070 con 0x558ec506ca30 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/3561416910 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9d3002b070 con 0x558ec506ca30 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d4a6e9b80 10 client.17298 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d4a6e9b80 10 client.17298 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d4a6e9b80 20 client.17298 populate_metadata read hostname 'smithi039' 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558ec508bfb0 0x558ec508e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d4a6e9b80 4 client.17298 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:29:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/3561416910 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ec4f9d4b0 con 0x558ec508bfb0 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.454+0000 7f9d39270700 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558ec508bfb0 0x558ec508e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.455+0000 7f9d39270700 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558ec508bfb0 0x558ec508e3d0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9667d1ee9b2bca server_cookie=455fae0c83ba02df in_seq=0 out_seq=0 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.455+0000 7f9d1d7fa700 10 client.17298 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.455+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/3561416910 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ec4f9d4b0 con 0x558ec508bfb0 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.455+0000 7f9d1d7fa700 10 client.17298 handle_command_reply: tid=0 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.455+0000 7f9d4a6e9b80 1 client.17298 shutdown 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/3561416910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 msgr2=0x558ec50790b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec50790b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009b10 tx=0x7f9d30012680 comp rx=0 tx=0).stop 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/3561416910 shutdown_connections 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558ec508bfb0 0x558ec508e3d0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec50790b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec507c0e0 0x558ec5079b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/3561416910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec506d380 0x558ec50795f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/3561416910 >> 172.21.15.39:0/3561416910 conn(0x558ec4eeffd0 msgr2=0x558ec506e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d1d7fa700 0 client.17298 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/3561416910 shutdown_connections 2024-05-04T03:29:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/3561416910 wait complete. 2024-05-04T03:29:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.456+0000 7f9d4a6e9b80 20 client.17298 trim_cache size 0 max 16384 2024-05-04T03:29:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d4a6e9b80 1 Processor -- start 2024-05-04T03:29:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d4a6e9b80 1 -- start start 2024-05-04T03:29:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec50882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec507b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 0x558ec50795a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341ac9a0 con 0x558ec506ca30 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341ae4e0 con 0x558ec506d380 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341ad070 con 0x558ec4eeffd0 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d39270700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec507b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d39270700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec507b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44910/0 (socket says 172.21.15.39:44910) 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d39270700 1 -- 172.21.15.39:0/2895129282 learned_addr learned my addr 172.21.15.39:0/2895129282 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d39a71700 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec50882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.484+0000 7f9d3a272700 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 0x558ec50795a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d39270700 1 -- 172.21.15.39:0/2895129282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 msgr2=0x558ec50882d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d39270700 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec50882d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d39270700 1 -- 172.21.15.39:0/2895129282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 msgr2=0x558ec50795a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:06.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d39270700 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 0x558ec50795a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d39270700 1 -- 172.21.15.39:0/2895129282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ec5077620 con 0x558ec506ca30 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d39a71700 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec50882d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d39270700 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec507b190 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9d240a9010 tx=0x7f9d24091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7103035d35caca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/2895129282 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d240c2020 con 0x558ec506ca30 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/2895129282 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d240b2170 con 0x558ec506ca30 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.485+0000 7f9d1f7fe700 1 -- 172.21.15.39:0/2895129282 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24020070 con 0x558ec506ca30 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/2895129282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 msgr2=0x558ec507b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec507b190 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9d240a9010 tx=0x7f9d24091040 comp rx=0 tx=0).stop 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/2895129282 shutdown_connections 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 0x558ec50795a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec50882d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/2895129282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec507b190 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/2895129282 >> 172.21.15.39:0/2895129282 conn(0x558ec5061bf0 msgr2=0x558ec5078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/2895129282 shutdown_connections 2024-05-04T03:29:06.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.486+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/2895129282 wait complete. 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 1 -- start start 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 1 Processor -- start 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec508ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec508f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 0x558ec50f5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341ade10 con 0x558ec506ca30 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341ad740 con 0x558ec506d380 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341aa960 con 0x558ec4eeffd0 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.487+0000 7f9d4a6e9b80 10 client.0 _ll_register_callbacks cb 0x558ec4f27ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec508ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39270700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec508f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec508ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:43762/0 (socket says 172.21.15.39:43762) 2024-05-04T03:29:06.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 -- 172.21.15.39:0/4079067723 learned_addr learned my addr 172.21.15.39:0/4079067723 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39270700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec508f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44924/0 (socket says 172.21.15.39:44924) 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d3a272700 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 0x558ec50f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 -- 172.21.15.39:0/4079067723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 msgr2=0x558ec50f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 0x558ec50f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 -- 172.21.15.39:0/4079067723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 msgr2=0x558ec508f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec508f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 -- 172.21.15.39:0/4079067723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ec508fcf0 con 0x558ec4eeffd0 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d39a71700 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec508ef00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d30012c20 tx=0x7f9d30013f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a6248efabd70e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:06.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.488+0000 7f9d1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/4079067723 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30044060 con 0x558ec4eeffd0 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/4079067723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d240b2170 con 0x558ec4eeffd0 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/4079067723 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f9d30003680 con 0x558ec4eeffd0 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/4079067723 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d30030070 con 0x558ec4eeffd0 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 10 client.26887 fetch_fsmap learned FSMap version 32 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/4079067723 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ec50f6a00 con 0x558ec4eeffd0 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/4079067723 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d30016040 con 0x558ec4eeffd0 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/4079067723 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9d3003e070 con 0x558ec4eeffd0 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 10 client.26887 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 10 client.26887 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 20 client.26887 populate_metadata read hostname 'smithi039' 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558ec507c0e0 0x558ec50f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:06.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 4 client.26887 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.9, "await_for": 42.0}] 2024-05-04T03:29:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.489+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/4079067723 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 80.9, "await_for": 42.0}) -- 0x7f9d240b2170 con 0x558ec507c0e0 2024-05-04T03:29:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.490+0000 7f9d39270700 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558ec507c0e0 0x558ec50f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.490+0000 7f9d1d7fa700 10 client.26887 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.490+0000 7f9d39270700 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558ec507c0e0 0x558ec50f9190 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40acb23f91234dcb server_cookie=e7d517478e86111b in_seq=0 out_seq=0 2024-05-04T03:29:06.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:06 smithi039 ceph-mon[29715]: pgmap v935: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-05-04T03:29:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d1d7fa700 1 -- 172.21.15.39:0/4079067723 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f9d240b2170 con 0x558ec507c0e0 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d1d7fa700 10 client.26887 handle_command_reply: tid=0 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 client.26887 shutdown 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/4079067723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 msgr2=0x558ec508ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec508ef00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d30012c20 tx=0x7f9d30013f30 comp rx=0 tx=0).stop 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/4079067723 shutdown_connections 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x558ec507c0e0 0x558ec50f9190 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558ec506d380 0x558ec50f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ec4eeffd0 0x558ec508ef00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 --2- 172.21.15.39:0/4079067723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558ec506ca30 0x558ec508f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d1d7fa700 0 client.26887 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/4079067723 >> 172.21.15.39:0/4079067723 conn(0x558ec5061bf0 msgr2=0x558ec5078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/4079067723 shutdown_connections 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 1 -- 172.21.15.39:0/4079067723 wait complete. 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.873+0000 7f9d4a6e9b80 20 client.26887 trim_cache size 0 max 16384 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 50, 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stdout: "f06f27a7": { 2024-05-04T03:29:06.877 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 50, 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 60.0, 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 80.9, 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-05-04T03:29:06.878 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:06.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:06.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:06.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:06.879 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:06.879 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d200b2880 msgr2=0x7f9d200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d200b2880 0x7f9d200b4d90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009cb0 tx=0x7f9d30009260 comp rx=0 tx=0).stop 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34107df0 msgr2=0x7f9d341a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34107df0 0x7f9d341a0ee0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400d980 tx=0x7f9d2400dd40 comp rx=0 tx=0).stop 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/1657193767 shutdown_connections 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f9d200b2880 0x7f9d200b4d90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d34108740 0x7f9d3419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d341034c0 0x7f9d341a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 --2- 172.21.15.39:0/1657193767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d34107df0 0x7f9d341a0ee0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/1657193767 >> 172.21.15.39:0/1657193767 conn(0x7f9d340fedc0 msgr2=0x7f9d341075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/1657193767 shutdown_connections 2024-05-04T03:29:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:06.874+0000 7f9d3bcd5700 1 -- 172.21.15.39:0/1657193767 wait complete. 2024-05-04T03:29:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:06 smithi167 ceph-mon[33614]: pgmap v935: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-05-04T03:29:07.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:06 smithi066 ceph-mon[33264]: pgmap v935: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 537 op/s 2024-05-04T03:29:07.084 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-05-04T03:29:07.084 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-04T03:29:07.084 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:11 . 2024-05-04T03:29:07.084 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:11 .. 2024-05-04T03:29:07.084 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 data 2024-05-04T03:29:07.085 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.085 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-05-04T03:29:07.085 INFO:teuthology.orchestra.run.smithi039.stdout:total 46 2024-05-04T03:29:07.085 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 . 2024-05-04T03:29:07.085 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:11 .. 2024-05-04T03:29:07.085 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 612750901 May 4 03:13 base 2024-05-04T03:29:07.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 4 03:11 current_logfiles 2024-05-04T03:29:07.086 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 4 03:11 global 2024-05-04T03:29:07.086 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1414 May 4 03:11 log 2024-05-04T03:29:07.086 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_commit_ts 2024-05-04T03:29:07.086 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_dynshmem 2024-05-04T03:29:07.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 4 03:11 pg_hba.conf 2024-05-04T03:29:07.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 4 03:11 pg_ident.conf 2024-05-04T03:29:07.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:29 pg_logical 2024-05-04T03:29:07.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 pg_multixact 2024-05-04T03:29:07.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_notify 2024-05-04T03:29:07.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_replslot 2024-05-04T03:29:07.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_serial 2024-05-04T03:29:07.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_snapshots 2024-05-04T03:29:07.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 4 03:29 pg_stat 2024-05-04T03:29:07.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:29 pg_stat_tmp 2024-05-04T03:29:07.088 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 May 4 03:29 pg_subtrans 2024-05-04T03:29:07.088 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_tblspc 2024-05-04T03:29:07.088 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 pg_twophase 2024-05-04T03:29:07.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 4 03:11 PG_VERSION 2024-05-04T03:29:07.088 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 989855744 May 4 03:29 pg_wal 2024-05-04T03:29:07.088 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:11 pg_xact 2024-05-04T03:29:07.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 4 03:11 postgresql.auto.conf 2024-05-04T03:29:07.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 4 03:11 postgresql.conf 2024-05-04T03:29:07.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 4 03:11 postmaster.opts 2024-05-04T03:29:07.089 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.089 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base: 2024-05-04T03:29:07.089 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-04T03:29:07.089 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 612750901 May 4 03:13 . 2024-05-04T03:29:07.089 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.089 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 1 2024-05-04T03:29:07.090 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 13493 2024-05-04T03:29:07.090 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 596824623 May 4 03:17 13494 2024-05-04T03:29:07.090 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:13 pgsql_tmp 2024-05-04T03:29:07.090 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.090 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-04T03:29:07.090 INFO:teuthology.orchestra.run.smithi039.stdout:total 7778 2024-05-04T03:29:07.090 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 . 2024-05-04T03:29:07.091 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 612750901 May 4 03:13 .. 2024-05-04T03:29:07.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 112 2024-05-04T03:29:07.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 113 2024-05-04T03:29:07.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 1247 2024-05-04T03:29:07.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1247_fsm 2024-05-04T03:29:07.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1247_vm 2024-05-04T03:29:07.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 4 03:11 1249 2024-05-04T03:29:07.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1249_fsm 2024-05-04T03:29:07.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1249_vm 2024-05-04T03:29:07.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 4 03:11 1255 2024-05-04T03:29:07.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1255_fsm 2024-05-04T03:29:07.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1255_vm 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 4 03:11 1259 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1259_fsm 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1259_vm 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 4 03:11 13344 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13344_fsm 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13344_vm 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13346 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13348 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13349_fsm 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349_vm 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13351 2024-05-04T03:29:07.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13353 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13354_fsm 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354_vm 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13356 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13358 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13359_fsm 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359_vm 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13361 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13363 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1417 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1418 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 174 2024-05-04T03:29:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 175 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2187 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2224 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2328 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2336 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2337 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2579 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2600 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2600_fsm 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2600_vm 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2601_fsm 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601_vm 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2602 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2602_fsm 2024-05-04T03:29:07.095 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2602_vm 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2603 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2603_fsm 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2603_vm 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2604 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2605 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2605_fsm 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2605_vm 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2606_fsm 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606_vm 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2607 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2607_fsm 2024-05-04T03:29:07.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2607_vm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 4 03:11 2608 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2608_fsm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2608_vm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 4 03:11 2609 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2609_fsm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2609_vm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2610 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2610_fsm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2610_vm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2611 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2612_fsm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612_vm 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2613 2024-05-04T03:29:07.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2615_fsm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615_vm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2616 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2616_fsm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2616_vm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 4 03:11 2617 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2617_fsm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2617_vm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 4 03:11 2618 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2618_fsm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2618_vm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 4 03:11 2619 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2619_fsm 2024-05-04T03:29:07.098 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2619_vm 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2620 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2650 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2651 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2652 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2653 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2654 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2655 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2656 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2657 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 4 03:11 2658 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 2659 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2660 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2661 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2662 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2663 2024-05-04T03:29:07.099 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2664 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2665 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2666 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2667 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2668 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2669 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2670 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 4 03:11 2673 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 4 03:11 2674 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:11 2675 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2678 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2679 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2680 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2681 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2682 2024-05-04T03:29:07.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2683 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2684 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2685 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2686 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2687 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2688 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2689 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 4 03:11 2690 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 4 03:11 2691 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2692 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2693 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2696 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2699 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2701 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2702 2024-05-04T03:29:07.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2703 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2704 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2753 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2753_fsm 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2753_vm 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2754 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2755 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2756 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2757 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2830 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2831 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2832 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2833 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2834 2024-05-04T03:29:07.102 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2835 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2836 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2837 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 4 03:11 2838 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2838_fsm 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2838_vm 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2839 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840_fsm 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2840_vm 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2841 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2995 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2996 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3079_fsm 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079_vm 2024-05-04T03:29:07.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3080 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3081 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3085 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3118 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3119 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 4 03:11 3164 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3256 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3257 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3258 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3350 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3351 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3379 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3380 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3381 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3394 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3394_fsm 2024-05-04T03:29:07.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3394_vm 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3395 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3429 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3430 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3431 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3433 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3439 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3440 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3455 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 4 03:11 3456 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3456_fsm 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3456_vm 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3466 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3467 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3468 2024-05-04T03:29:07.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3501 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3502 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3503 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3534 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3541_fsm 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541_vm 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3542 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3574 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3575 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3576 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3596 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3597 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3598 2024-05-04T03:29:07.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3599 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3600_fsm 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600_vm 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3601_fsm 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601_vm 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3602_fsm 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602_vm 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603_fsm 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3603_vm 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3604 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3605 2024-05-04T03:29:07.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3606 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3607 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3608 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 3609 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3712 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3764_fsm 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764_vm 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3766 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3767 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3997 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4143 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4144 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4145 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4146 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4147 2024-05-04T03:29:07.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4148 2024-05-04T03:29:07.109 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4149 2024-05-04T03:29:07.109 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4150 2024-05-04T03:29:07.109 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4151 2024-05-04T03:29:07.358 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4152 2024-05-04T03:29:07.358 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4153 2024-05-04T03:29:07.358 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4154 2024-05-04T03:29:07.358 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4155 2024-05-04T03:29:07.358 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4156 2024-05-04T03:29:07.358 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4157 2024-05-04T03:29:07.358 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4158 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4159 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4160 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4161 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4162 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4163 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4164 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4165 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4166 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4167 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4168 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4169 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4170 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4171 2024-05-04T03:29:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4172 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4173 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4174 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 5002 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 548 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 549 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6102 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6104 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6106 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6110 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6111 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6112 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6113 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6117 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 826 2024-05-04T03:29:07.360 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 827 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 828 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 4 03:11 pg_filenode.map 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 4 03:11 PG_VERSION 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:total 7778 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 4 03:11 . 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 612750901 May 4 03:13 .. 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 112 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 113 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 1247 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1247_fsm 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1247_vm 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 4 03:11 1249 2024-05-04T03:29:07.361 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1249_fsm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1249_vm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 4 03:11 1255 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1255_fsm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1255_vm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 4 03:11 1259 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1259_fsm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1259_vm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 4 03:11 13344 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13344_fsm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13344_vm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13346 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13348 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13349_fsm 2024-05-04T03:29:07.362 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349_vm 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13351 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13353 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13354_fsm 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354_vm 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13356 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13358 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13359_fsm 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359_vm 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13361 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13363 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1417 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1418 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 174 2024-05-04T03:29:07.363 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 175 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2187 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2224 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2328 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2336 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2337 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2579 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2600 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2600_fsm 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2600_vm 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2601_fsm 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601_vm 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2602 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2602_fsm 2024-05-04T03:29:07.364 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2602_vm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2603 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2603_fsm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2603_vm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2604 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2605 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2605_fsm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2605_vm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2606_fsm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2606_vm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2607 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2607_fsm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2607_vm 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 4 03:11 2608 2024-05-04T03:29:07.365 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2608_fsm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2608_vm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 4 03:11 2609 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2609_fsm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2609_vm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2610 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2610_fsm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2610_vm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2611 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2612_fsm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612_vm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2613 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2615_fsm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615_vm 2024-05-04T03:29:07.366 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2616 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2616_fsm 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2616_vm 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 4 03:11 2617 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2617_fsm 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2617_vm 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 4 03:11 2618 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2618_fsm 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2618_vm 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 4 03:11 2619 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2619_fsm 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2619_vm 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2620 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2650 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2651 2024-05-04T03:29:07.367 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2652 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2653 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2654 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2655 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2656 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2657 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 4 03:11 2658 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:11 2659 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2660 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2661 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2662 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2663 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2664 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2665 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2666 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2667 2024-05-04T03:29:07.368 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2668 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2669 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2670 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 4 03:11 2673 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 4 03:11 2674 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:11 2675 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2678 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2679 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2680 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2681 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2682 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2683 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2684 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2685 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2686 2024-05-04T03:29:07.369 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2687 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2688 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2689 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 4 03:11 2690 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 4 03:11 2691 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2692 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2693 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2696 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2699 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2701 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2702 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2703 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2704 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2753 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2753_fsm 2024-05-04T03:29:07.370 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2753_vm 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2754 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2755 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2756 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2757 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2830 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2831 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2832 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2833 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2834 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2835 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2836 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2837 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 4 03:11 2838 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2838_fsm 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2838_vm 2024-05-04T03:29:07.371 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2839 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840_fsm 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2840_vm 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2841 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2995 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2996 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3079_fsm 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079_vm 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3080 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3081 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3085 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3118 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3119 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 4 03:11 3164 2024-05-04T03:29:07.372 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3256 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3257 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3258 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3350 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3351 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3379 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3380 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3381 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3394 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3394_fsm 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3394_vm 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3395 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3429 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3430 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3431 2024-05-04T03:29:07.373 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3433 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3439 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3440 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3455 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 4 03:11 3456 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3456_fsm 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3456_vm 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3466 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3467 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3468 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3501 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3502 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3503 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3534 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3541_fsm 2024-05-04T03:29:07.374 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541_vm 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3542 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3574 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3575 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3576 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3596 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3597 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3598 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3599 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3600_fsm 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600_vm 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3601_fsm 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601_vm 2024-05-04T03:29:07.375 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3602_fsm 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602_vm 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603_fsm 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3603_vm 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3604 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3605 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3606 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3607 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3608 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 3609 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3712 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3764_fsm 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764_vm 2024-05-04T03:29:07.376 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3766 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3767 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3997 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4143 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4144 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4145 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4146 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4147 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4148 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4149 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4150 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4151 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4152 2024-05-04T03:29:07.377 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4153 2024-05-04T03:29:07.570 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f06f27a7', quiesce took 7.6 seconds. Will release after: 20.9 2024-05-04T03:29:07.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4154 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4155 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4156 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4157 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4158 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4159 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4160 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4161 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4162 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4163 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4164 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4165 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4166 2024-05-04T03:29:07.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4167 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4168 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4169 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4170 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4171 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4172 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4173 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4174 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 5002 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 548 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 549 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6102 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6104 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6106 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6110 2024-05-04T03:29:07.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6111 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6112 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6113 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6117 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 826 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 827 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 828 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 4 03:11 pg_filenode.map 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 4 03:11 PG_VERSION 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:total 582839 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 596824623 May 4 03:17 . 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 612750901 May 4 03:13 .. 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 112 2024-05-04T03:29:07.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 113 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:14 1247 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1247_fsm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:13 1247_vm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 4 03:14 1249 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:13 1249_fsm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:13 1249_vm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 4 03:11 1255 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1255_fsm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1255_vm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 4 03:28 1259 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1259_fsm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:13 1259_vm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 4 03:11 13344 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13344_fsm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13344_vm 2024-05-04T03:29:07.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13346 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13348 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13349_fsm 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13349_vm 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13351 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13353 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13354_fsm 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13354_vm 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13356 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13358 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 13359_fsm 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13359_vm 2024-05-04T03:29:07.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 13361 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 13363 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1417 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 1418 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 458801152 May 4 03:29 16396 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 4 03:26 16396_fsm 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:25 16396_vm 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 6651904 May 4 03:29 16397 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:28 16397_fsm 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:29 16397_vm 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 11173888 May 4 03:29 16399 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:28 16399_fsm 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:28 16399_vm 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:29 16400 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 4 03:27 16402 2024-05-04T03:29:07.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 4 03:29 16404 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 39731200 May 4 03:29 16406 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:29 16406_fsm 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:28 16406_vm 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 174 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 175 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2187 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2224 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2328 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2336 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2337 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:14 2579 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2600 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2600_fsm 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2600_vm 2024-05-04T03:29:07.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2601_fsm 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2601_vm 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2602 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2602_fsm 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2602_vm 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2603 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2603_fsm 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2603_vm 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2604 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2605 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2605_fsm 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2605_vm 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:14 2606 2024-05-04T03:29:07.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2606_fsm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:13 2606_vm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2607 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2607_fsm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2607_vm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 4 03:14 2608 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2608_fsm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:13 2608_vm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 4 03:11 2609 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2609_fsm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2609_vm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:14 2610 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2610_fsm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:13 2610_vm 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2611 2024-05-04T03:29:07.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2612_fsm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2612_vm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2613 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2615_fsm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2615_vm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2616 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2616_fsm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2616_vm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 4 03:11 2617 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2617_fsm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2617_vm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 4 03:11 2618 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2618_fsm 2024-05-04T03:29:07.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2618_vm 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 May 4 03:28 2619 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:25 2619_fsm 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:13 2619_vm 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2620 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2650 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2651 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2652 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2653 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2654 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2655 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2656 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2657 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 4 03:14 2658 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 4 03:14 2659 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2660 2024-05-04T03:29:07.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2661 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:14 2662 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:14 2663 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:14 2664 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:14 2665 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:14 2666 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:14 2667 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2668 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2669 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2670 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 4 03:14 2673 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 4 03:14 2674 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:11 2675 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:14 2678 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:14 2679 2024-05-04T03:29:07.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2680 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2681 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2682 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2683 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2684 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2685 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2686 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2687 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 2688 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 4 03:11 2689 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 4 03:11 2690 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 4 03:11 2691 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2692 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2693 2024-05-04T03:29:07.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:28 2696 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2699 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2701 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2702 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:14 2703 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:14 2704 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2753 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2753_fsm 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2753_vm 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2754 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2755 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2756 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 2757 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2830 2024-05-04T03:29:07.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2831 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2832 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2833 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2834 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2835 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2836 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2837 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 4 03:11 2838 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2838_fsm 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2838_vm 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2839 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2840_fsm 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2840_vm 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2841 2024-05-04T03:29:07.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2995 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2996 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3079_fsm 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3079_vm 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3080 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3081 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:11 3085 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3118 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3119 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 4 03:11 3164 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3256 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3257 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3258 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3350 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3351 2024-05-04T03:29:07.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3379 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3380 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3381 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3394 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3394_fsm 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3394_vm 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3395 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3429 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3430 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3431 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3433 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3439 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3440 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 4 03:14 3455 2024-05-04T03:29:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 4 03:11 3456 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3456_fsm 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3456_vm 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3466 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3467 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3468 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3501 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3502 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3503 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3534 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3541_fsm 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3541_vm 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3542 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3574 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3575 2024-05-04T03:29:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3576 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3596 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3597 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3598 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3599 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3600_fsm 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3600_vm 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3601_fsm 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3601_vm 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3602_fsm 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3602_vm 2024-05-04T03:29:07.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3603_fsm 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3603_vm 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3604 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3605 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3606 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3607 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3608 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 4 03:11 3609 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3712 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 3764_fsm 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3764_vm 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3766 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 3767 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3997 2024-05-04T03:29:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4143 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4144 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4145 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4146 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4147 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4148 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4149 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4150 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4151 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4152 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4153 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4154 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4155 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4156 2024-05-04T03:29:07.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4157 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4158 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4159 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4160 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4161 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4162 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4163 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4164 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4165 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4166 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4167 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4168 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4169 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4170 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4171 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4172 2024-05-04T03:29:07.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4173 2024-05-04T03:29:07.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4174 2024-05-04T03:29:07.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 5002 2024-05-04T03:29:07.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 548 2024-05-04T03:29:07.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 549 2024-05-04T03:29:07.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6102 2024-05-04T03:29:07.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6104 2024-05-04T03:29:07.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6106 2024-05-04T03:29:07.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6110 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6111 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6112 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6113 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6117 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 826 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 827 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 828 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 4 03:11 pg_filenode.map 2024-05-04T03:29:07.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 4 03:11 pg_internal.init 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 4 03:11 PG_VERSION 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:13 . 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 612750901 May 4 03:13 .. 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/global: 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:total 561 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 4 03:11 . 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1213 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1213_fsm 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1213_vm 2024-05-04T03:29:07.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1214 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1214_fsm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1214_vm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 1232 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 1233 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1260 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1260_fsm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1260_vm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1261 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1261_fsm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1261_vm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1262 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 1262_fsm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 1262_vm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2396 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 4 03:11 2396_fsm 2024-05-04T03:29:07.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2396_vm 2024-05-04T03:29:07.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2397 2024-05-04T03:29:07.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2671 2024-05-04T03:29:07.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2672 2024-05-04T03:29:07.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2676 2024-05-04T03:29:07.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2677 2024-05-04T03:29:07.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2694 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2695 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2697 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 2698 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2846 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2847 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2964 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2965 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 2966 2024-05-04T03:29:07.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 2967 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 3592 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 3593 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4060 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4061 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4175 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4176 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4177 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4178 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4181 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4182 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4183 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4184 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 4185 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 4186 2024-05-04T03:29:07.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6000 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6001 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6002 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 6100 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6114 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 6115 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:29 pg_control 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 4 03:11 pg_filenode.map 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 4 03:11 pg_internal.init 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/log: 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1485 May 4 03:11 . 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1485 May 4 03:29 postgresql-Sat.log 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-04T03:29:07.735 INFO:teuthology.orchestra.run.smithi039.stdout:total 3 2024-05-04T03:29:07.788 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:29 . 2024-05-04T03:29:07.788 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.788 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 mappings 2024-05-04T03:29:07.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:29 replorigin_checkpoint 2024-05-04T03:29:07.788 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 snapshots 2024-05-04T03:29:07.788 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.788 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-04T03:29:07.788 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:29 .. 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 4 03:29 .. 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 . 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 members 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 offsets 2024-05-04T03:29:07.789 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 . 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 .. 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 0000 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:total 9 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:11 . 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 4 03:11 .. 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 4 03:16 0000 2024-05-04T03:29:07.790 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.791 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:total 16 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 4 03:29 . 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 4 03:29 db_0.stat 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 4 03:29 db_13494.stat 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 4 03:29 global.stat 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.792 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-04T03:29:07.793 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.793 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:29 . 2024-05-04T03:29:07.793 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.793 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.793 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-04T03:29:07.793 INFO:teuthology.orchestra.run.smithi039.stdout:total 161 2024-05-04T03:29:07.793 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 May 4 03:29 . 2024-05-04T03:29:07.793 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 May 4 03:28 000B 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout:total 966658 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 989855744 May 4 03:29 . 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:29 000000010000000000000080 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:22 000000010000000000000081 2024-05-04T03:29:07.850 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:23 000000010000000000000082 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:24 000000010000000000000083 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:25 000000010000000000000084 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:22 000000010000000000000085 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:22 000000010000000000000086 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:24 000000010000000000000087 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:21 000000010000000000000088 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:21 000000010000000000000089 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:24 00000001000000000000008A 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:23 00000001000000000000008B 2024-05-04T03:29:07.851 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:24 00000001000000000000008C 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:22 00000001000000000000008D 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:24 00000001000000000000008E 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:21 00000001000000000000008F 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:23 000000010000000000000090 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:25 000000010000000000000091 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:22 000000010000000000000092 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:23 000000010000000000000093 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:25 000000010000000000000094 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:24 000000010000000000000095 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:21 000000010000000000000096 2024-05-04T03:29:07.852 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:22 000000010000000000000097 2024-05-04T03:29:07.853 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:21 000000010000000000000098 2024-05-04T03:29:07.853 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:22 000000010000000000000099 2024-05-04T03:29:07.853 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:22 00000001000000000000009A 2024-05-04T03:29:07.853 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:24 00000001000000000000009B 2024-05-04T03:29:07.858 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:23 00000001000000000000009C 2024-05-04T03:29:07.858 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:21 00000001000000000000009D 2024-05-04T03:29:07.858 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:21 00000001000000000000009E 2024-05-04T03:29:07.858 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:21 00000001000000000000009F 2024-05-04T03:29:07.858 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:24 0000000100000000000000A0 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:26 0000000100000000000000A1 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000A2 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000A3 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:28 0000000100000000000000A4 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:28 0000000100000000000000A5 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000A6 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000A7 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000A8 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:26 0000000100000000000000A9 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:28 0000000100000000000000AA 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:26 0000000100000000000000AB 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000AC 2024-05-04T03:29:07.859 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:26 0000000100000000000000AD 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:26 0000000100000000000000AE 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000AF 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000B0 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000B1 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:28 0000000100000000000000B2 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:26 0000000100000000000000B3 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000B4 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000B5 2024-05-04T03:29:07.860 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000B6 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000B7 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000B8 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000B9 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 4 03:27 0000000100000000000000BA 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 archive_status 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 4 03:11 . 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 989855744 May 4 03:29 .. 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:total 193 2024-05-04T03:29:07.861 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:11 . 2024-05-04T03:29:07.862 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1603621962 May 4 03:29 .. 2024-05-04T03:29:07.862 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 4 03:28 0000 2024-05-04T03:29:07.863 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-04T03:29:07.877 INFO:tasks.quiescer:joining Quiescers 2024-05-04T03:29:07.877 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-04T03:29:07.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:29:08.127 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:08.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:08 smithi167 ceph-mon[33614]: pgmap v936: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-05-04T03:29:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:08 smithi039 ceph-mon[29715]: pgmap v936: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-05-04T03:29:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:08 smithi066 ceph-mon[33264]: pgmap v936: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-05-04T03:29:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 -- 172.21.15.39:0/1377682741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c104eb0 msgr2=0x7f210c1052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 --2- 172.21.15.39:0/1377682741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c104eb0 0x7f210c1052c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2100009900 tx=0x7f210002a410 comp rx=0 tx=0).stop 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 -- 172.21.15.39:0/1377682741 shutdown_connections 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 --2- 172.21.15.39:0/1377682741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210c109bb0 0x7f210c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 --2- 172.21.15.39:0/1377682741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c104eb0 0x7f210c1052c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 --2- 172.21.15.39:0/1377682741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f210c103c70 0x7f210c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 -- 172.21.15.39:0/1377682741 >> 172.21.15.39:0/1377682741 conn(0x7f210c0fee40 msgr2=0x7f210c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 -- 172.21.15.39:0/1377682741 shutdown_connections 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.547+0000 7f21132af640 1 -- 172.21.15.39:0/1377682741 wait complete. 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f21132af640 1 Processor -- start 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f21132af640 1 -- start start 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f21132af640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c103c70 0x7f210c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f21132af640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210c104eb0 0x7f210c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f21132af640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f210c109bb0 0x7f210c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f21132af640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f210c10f650 con 0x7f210c103c70 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f21132af640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f210c10f4d0 con 0x7f210c109bb0 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f21132af640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f210c10f7d0 con 0x7f210c104eb0 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f211104b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c103c70 0x7f210c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f211104b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c103c70 0x7f210c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44944/0 (socket says 172.21.15.39:44944) 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f211104b640 1 -- 172.21.15.39:0/2424563913 learned_addr learned my addr 172.21.15.39:0/2424563913 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f211184c640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f210c109bb0 0x7f210c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.548+0000 7f211084a640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210c104eb0 0x7f210c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f211104b640 1 -- 172.21.15.39:0/2424563913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210c104eb0 msgr2=0x7f210c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f211104b640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210c104eb0 0x7f210c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f211104b640 1 -- 172.21.15.39:0/2424563913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f210c109bb0 msgr2=0x7f210c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f211104b640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f210c109bb0 0x7f210c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f211104b640 1 -- 172.21.15.39:0/2424563913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c1a1da0 con 0x7f210c103c70 2024-05-04T03:29:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f211104b640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c103c70 0x7f210c19a260 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2104005fd0 tx=0x7f210400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f210a7fc640 1 -- 172.21.15.39:0/2424563913 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104017070 con 0x7f210c103c70 2024-05-04T03:29:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f210a7fc640 1 -- 172.21.15.39:0/2424563913 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2104009c30 con 0x7f210c103c70 2024-05-04T03:29:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f21132af640 1 -- 172.21.15.39:0/2424563913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210c1a2090 con 0x7f210c103c70 2024-05-04T03:29:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f210a7fc640 1 -- 172.21.15.39:0/2424563913 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104003c90 con 0x7f210c103c70 2024-05-04T03:29:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.549+0000 7f21132af640 1 -- 172.21.15.39:0/2424563913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210c1a25e0 con 0x7f210c103c70 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.552+0000 7f211a164740 1 Processor -- start 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.552+0000 7f211a164740 1 -- start start 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211a164740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29745b1b0 0x55b2977fca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211a164740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211a164740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297803ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211a164740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b2977615d0 con 0x55b2974322f0 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211a164740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b29771a890 con 0x55b29743a4d0 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211a164740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b29775d590 con 0x55b29745b1b0 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f210a7fc640 1 -- 172.21.15.39:0/2424563913 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f2104003e30 con 0x7f210c103c70 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211184c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29745b1b0 0x55b2977fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211184c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29745b1b0 0x55b2977fca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.39:43786/0 (socket says 172.21.15.39:43786) 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211184c640 1 -- 172.21.15.39:0/3801548281 learned_addr learned my addr 172.21.15.39:0/3801548281 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211104b640 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211084a640 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297803ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211184c640 1 -- 172.21.15.39:0/3801548281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 msgr2=0x55b297803ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211184c640 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297803ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211184c640 1 -- 172.21.15.39:0/3801548281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 msgr2=0x55b2977fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211184c640 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977fcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211184c640 1 -- 172.21.15.39:0/3801548281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b297473210 con 0x55b29745b1b0 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211104b640 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977fcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.553+0000 7f211084a640 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297803ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.554+0000 7f211184c640 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29745b1b0 0x55b2977fca60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009f70 tx=0x7f20fc010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e6742a36126fad server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.554+0000 7f210a7fc640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20f40b27e0 0x7f20f40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.554+0000 7f20f37fe640 1 -- 172.21.15.39:0/3801548281 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc016070 con 0x55b29745b1b0 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.554+0000 7f211104b640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20f40b27e0 0x7f20f40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.554+0000 7f20f37fe640 1 -- 172.21.15.39:0/3801548281 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20fc004d20 con 0x55b29745b1b0 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.554+0000 7f20f37fe640 1 -- 172.21.15.39:0/3801548281 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc012740 con 0x55b29745b1b0 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.554+0000 7f210a7fc640 1 -- 172.21.15.39:0/2424563913 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f21040e1ef0 con 0x7f210c103c70 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 -- 172.21.15.39:0/3801548281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29745b1b0 msgr2=0x55b2977fca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211104b640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20f40b27e0 0x7f20f40b4cf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55b2977fde40 tx=0x7f210400bf60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29745b1b0 0x55b2977fca60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009f70 tx=0x7f20fc010800 comp rx=0 tx=0).stop 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 -- 172.21.15.39:0/3801548281 shutdown_connections 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297803ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977fcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 --2- 172.21.15.39:0/3801548281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29745b1b0 0x55b2977fca60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 -- 172.21.15.39:0/3801548281 >> 172.21.15.39:0/3801548281 conn(0x55b29744f300 msgr2=0x55b2977f83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 -- 172.21.15.39:0/3801548281 shutdown_connections 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 -- 172.21.15.39:0/3801548281 wait complete. 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 -- start start 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.555+0000 7f211a164740 1 Processor -- start 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211a164740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2974322f0 0x55b297806700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211a164740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297806c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211a164740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b29745b1b0 0x55b29780dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211a164740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b297804930 con 0x55b29745b1b0 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211a164740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b29771e370 con 0x55b29743a4d0 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211a164740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b297804ab0 con 0x55b2974322f0 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211a164740 10 client.0 _ll_register_callbacks cb 0x55b297472a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211184c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b29745b1b0 0x55b29780dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211184c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b29745b1b0 0x55b29780dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44964/0 (socket says 172.21.15.39:44964) 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211184c640 1 -- 172.21.15.39:0/844650789 learned_addr learned my addr 172.21.15.39:0/844650789 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211184c640 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2974322f0 0x55b297806700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.556+0000 7f211084a640 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297806c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211184c640 1 -- 172.21.15.39:0/844650789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2974322f0 msgr2=0x55b297806700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211184c640 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2974322f0 0x55b297806700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211184c640 1 -- 172.21.15.39:0/844650789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 msgr2=0x55b297806c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211184c640 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297806c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211184c640 1 -- 172.21.15.39:0/844650789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29753f620 con 0x55b29745b1b0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211184c640 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b29745b1b0 0x55b29780dba0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f20fc005970 tx=0x7f20fc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0ca8a60bf985a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f20f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f20f0ff9640 1 -- 172.21.15.39:0/844650789 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc016030 con 0x55b29745b1b0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f20f0ff9640 1 -- 172.21.15.39:0/844650789 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f20fc003970 con 0x55b29745b1b0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211a164740 1 -- 172.21.15.39:0/844650789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20fc004d20 con 0x55b29745b1b0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f20f0ff9640 1 -- 172.21.15.39:0/844650789 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f20fc031050 con 0x55b29745b1b0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211a164740 10 client.17316 fetch_fsmap learned FSMap version 32 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f211a164740 1 -- 172.21.15.39:0/844650789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b297810ad0 con 0x55b29745b1b0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.557+0000 7f20f0ff9640 1 -- 172.21.15.39:0/844650789 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20fc004d20 con 0x55b29745b1b0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f20f0ff9640 1 -- 172.21.15.39:0/844650789 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f20fc00f040 con 0x55b29745b1b0 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f211a164740 10 client.17316 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f211a164740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f211a164740 20 client.17316 populate_metadata read hostname 'smithi039' 2024-05-04T03:29:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f211a164740 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b297419d80 0x55b297813f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f211a164740 4 client.17316 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:29:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f211a164740 1 -- 172.21.15.39:0/844650789 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b29776f1e0 con 0x55b297419d80 2024-05-04T03:29:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f211084a640 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b297419d80 0x55b297813f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f20f0ff9640 10 client.17316 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.558+0000 7f211084a640 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b297419d80 0x55b297813f40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ce6cf2f76f36bc server_cookie=e67ff96f9da5e74d in_seq=0 out_seq=0 2024-05-04T03:29:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f20f0ff9640 1 -- 172.21.15.39:0/844650789 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b29776f1e0 con 0x55b297419d80 2024-05-04T03:29:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f20f0ff9640 10 client.17316 handle_command_reply: tid=0 2024-05-04T03:29:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f211a164740 1 client.17316 shutdown 2024-05-04T03:29:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f211a164740 1 -- 172.21.15.39:0/844650789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b29745b1b0 msgr2=0x55b29780dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f211a164740 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b29745b1b0 0x55b29780dba0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f20fc005970 tx=0x7f20fc0058a0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f211a164740 1 -- 172.21.15.39:0/844650789 shutdown_connections 2024-05-04T03:29:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f211a164740 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b297419d80 0x55b297813f40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f211a164740 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b29745b1b0 0x55b29780dba0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.559+0000 7f211a164740 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b29743a4d0 0x55b297806c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.560+0000 7f211a164740 1 --2- 172.21.15.39:0/844650789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2974322f0 0x55b297806700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.560+0000 7f20f0ff9640 0 client.17316 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.560+0000 7f211a164740 1 -- 172.21.15.39:0/844650789 >> 172.21.15.39:0/844650789 conn(0x55b29744f300 msgr2=0x55b2977fb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.560+0000 7f211a164740 1 -- 172.21.15.39:0/844650789 shutdown_connections 2024-05-04T03:29:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.560+0000 7f211a164740 1 -- 172.21.15.39:0/844650789 wait complete. 2024-05-04T03:29:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.560+0000 7f211a164740 20 client.17316 trim_cache size 0 max 16384 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.575+0000 7f211a164740 1 Processor -- start 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.575+0000 7f211a164740 1 -- start start 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211a164740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b297419d80 0x55b2977f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211a164740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211a164740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 0x55b29780dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211a164740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1ab270 con 0x55b2974322f0 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211a164740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1acdb0 con 0x55b297419d80 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211a164740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1ab940 con 0x55b29743a4d0 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44966/0 (socket says 172.21.15.39:44966) 2024-05-04T03:29:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 -- 172.21.15.39:0/4265245097 learned_addr learned my addr 172.21.15.39:0/4265245097 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211184c640 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b297419d80 0x55b2977f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211184c640 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 0x55b29780dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 -- 172.21.15.39:0/4265245097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 msgr2=0x55b29780dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 0x55b29780dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 -- 172.21.15.39:0/4265245097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b297419d80 msgr2=0x55b2977f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b297419d80 0x55b2977f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 -- 172.21.15.39:0/4265245097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2977f8d00 con 0x55b2974322f0 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211184c640 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 0x55b29780dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f211084a640 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977f82a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f210002a930 tx=0x7f2100031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc10482bd1a6c287 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.576+0000 7f20f2ffd640 1 -- 172.21.15.39:0/4265245097 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210004a070 con 0x55b2974322f0 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f20f2ffd640 1 -- 172.21.15.39:0/4265245097 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2100031c90 con 0x55b2974322f0 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f20f2ffd640 1 -- 172.21.15.39:0/4265245097 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100035040 con 0x55b2974322f0 2024-05-04T03:29:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 -- 172.21.15.39:0/4265245097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 msgr2=0x55b2977f82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977f82a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f210002a930 tx=0x7f2100031400 comp rx=0 tx=0).stop 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 -- 172.21.15.39:0/4265245097 shutdown_connections 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 0x55b29780dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b2974322f0 0x55b2977f82a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 --2- 172.21.15.39:0/4265245097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b297419d80 0x55b2977f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 -- 172.21.15.39:0/4265245097 >> 172.21.15.39:0/4265245097 conn(0x55b29744f300 msgr2=0x55b297800b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 -- 172.21.15.39:0/4265245097 shutdown_connections 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.577+0000 7f211a164740 1 -- 172.21.15.39:0/4265245097 wait complete. 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 1 -- start start 2024-05-04T03:29:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 1 Processor -- start 2024-05-04T03:29:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b297419d80 0x55b29780b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b2974322f0 0x55b29780be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 0x55b29780cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1ac6e0 con 0x55b297419d80 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1ac010 con 0x55b2974322f0 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1a9250 con 0x55b29743a4d0 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211a164740 10 client.0 _ll_register_callbacks cb 0x7f20fc009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211184c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b297419d80 0x55b29780b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211184c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b297419d80 0x55b29780b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44978/0 (socket says 172.21.15.39:44978) 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.578+0000 7f211184c640 1 -- 172.21.15.39:0/3061003700 learned_addr learned my addr 172.21.15.39:0/3061003700 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211084a640 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b2974322f0 0x55b29780be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211184c640 1 -- 172.21.15.39:0/3061003700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 msgr2=0x55b29780cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211184c640 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 0x55b29780cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211184c640 1 -- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b2974322f0 msgr2=0x55b29780be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211184c640 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b2974322f0 0x55b29780be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211184c640 1 -- 172.21.15.39:0/3061003700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29780d300 con 0x55b297419d80 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211084a640 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b2974322f0 0x55b29780be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211184c640 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b297419d80 0x55b29780b8d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f20fc006010 tx=0x7f20fc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9683078f0ec0c244 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f20f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f20f0ff9640 1 -- 172.21.15.39:0/3061003700 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc016040 con 0x55b297419d80 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f20f0ff9640 1 -- 172.21.15.39:0/3061003700 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f20fc01aa60 con 0x55b297419d80 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211a164740 1 -- 172.21.15.39:0/3061003700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2100031c90 con 0x55b297419d80 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f20f0ff9640 1 -- 172.21.15.39:0/3061003700 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f20fc023040 con 0x55b297419d80 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211a164740 10 client.17322 fetch_fsmap learned FSMap version 32 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f211a164740 1 -- 172.21.15.39:0/3061003700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b29780e5a0 con 0x55b297419d80 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.579+0000 7f20f0ff9640 1 -- 172.21.15.39:0/3061003700 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2100031c90 con 0x55b297419d80 2024-05-04T03:29:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f20f0ff9640 1 -- 172.21.15.39:0/3061003700 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f20fc012b00 con 0x55b297419d80 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f211a164740 10 client.17322 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f211a164740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f211a164740 20 client.17322 populate_metadata read hostname 'smithi039' 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f211a164740 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b29745b1b0 0x55b29787e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f211a164740 4 client.17322 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f211a164740 1 -- 172.21.15.39:0/3061003700 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b297769050 con 0x55b29745b1b0 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f211084a640 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b29745b1b0 0x55b29787e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f20f0ff9640 10 client.17322 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.580+0000 7f211084a640 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b29745b1b0 0x55b29787e9d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f862682027db447 server_cookie=fee2cd77bcb7a446 in_seq=0 out_seq=0 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f20f0ff9640 1 -- 172.21.15.39:0/3061003700 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b297769050 con 0x55b29745b1b0 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f20f0ff9640 10 client.17322 handle_command_reply: tid=0 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 client.17322 shutdown 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 -- 172.21.15.39:0/3061003700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b297419d80 msgr2=0x55b29780b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b297419d80 0x55b29780b8d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f20fc006010 tx=0x7f20fc000f40 comp rx=0 tx=0).stop 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 -- 172.21.15.39:0/3061003700 shutdown_connections 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x55b29745b1b0 0x55b29787e9d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b29743a4d0 0x55b29780cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b2974322f0 0x55b29780be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f20f0ff9640 0 client.17322 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 --2- 172.21.15.39:0/3061003700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b297419d80 0x55b29780b8d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 -- 172.21.15.39:0/3061003700 >> 172.21.15.39:0/3061003700 conn(0x55b29744f300 msgr2=0x55b2977f7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 -- 172.21.15.39:0/3061003700 shutdown_connections 2024-05-04T03:29:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 1 -- 172.21.15.39:0/3061003700 wait complete. 2024-05-04T03:29:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.590+0000 7f211a164740 20 client.17322 trim_cache size 0 max 16384 2024-05-04T03:29:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:{ 2024-05-04T03:29:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "message": "", 2024-05-04T03:29:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout: "return_code": 0 2024-05-04T03:29:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stdout:} 2024-05-04T03:29:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 -- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20f40b27e0 msgr2=0x7f20f40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20f40b27e0 0x7f20f40b4cf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55b2977fde40 tx=0x7f210400bf60 comp rx=0 tx=0).stop 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 -- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c103c70 msgr2=0x7f210c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c103c70 0x7f210c19a260 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2104005fd0 tx=0x7f210400dde0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 -- 172.21.15.39:0/2424563913 shutdown_connections 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f20f40b27e0 0x7f20f40b4cf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f210c109bb0 0x7f210c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210c104eb0 0x7f210c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 --2- 172.21.15.39:0/2424563913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c103c70 0x7f210c19a260 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 -- 172.21.15.39:0/2424563913 >> 172.21.15.39:0/2424563913 conn(0x7f210c0fee40 msgr2=0x7f210c105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 -- 172.21.15.39:0/2424563913 shutdown_connections 2024-05-04T03:29:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi039.front.sepia.ceph.com.smithi039.stderr:2024-05-04T03:29:09.591+0000 7f21132af640 1 -- 172.21.15.39:0/2424563913 wait complete. 2024-05-04T03:29:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.324+0000 7f95ff0e7700 1 -- 172.21.15.39:0/833214127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f8107e10 msgr2=0x7f95f8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.325+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/833214127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f8107e10 0x7f95f8108220 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f95e40099c0 tx=0x7f95e4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.325+0000 7f95ff0e7700 1 -- 172.21.15.39:0/833214127 shutdown_connections 2024-05-04T03:29:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.325+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/833214127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f8108760 0x7f95f81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.325+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/833214127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95f81034e0 0x7f95f81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.325+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/833214127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f8107e10 0x7f95f8108220 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.325+0000 7f95ff0e7700 1 -- 172.21.15.39:0/833214127 >> 172.21.15.39:0/833214127 conn(0x7f95f80fef30 msgr2=0x7f95f8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.325+0000 7f95ff0e7700 1 -- 172.21.15.39:0/833214127 shutdown_connections 2024-05-04T03:29:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.325+0000 7f95ff0e7700 1 -- 172.21.15.39:0/833214127 wait complete. 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95ff0e7700 1 Processor -- start 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95ff0e7700 1 -- start start 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95ff0e7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81034e0 0x7f95f80ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95ff0e7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f8107e10 0x7f95f80ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95ff0e7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95f8108760 0x7f95f80ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95ff0e7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f95f8113270 con 0x7f95f81034e0 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95ff0e7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81130f0 con 0x7f95f8107e10 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95ff0e7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81133f0 con 0x7f95f8108760 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.326+0000 7f95fce83700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81034e0 0x7f95f80ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fce83700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81034e0 0x7f95f80ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44998/0 (socket says 172.21.15.39:44998) 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fce83700 1 -- 172.21.15.39:0/1478774927 learned_addr learned my addr 172.21.15.39:0/1478774927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95f7fff700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f8107e10 0x7f95f80ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fd684700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95f8108760 0x7f95f80ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fce83700 1 -- 172.21.15.39:0/1478774927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95f8108760 msgr2=0x7f95f80ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fce83700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95f8108760 0x7f95f80ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fce83700 1 -- 172.21.15.39:0/1478774927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f8107e10 msgr2=0x7f95f80ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fce83700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f8107e10 0x7f95f80ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fce83700 1 -- 172.21.15.39:0/1478774927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f81a9130 con 0x7f95f81034e0 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95fce83700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81034e0 0x7f95f80ff410 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00e9a0 tx=0x7f95ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95f5ffb700 1 -- 172.21.15.39:0/1478774927 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00ca10 con 0x7f95f81034e0 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f81a9420 con 0x7f95f81034e0 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95f5ffb700 1 -- 172.21.15.39:0/1478774927 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95ec00cbb0 con 0x7f95f81034e0 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.327+0000 7f95f5ffb700 1 -- 172.21.15.39:0/1478774927 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec018680 con 0x7f95f81034e0 2024-05-04T03:29:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.328+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f81a9980 con 0x7f95f81034e0 2024-05-04T03:29:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.329+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f80512c0 con 0x7f95f81034e0 2024-05-04T03:29:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.331+0000 7f95f5ffb700 1 -- 172.21.15.39:0/1478774927 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f95ec010430 con 0x7f95f81034e0 2024-05-04T03:29:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.331+0000 7f95f5ffb700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f95e80b2880 0x7f95e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.331+0000 7f95f7fff700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f95e80b2880 0x7f95e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.332+0000 7f95f5ffb700 1 -- 172.21.15.39:0/1478774927 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f95ec0e27b0 con 0x7f95f81034e0 2024-05-04T03:29:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.332+0000 7f95f7fff700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f95e80b2880 0x7f95e80b4d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f95e4000c00 tx=0x7f95e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.335+0000 7f95f5ffb700 1 -- 172.21.15.39:0/1478774927 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95ec0aa480 con 0x7f95f81034e0 2024-05-04T03:29:10.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.599+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95f8100940 con 0x7f95f81034e0 2024-05-04T03:29:10.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.600+0000 7f95f5ffb700 1 -- 172.21.15.39:0/1478774927 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7f95ec0a9d80 con 0x7f95f81034e0 2024-05-04T03:29:10.602 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:10.602 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:29:10.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f95e80b2880 msgr2=0x7f95e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:10.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f95e80b2880 0x7f95e80b4d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f95e4000c00 tx=0x7f95e401a040 comp rx=0 tx=0).stop 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81034e0 msgr2=0x7f95f80ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81034e0 0x7f95f80ff410 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00e9a0 tx=0x7f95ec00ecb0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 shutdown_connections 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f95e80b2880 0x7f95e80b4d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81034e0 0x7f95f80ff410 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95f8108760 0x7f95f80ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 --2- 172.21.15.39:0/1478774927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95f8107e10 0x7f95f80ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 >> 172.21.15.39:0/1478774927 conn(0x7f95f80fef30 msgr2=0x7f95f806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 shutdown_connections 2024-05-04T03:29:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:10.602+0000 7f95ff0e7700 1 -- 172.21.15.39:0/1478774927 wait complete. 2024-05-04T03:29:10.606 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:29:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:10 smithi167 ceph-mon[33614]: pgmap v937: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.5 MiB/s wr, 811 op/s 2024-05-04T03:29:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:10 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:29:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:10 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1478774927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:29:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:10 smithi039 ceph-mon[29715]: pgmap v937: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.5 MiB/s wr, 811 op/s 2024-05-04T03:29:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:10 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:29:10.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:10 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1478774927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:29:11.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:10 smithi066 ceph-mon[33264]: pgmap v937: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.5 MiB/s wr, 811 op/s 2024-05-04T03:29:11.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:10 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:29:11.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:10 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1478774927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:29:11.438 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24506 quiesce db --cancel --all' 2024-05-04T03:29:11.439 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24506 quiesce db --cancel --all 2024-05-04T03:29:11.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:11 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:29:11.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:11 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:29:11.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:11 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:29:11.733 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:11.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:11 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:29:11.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:11 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:29:11.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:11 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:29:12.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:11 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:29:12.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:11 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:29:12.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:11 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:29:12.803 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:12 smithi039 ceph-mon[29715]: pgmap v938: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.5 MiB/s wr, 813 op/s 2024-05-04T03:29:12.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:12 smithi167 ceph-mon[33614]: pgmap v938: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.5 MiB/s wr, 813 op/s 2024-05-04T03:29:13.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:12 smithi066 ceph-mon[33264]: pgmap v938: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.5 MiB/s wr, 813 op/s 2024-05-04T03:29:13.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/2275875308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 msgr2=0x7f8c841100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/2275875308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 0x7f8c841100c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400b0d0 tx=0x7f8c7400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/2275875308 shutdown_connections 2024-05-04T03:29:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/2275875308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 0x7f8c841100c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/2275875308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c841034e0 0x7f8c841038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/2275875308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c84107e10 0x7f8c84108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/2275875308 >> 172.21.15.39:0/2275875308 conn(0x7f8c840fef30 msgr2=0x7f8c84101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/2275875308 shutdown_connections 2024-05-04T03:29:13.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.392+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/2275875308 wait complete. 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8bf1b700 1 Processor -- start 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8bf1b700 1 -- start start 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8bf1b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c841034e0 0x7f8c8419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8bf1b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c84107e10 0x7f8c8419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8bf1b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 0x7f8c841a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8bf1b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84113240 con 0x7f8c84108760 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8bf1b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841130c0 con 0x7f8c841034e0 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8bf1b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841133c0 con 0x7f8c84107e10 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8a4b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 0x7f8c841a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c89cb7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c841034e0 0x7f8c8419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8a4b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 0x7f8c841a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45008/0 (socket says 172.21.15.39:45008) 2024-05-04T03:29:13.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c8a4b8700 1 -- 172.21.15.39:0/3958732679 learned_addr learned my addr 172.21.15.39:0/3958732679 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.393+0000 7f8c894b6700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c84107e10 0x7f8c8419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c8a4b8700 1 -- 172.21.15.39:0/3958732679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c84107e10 msgr2=0x7f8c8419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c84107e10 0x7f8c8419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c8a4b8700 1 -- 172.21.15.39:0/3958732679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c841034e0 msgr2=0x7f8c8419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c841034e0 0x7f8c8419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c8a4b8700 1 -- 172.21.15.39:0/3958732679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c841a1e00 con 0x7f8c84108760 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c841034e0 0x7f8c8419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 0x7f8c841a1760 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c74007a00 tx=0x7f8c74007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c82ffd700 1 -- 172.21.15.39:0/3958732679 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401f070 con 0x7f8c84108760 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/3958732679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c841a2090 con 0x7f8c84108760 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c82ffd700 1 -- 172.21.15.39:0/3958732679 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c74004c50 con 0x7f8c84108760 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c82ffd700 1 -- 172.21.15.39:0/3958732679 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401b690 con 0x7f8c84108760 2024-05-04T03:29:13.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.394+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/3958732679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a2570 con 0x7f8c84108760 2024-05-04T03:29:13.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.397+0000 7f8c82ffd700 1 -- 172.21.15.39:0/3958732679 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f8c74022070 con 0x7f8c84108760 2024-05-04T03:29:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.398+0000 7f8c82ffd700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c6c0b2880 0x7f8c6c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.398+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c6c0b2880 0x7f8c6c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.398+0000 7f8c82ffd700 1 -- 172.21.15.39:0/3958732679 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8c7400f020 con 0x7f8c84108760 2024-05-04T03:29:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c6c0b2880 0x7f8c6c0b4d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c011490 tx=0x7f8c7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c9a92fb80 1 Processor -- start 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c9a92fb80 1 -- start start 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca492e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 0x5644ca49d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca49dbb0 0x5644ca49ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5644ca3b59b0 con 0x5644ca49dbb0 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5644ca31ae10 con 0x5644ca492a40 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5644ca3b1970 con 0x5644ca493390 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c8a4b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca49dbb0 0x5644ca49ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c8a4b8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca49dbb0 0x5644ca49ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45024/0 (socket says 172.21.15.39:45024) 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c8a4b8700 1 -- 172.21.15.39:0/2300198987 learned_addr learned my addr 172.21.15.39:0/2300198987 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.399+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca492e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c894b6700 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 0x5644ca49d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c8a4b8700 1 -- 172.21.15.39:0/2300198987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 msgr2=0x5644ca49d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 0x5644ca49d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c8a4b8700 1 -- 172.21.15.39:0/2300198987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 msgr2=0x5644ca492e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca492e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c8a4b8700 1 -- 172.21.15.39:0/2300198987 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ca302940 con 0x5644ca49dbb0 2024-05-04T03:29:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca492e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:29:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca49dbb0 0x5644ca49ffd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c740a69e0 tx=0x7f8c74009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2588d5c3e072ed server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c6bfff700 1 -- 172.21.15.39:0/2300198987 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74004e00 con 0x5644ca49dbb0 2024-05-04T03:29:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c6bfff700 1 -- 172.21.15.39:0/2300198987 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c740b3070 con 0x5644ca49dbb0 2024-05-04T03:29:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.400+0000 7f8c6bfff700 1 -- 172.21.15.39:0/2300198987 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740b3730 con 0x5644ca49dbb0 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2300198987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca49dbb0 msgr2=0x5644ca49ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca49dbb0 0x5644ca49ffd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c740a69e0 tx=0x7f8c74009b80 comp rx=0 tx=0).stop 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2300198987 shutdown_connections 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca49dbb0 0x5644ca49ffd0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca492e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2300198987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 0x5644ca49d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2300198987 >> 172.21.15.39:0/2300198987 conn(0x5644ca315eb0 msgr2=0x5644ca48e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2300198987 shutdown_connections 2024-05-04T03:29:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.401+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2300198987 wait complete. 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 1 -- start start 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 1 Processor -- start 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca492a40 0x5644ca49ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca49b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca49dbb0 0x5644ca49b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5644ca4a0740 con 0x5644ca492a40 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5644ca36ef40 con 0x5644ca493390 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5644ca4a08c0 con 0x5644ca49dbb0 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c9a92fb80 10 client.0 _ll_register_callbacks cb 0x5644ca34d9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c89cb7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca492a40 0x5644ca49ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c89cb7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca492a40 0x5644ca49ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45038/0 (socket says 172.21.15.39:45038) 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c89cb7700 1 -- 172.21.15.39:0/2022544191 learned_addr learned my addr 172.21.15.39:0/2022544191 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca49dbb0 0x5644ca49b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.402+0000 7f8c894b6700 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca49b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c89cb7700 1 -- 172.21.15.39:0/2022544191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca49dbb0 msgr2=0x5644ca49b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca49dbb0 0x5644ca49b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c89cb7700 1 -- 172.21.15.39:0/2022544191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 msgr2=0x5644ca49b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca49b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c89cb7700 1 -- 172.21.15.39:0/2022544191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ca36c7d0 con 0x5644ca492a40 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca492a40 0x5644ca49ac60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c003a60 tx=0x7f8c7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba49af578e43fb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c697fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c697fa700 1 -- 172.21.15.39:0/2022544191 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00ea40 con 0x5644ca492a40 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c697fa700 1 -- 172.21.15.39:0/2022544191 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8c7c00ebe0 con 0x5644ca492a40 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2022544191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644ca3c3390 con 0x5644ca492a40 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c697fa700 1 -- 172.21.15.39:0/2022544191 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c00ede0 con 0x5644ca492a40 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c9a92fb80 10 client.17340 fetch_fsmap learned FSMap version 32 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.403+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2022544191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644ca4ab390 con 0x5644ca492a40 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c697fa700 1 -- 172.21.15.39:0/2022544191 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5644ca3c3390 con 0x5644ca492a40 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c697fa700 1 -- 172.21.15.39:0/2022544191 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8c7c01f490 con 0x5644ca492a40 2024-05-04T03:29:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c9a92fb80 10 client.17340 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:29:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c9a92fb80 10 client.17340 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:29:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c9a92fb80 20 client.17340 populate_metadata read hostname 'smithi039' 2024-05-04T03:29:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5644ca4ada20 0x5644ca4afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c9a92fb80 4 client.17340 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:29:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2022544191 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644ca3bd430 con 0x5644ca4ada20 2024-05-04T03:29:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.404+0000 7f8c894b6700 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5644ca4ada20 0x5644ca4afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.405+0000 7f8c697fa700 10 client.17340 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.405+0000 7f8c894b6700 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5644ca4ada20 0x5644ca4afea0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e62c372d291bfa8 server_cookie=659edbf0aa795580 in_seq=0 out_seq=0 2024-05-04T03:29:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c697fa700 1 -- 172.21.15.39:0/2022544191 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644ca3bd430 con 0x5644ca4ada20 2024-05-04T03:29:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c697fa700 10 client.17340 handle_command_reply: tid=0 2024-05-04T03:29:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 client.17340 shutdown 2024-05-04T03:29:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2022544191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca492a40 msgr2=0x5644ca49ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca492a40 0x5644ca49ac60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c003a60 tx=0x7f8c7c0047c0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2022544191 shutdown_connections 2024-05-04T03:29:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5644ca4ada20 0x5644ca4afea0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca492a40 0x5644ca49ac60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca49dbb0 0x5644ca49b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2022544191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca49b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c697fa700 0 client.17340 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2022544191 >> 172.21.15.39:0/2022544191 conn(0x5644ca315eb0 msgr2=0x5644ca4944e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2022544191 shutdown_connections 2024-05-04T03:29:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2022544191 wait complete. 2024-05-04T03:29:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.406+0000 7f8c9a92fb80 20 client.17340 trim_cache size 0 max 16384 2024-05-04T03:29:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.426+0000 7f8c9a92fb80 1 Processor -- start 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.426+0000 7f8c9a92fb80 1 -- start start 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.426+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca494ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.426+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca49b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.426+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 0x5644ca49ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841b0d20 con 0x5644ca315eb0 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841b2860 con 0x5644ca492a40 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841b13f0 con 0x5644ca493390 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca494ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca494ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45050/0 (socket says 172.21.15.39:45050) 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 -- 172.21.15.39:0/1252062808 learned_addr learned my addr 172.21.15.39:0/1252062808 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 0x5644ca49ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c894b6700 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca49b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 -- 172.21.15.39:0/1252062808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 msgr2=0x5644ca49ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 0x5644ca49ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 -- 172.21.15.39:0/1252062808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 msgr2=0x5644ca49b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca49b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 -- 172.21.15.39:0/1252062808 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ca4991d0 con 0x5644ca315eb0 2024-05-04T03:29:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca494ed0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c01f8b0 tx=0x7f8c7c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518e4dd164f60d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c6b7fe700 1 -- 172.21.15.39:0/1252062808 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c01f400 con 0x5644ca315eb0 2024-05-04T03:29:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c6b7fe700 1 -- 172.21.15.39:0/1252062808 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c7c023070 con 0x5644ca315eb0 2024-05-04T03:29:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.427+0000 7f8c6b7fe700 1 -- 172.21.15.39:0/1252062808 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c028a10 con 0x5644ca315eb0 2024-05-04T03:29:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/1252062808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 msgr2=0x5644ca494ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca494ed0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c01f8b0 tx=0x7f8c7c0046d0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/1252062808 shutdown_connections 2024-05-04T03:29:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca494ed0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca493390 0x5644ca49ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/1252062808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca492a40 0x5644ca49b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/1252062808 >> 172.21.15.39:0/1252062808 conn(0x5644ca487ad0 msgr2=0x5644ca49c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/1252062808 shutdown_connections 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.428+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/1252062808 wait complete. 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 1 -- start start 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 1 Processor -- start 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca4a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca492a40 0x5644ca4a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca4a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841b2190 con 0x5644ca315eb0 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841b1ac0 con 0x5644ca493390 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841aece0 con 0x5644ca492a40 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c9a92fb80 10 client.0 _ll_register_callbacks cb 0x5644ca34d9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.429+0000 7f8c89cb7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca4a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c89cb7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca4a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45052/0 (socket says 172.21.15.39:45052) 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c89cb7700 1 -- 172.21.15.39:0/2544129456 learned_addr learned my addr 172.21.15.39:0/2544129456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c894b6700 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca492a40 0x5644ca4a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca4a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c89cb7700 1 -- 172.21.15.39:0/2544129456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca492a40 msgr2=0x5644ca4a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca492a40 0x5644ca4a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c89cb7700 1 -- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 msgr2=0x5644ca4a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca4a2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c89cb7700 1 -- 172.21.15.39:0/2544129456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ca4a35d0 con 0x5644ca315eb0 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c8a4b8700 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca4a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c89cb7700 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca4a5e80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c01f930 tx=0x7f8c7c01fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac43674ec95bc614 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c697fa700 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c697fa700 1 -- 172.21.15.39:0/2544129456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c040060 con 0x5644ca315eb0 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c697fa700 1 -- 172.21.15.39:0/2544129456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f8c7c00e890 con 0x5644ca315eb0 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2544129456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c023070 con 0x5644ca315eb0 2024-05-04T03:29:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.430+0000 7f8c697fa700 1 -- 172.21.15.39:0/2544129456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c00ead0 con 0x5644ca315eb0 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c9a92fb80 10 client.17346 fetch_fsmap learned FSMap version 32 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2544129456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5644ca518ff0 con 0x5644ca315eb0 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c697fa700 1 -- 172.21.15.39:0/2544129456 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c7c023070 con 0x5644ca315eb0 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c697fa700 1 -- 172.21.15.39:0/2544129456 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8c7c004c10 con 0x5644ca315eb0 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c9a92fb80 10 client.17346 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c9a92fb80 10 client.17346 resolve_mds: validated gid 24506 aka daemon mds.b 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c9a92fb80 20 client.17346 populate_metadata read hostname 'smithi039' 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5644ca49dbb0 0x5644ca51b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c9a92fb80 4 client.17346 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2544129456 --> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5644ca2352c0 con 0x5644ca49dbb0 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.431+0000 7f8c894b6700 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5644ca49dbb0 0x5644ca51b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c697fa700 10 client.17346 ms_handle_connect on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c894b6700 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5644ca49dbb0 0x5644ca51b660 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f798b0e0b4c68b2 server_cookie=6edac499fddb281 in_seq=0 out_seq=0 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c697fa700 1 -- 172.21.15.39:0/2544129456 <== mds.0 v2:172.21.15.66:6838/4050747247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5644ca2352c0 con 0x5644ca49dbb0 2024-05-04T03:29:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c697fa700 10 client.17346 handle_command_reply: tid=0 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 client.17346 shutdown 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2544129456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 msgr2=0x5644ca4a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca4a5e80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c01f930 tx=0x7f8c7c01fcc0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2544129456 shutdown_connections 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:6838/4050747247,v1:172.21.15.66:6839/4050747247] conn(0x5644ca49dbb0 0x5644ca51b660 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5644ca315eb0 0x5644ca4a5e80 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5644ca493390 0x5644ca4a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c697fa700 0 client.17346 ms_handle_reset on v2:172.21.15.66:6838/4050747247 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 --2- 172.21.15.39:0/2544129456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ca492a40 0x5644ca4a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.432+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2544129456 >> 172.21.15.39:0/2544129456 conn(0x5644ca487ad0 msgr2=0x5644ca4a9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2544129456 shutdown_connections 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c9a92fb80 1 -- 172.21.15.39:0/2544129456 wait complete. 2024-05-04T03:29:13.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c9a92fb80 20 client.17346 trim_cache size 0 max 16384 2024-05-04T03:29:13.440 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:29:13.440 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 32, 2024-05-04T03:29:13.440 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24506, 2024-05-04T03:29:13.440 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 51, 2024-05-04T03:29:13.440 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": {} 2024-05-04T03:29:13.440 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:29:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c6c0b2880 msgr2=0x7f8c6c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c6c0b2880 0x7f8c6c0b4d90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c011490 tx=0x7f8c7c009260 comp rx=0 tx=0).stop 2024-05-04T03:29:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 msgr2=0x7f8c841a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 0x7f8c841a1760 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c74007a00 tx=0x7f8c74007a30 comp rx=0 tx=0).stop 2024-05-04T03:29:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/3958732679 shutdown_connections 2024-05-04T03:29:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f8c6c0b2880 0x7f8c6c0b4d90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c84108760 0x7f8c841a1760 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c841034e0 0x7f8c8419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 --2- 172.21.15.39:0/3958732679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c84107e10 0x7f8c8419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/3958732679 >> 172.21.15.39:0/3958732679 conn(0x7f8c840fef30 msgr2=0x7f8c8410ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/3958732679 shutdown_connections 2024-05-04T03:29:13.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:13.433+0000 7f8c8bf1b700 1 -- 172.21.15.39:0/3958732679 wait complete. 2024-05-04T03:29:14.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:13 smithi039 ceph-mon[29715]: pgmap v939: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 748 op/s 2024-05-04T03:29:14.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:13 smithi066 ceph-mon[33264]: pgmap v939: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 748 op/s 2024-05-04T03:29:14.175 INFO:tasks.quiescer:done joining Quiescers 2024-05-04T03:29:14.175 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-04T03:29:14.186 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T03:29:14.187 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:13 smithi167 ceph-mon[33614]: pgmap v939: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.9 MiB/s wr, 748 op/s 2024-05-04T03:29:14.195 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-04T03:29:14.230 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T03:29:14.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 -- 172.21.15.39:0/548193768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 msgr2=0x7fbad010ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 --2- 172.21.15.39:0/548193768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 0x7fbad010ae50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009ca0 tx=0x7fbac802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 -- 172.21.15.39:0/548193768 shutdown_connections 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 --2- 172.21.15.39:0/548193768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 0x7fbad010ae50 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 --2- 172.21.15.39:0/548193768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad0103d30 0x7fbad0104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 --2- 172.21.15.39:0/548193768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0102af0 0x7fbad01037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 -- 172.21.15.39:0/548193768 >> 172.21.15.39:0/548193768 conn(0x7fbad00fdce0 msgr2=0x7fbad0100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 -- 172.21.15.39:0/548193768 shutdown_connections 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.393+0000 7fbad638c640 1 -- 172.21.15.39:0/548193768 wait complete. 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad638c640 1 Processor -- start 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad638c640 1 -- start start 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad638c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad0102af0 0x7fbad00ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad638c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0103d30 0x7fbad00fe210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad638c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 0x7fbad00fe750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad638c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbad010e500 con 0x7fbad0108a30 2024-05-04T03:29:14.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad638c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbad010e380 con 0x7fbad0102af0 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad638c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbad010e680 con 0x7fbad0103d30 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad4929640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 0x7fbad00fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad4929640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 0x7fbad00fe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45070/0 (socket says 172.21.15.39:45070) 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbad4929640 1 -- 172.21.15.39:0/298277052 learned_addr learned my addr 172.21.15.39:0/298277052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbacffff640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad0102af0 0x7fbad00ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.394+0000 7fbacf7fe640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0103d30 0x7fbad00fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbad4929640 1 -- 172.21.15.39:0/298277052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0103d30 msgr2=0x7fbad00fe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbad4929640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0103d30 0x7fbad00fe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbad4929640 1 -- 172.21.15.39:0/298277052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad0102af0 msgr2=0x7fbad00ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbad4929640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad0102af0 0x7fbad00ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbad4929640 1 -- 172.21.15.39:0/298277052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad00fee80 con 0x7fbad0108a30 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbad4929640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 0x7fbad00fe750 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbac8031040 tx=0x7fbac8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbacd7fa640 1 -- 172.21.15.39:0/298277052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8003d60 con 0x7fbad0108a30 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbacd7fa640 1 -- 172.21.15.39:0/298277052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbac80044d0 con 0x7fbad0108a30 2024-05-04T03:29:14.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad01a42f0 con 0x7fbad0108a30 2024-05-04T03:29:14.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbacd7fa640 1 -- 172.21.15.39:0/298277052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8030c50 con 0x7fbad0108a30 2024-05-04T03:29:14.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.395+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad01a4750 con 0x7fbad0108a30 2024-05-04T03:29:14.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.396+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad01a49e0 con 0x7fbad0108a30 2024-05-04T03:29:14.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.398+0000 7fbacd7fa640 1 -- 172.21.15.39:0/298277052 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fbac8004040 con 0x7fbad0108a30 2024-05-04T03:29:14.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.399+0000 7fbacd7fa640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbabc0b2870 0x7fbabc0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:14.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.399+0000 7fbacffff640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbabc0b2870 0x7fbabc0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:14.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.399+0000 7fbacd7fa640 1 -- 172.21.15.39:0/298277052 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fbac81036d0 con 0x7fbad0108a30 2024-05-04T03:29:14.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.399+0000 7fbacffff640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbabc0b2870 0x7fbabc0b4d80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbad01036f0 tx=0x7fbac0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:14.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.403+0000 7fbacd7fa640 1 -- 172.21.15.39:0/298277052 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbac80cb3a0 con 0x7fbad0108a30 2024-05-04T03:29:14.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.596+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbad01a4bb0 con 0x7fbad0108a30 2024-05-04T03:29:14.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.597+0000 7fbacd7fa640 1 -- 172.21.15.39:0/298277052 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22541 (secure 0 0 0) 0x7fbac80caca0 con 0x7fbad0108a30 2024-05-04T03:29:14.600 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:14.600 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:08:16.237923+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.599+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbabc0b2870 msgr2=0x7fbabc0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.599+0000 7fbad638c640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbabc0b2870 0x7fbabc0b4d80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbad01036f0 tx=0x7fbac0005f90 comp rx=0 tx=0).stop 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.599+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 msgr2=0x7fbad00fe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.599+0000 7fbad638c640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 0x7fbad00fe750 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbac8031040 tx=0x7fbac8003970 comp rx=0 tx=0).stop 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.600+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 shutdown_connections 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.600+0000 7fbad638c640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fbabc0b2870 0x7fbabc0b4d80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.600+0000 7fbad638c640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad0108a30 0x7fbad00fe750 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.600+0000 7fbad638c640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0103d30 0x7fbad00fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.600+0000 7fbad638c640 1 --2- 172.21.15.39:0/298277052 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad0102af0 0x7fbad00ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.600+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 >> 172.21.15.39:0/298277052 conn(0x7fbad00fdce0 msgr2=0x7fbad0109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:14.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.600+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 shutdown_connections 2024-05-04T03:29:14.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.600+0000 7fbad638c640 1 -- 172.21.15.39:0/298277052 wait complete. 2024-05-04T03:29:14.614 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T03:02:42.983535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T03:07:32.727181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T03:07:36.009986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T03:29:14.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T03:29:14.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 -- 172.21.15.39:0/2888827199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 msgr2=0x7fb94c104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 --2- 172.21.15.39:0/2888827199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 0x7fb94c104920 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb940009900 tx=0x7fb94002a410 comp rx=0 tx=0).stop 2024-05-04T03:29:14.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 -- 172.21.15.39:0/2888827199 shutdown_connections 2024-05-04T03:29:14.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 --2- 172.21.15.39:0/2888827199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c109b60 0x7fb94c10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 --2- 172.21.15.39:0/2888827199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c104e60 0x7fb94c105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 --2- 172.21.15.39:0/2888827199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 0x7fb94c104920 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 -- 172.21.15.39:0/2888827199 >> 172.21.15.39:0/2888827199 conn(0x7fb94c0fee10 msgr2=0x7fb94c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 -- 172.21.15.39:0/2888827199 shutdown_connections 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.727+0000 7fb952ea8640 1 -- 172.21.15.39:0/2888827199 wait complete. 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb952ea8640 1 Processor -- start 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb952ea8640 1 -- start start 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb952ea8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 0x7fb94c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb952ea8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c104e60 0x7fb94c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb952ea8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c109b60 0x7fb94c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb952ea8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10f5d0 con 0x7fb94c103c20 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb952ea8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10f450 con 0x7fb94c109b60 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb952ea8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10f750 con 0x7fb94c104e60 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb950c44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 0x7fb94c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb950c44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 0x7fb94c19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45088/0 (socket says 172.21.15.39:45088) 2024-05-04T03:29:14.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb950c44640 1 -- 172.21.15.39:0/379960983 learned_addr learned my addr 172.21.15.39:0/379960983 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb94bfff640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c104e60 0x7fb94c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb951445640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c109b60 0x7fb94c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb950c44640 1 -- 172.21.15.39:0/379960983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c104e60 msgr2=0x7fb94c19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb950c44640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c104e60 0x7fb94c19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb950c44640 1 -- 172.21.15.39:0/379960983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c109b60 msgr2=0x7fb94c1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb950c44640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c109b60 0x7fb94c1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.728+0000 7fb950c44640 1 -- 172.21.15.39:0/379960983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c1a1dd0 con 0x7fb94c103c20 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.729+0000 7fb950c44640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 0x7fb94c19a290 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb940005760 tx=0x7fb940004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.729+0000 7fb949ffb640 1 -- 172.21.15.39:0/379960983 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94003f030 con 0x7fb94c103c20 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.729+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c1a2060 con 0x7fb94c103c20 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.729+0000 7fb949ffb640 1 -- 172.21.15.39:0/379960983 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb940003a80 con 0x7fb94c103c20 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.729+0000 7fb949ffb640 1 -- 172.21.15.39:0/379960983 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940035070 con 0x7fb94c103c20 2024-05-04T03:29:14.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.729+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c1a2550 con 0x7fb94c103c20 2024-05-04T03:29:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.730+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c108970 con 0x7fb94c103c20 2024-05-04T03:29:14.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.732+0000 7fb949ffb640 1 -- 172.21.15.39:0/379960983 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb940042030 con 0x7fb94c103c20 2024-05-04T03:29:14.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.733+0000 7fb949ffb640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb93c0b27f0 0x7fb93c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:14.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.733+0000 7fb94bfff640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb93c0b27f0 0x7fb93c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:14.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.733+0000 7fb949ffb640 1 -- 172.21.15.39:0/379960983 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb940102dd0 con 0x7fb94c103c20 2024-05-04T03:29:14.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.733+0000 7fb94bfff640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb93c0b27f0 0x7fb93c0b4d00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb938005d00 tx=0x7fb938005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:14.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.737+0000 7fb949ffb640 1 -- 172.21.15.39:0/379960983 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb9400cab00 con 0x7fb94c103c20 2024-05-04T03:29:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.932+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb94c02ddc0 con 0x7fb94c103c20 2024-05-04T03:29:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.932+0000 7fb949ffb640 1 -- 172.21.15.39:0/379960983 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fb94c02ddc0 con 0x7fb94c103c20 2024-05-04T03:29:14.934 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-05-04T03:29:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb93c0b27f0 msgr2=0x7fb93c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb93c0b27f0 0x7fb93c0b4d00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb938005d00 tx=0x7fb938005c70 comp rx=0 tx=0).stop 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 msgr2=0x7fb94c19a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 0x7fb94c19a290 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb940005760 tx=0x7fb940004270 comp rx=0 tx=0).stop 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 shutdown_connections 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb93c0b27f0 0x7fb93c0b4d00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb94c109b60 0x7fb94c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c104e60 0x7fb94c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 --2- 172.21.15.39:0/379960983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb94c103c20 0x7fb94c19a290 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 >> 172.21.15.39:0/379960983 conn(0x7fb94c0fee10 msgr2=0x7fb94c105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 shutdown_connections 2024-05-04T03:29:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:14.934+0000 7fb952ea8640 1 -- 172.21.15.39:0/379960983 wait complete. 2024-05-04T03:29:14.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:14 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/298277052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:29:14.948 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T03:29:15.071 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:14 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/298277052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:29:15.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 -- 172.21.15.39:0/2298687214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8108a50 msgr2=0x7f3db810ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 --2- 172.21.15.39:0/2298687214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8108a50 0x7f3db810ae70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009ca0 tx=0x7f3db402a8c0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 -- 172.21.15.39:0/2298687214 shutdown_connections 2024-05-04T03:29:15.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 --2- 172.21.15.39:0/2298687214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8108a50 0x7f3db810ae70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 --2- 172.21.15.39:0/2298687214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db8103d50 0x7f3db8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 --2- 172.21.15.39:0/2298687214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db8102b10 0x7f3db8103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 -- 172.21.15.39:0/2298687214 >> 172.21.15.39:0/2298687214 conn(0x7f3db80fdd00 msgr2=0x7f3db81001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:15.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 -- 172.21.15.39:0/2298687214 shutdown_connections 2024-05-04T03:29:15.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 -- 172.21.15.39:0/2298687214 wait complete. 2024-05-04T03:29:15.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 Processor -- start 2024-05-04T03:29:15.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.063+0000 7f3dbf714640 1 -- start start 2024-05-04T03:29:15.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbf714640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db8102b10 0x7f3db806aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbf714640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8103d50 0x7f3db819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbf714640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db8108a50 0x7f3db81a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbf714640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3db810e520 con 0x7f3db8103d50 2024-05-04T03:29:15.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbf714640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3db810e3a0 con 0x7f3db8108a50 2024-05-04T03:29:15.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbf714640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3db810e6a0 con 0x7f3db8102b10 2024-05-04T03:29:15.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8103d50 0x7f3db819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8103d50 0x7f3db819f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45104/0 (socket says 172.21.15.39:45104) 2024-05-04T03:29:15.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 -- 172.21.15.39:0/3063259502 learned_addr learned my addr 172.21.15.39:0/3063259502 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:15.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbd4b0640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db8102b10 0x7f3db806aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 -- 172.21.15.39:0/3063259502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db8102b10 msgr2=0x7f3db806aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db8102b10 0x7f3db806aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 -- 172.21.15.39:0/3063259502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db8108a50 msgr2=0x7f3db81a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbdcb1640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db8108a50 0x7f3db81a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db8108a50 0x7f3db81a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 -- 172.21.15.39:0/3063259502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db81a2010 con 0x7f3db8103d50 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dbccaf640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8103d50 0x7f3db819f2a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3da800d950 tx=0x7f3da800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.064+0000 7f3dae7fc640 1 -- 172.21.15.39:0/3063259502 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8017070 con 0x7f3db8103d50 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.065+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db81a2300 con 0x7f3db8103d50 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.065+0000 7f3dae7fc640 1 -- 172.21.15.39:0/3063259502 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3da8009c30 con 0x7f3db8103d50 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.065+0000 7f3dae7fc640 1 -- 172.21.15.39:0/3063259502 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da800f620 con 0x7f3db8103d50 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.065+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81a2850 con 0x7f3db8103d50 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.066+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db8107860 con 0x7f3db8103d50 2024-05-04T03:29:15.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.067+0000 7f3dae7fc640 1 -- 172.21.15.39:0/3063259502 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f3da8010460 con 0x7f3db8103d50 2024-05-04T03:29:15.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.068+0000 7f3dae7fc640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3da40b27f0 0x7f3da40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.068+0000 7f3dbd4b0640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3da40b27f0 0x7f3da40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.068+0000 7f3dae7fc640 1 -- 172.21.15.39:0/3063259502 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f3da80e1ee0 con 0x7f3db8103d50 2024-05-04T03:29:15.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.068+0000 7f3dbd4b0640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3da40b27f0 0x7f3da40b4d00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3db0005fd0 tx=0x7f3db0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:15.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.071+0000 7f3dae7fc640 1 -- 172.21.15.39:0/3063259502 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3da80a9c10 con 0x7f3db8103d50 2024-05-04T03:29:15.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:14 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/298277052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:29:15.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.264+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3db802dd80 con 0x7f3db8103d50 2024-05-04T03:29:15.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.264+0000 7f3dae7fc640 1 -- 172.21.15.39:0/3063259502 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f3da8009de0 con 0x7f3db8103d50 2024-05-04T03:29:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3da40b27f0 msgr2=0x7f3da40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3da40b27f0 0x7f3da40b4d00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3db0005fd0 tx=0x7f3db0005850 comp rx=0 tx=0).stop 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8103d50 msgr2=0x7f3db819f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8103d50 0x7f3db819f2a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3da800d950 tx=0x7f3da800de10 comp rx=0 tx=0).stop 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 shutdown_connections 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f3da40b27f0 0x7f3da40b4d00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3db8108a50 0x7f3db81a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8103d50 0x7f3db819f2a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 --2- 172.21.15.39:0/3063259502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db8102b10 0x7f3db806aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.266+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 >> 172.21.15.39:0/3063259502 conn(0x7f3db80fdd00 msgr2=0x7f3db8104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.267+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 shutdown_connections 2024-05-04T03:29:15.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.267+0000 7f3dbf714640 1 -- 172.21.15.39:0/3063259502 wait complete. 2024-05-04T03:29:15.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T03:29:15.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.393+0000 7fd571cd1640 1 -- 172.21.15.39:0/3094533074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c102b30 msgr2=0x7fd56c103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.393+0000 7fd571cd1640 1 --2- 172.21.15.39:0/3094533074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c102b30 0x7fd56c103830 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009900 tx=0x7fd55c02a410 comp rx=0 tx=0).stop 2024-05-04T03:29:15.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.393+0000 7fd571cd1640 1 -- 172.21.15.39:0/3094533074 shutdown_connections 2024-05-04T03:29:15.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.393+0000 7fd571cd1640 1 --2- 172.21.15.39:0/3094533074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd56c108a70 0x7fd56c10ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.393+0000 7fd571cd1640 1 --2- 172.21.15.39:0/3094533074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56c103d70 0x7fd56c104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.393+0000 7fd571cd1640 1 --2- 172.21.15.39:0/3094533074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c102b30 0x7fd56c103830 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.393+0000 7fd571cd1640 1 -- 172.21.15.39:0/3094533074 >> 172.21.15.39:0/3094533074 conn(0x7fd56c0fdd20 msgr2=0x7fd56c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:15.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.393+0000 7fd571cd1640 1 -- 172.21.15.39:0/3094533074 shutdown_connections 2024-05-04T03:29:15.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.394+0000 7fd571cd1640 1 -- 172.21.15.39:0/3094533074 wait complete. 2024-05-04T03:29:15.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.394+0000 7fd571cd1640 1 Processor -- start 2024-05-04T03:29:15.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.394+0000 7fd571cd1640 1 -- start start 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.394+0000 7fd571cd1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56c102b30 0x7fd56c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.394+0000 7fd571cd1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c103d70 0x7fd56c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd571cd1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd56c108a70 0x7fd56c1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd571cd1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c10f610 con 0x7fd56c103d70 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd571cd1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c10f490 con 0x7fd56c102b30 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd571cd1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c10f790 con 0x7fd56c108a70 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c103d70 0x7fd56c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c103d70 0x7fd56c19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45134/0 (socket says 172.21.15.39:45134) 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 -- 172.21.15.39:0/2918253825 learned_addr learned my addr 172.21.15.39:0/2918253825 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56b7fe640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56c102b30 0x7fd56c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56bfff640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd56c108a70 0x7fd56c1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 -- 172.21.15.39:0/2918253825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd56c108a70 msgr2=0x7fd56c1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd56c108a70 0x7fd56c1a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 -- 172.21.15.39:0/2918253825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56c102b30 msgr2=0x7fd56c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56c102b30 0x7fd56c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 -- 172.21.15.39:0/2918253825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c1a1e50 con 0x7fd56c103d70 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56b7fe640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56c102b30 0x7fd56c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd56affd640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c103d70 0x7fd56c19a850 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd56000d950 tx=0x7fd56000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd568ff9640 1 -- 172.21.15.39:0/2918253825 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560017030 con 0x7fd56c103d70 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd568ff9640 1 -- 172.21.15.39:0/2918253825 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd560004020 con 0x7fd56c103d70 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56c1a2140 con 0x7fd56c103d70 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd568ff9640 1 -- 172.21.15.39:0/2918253825 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5600055d0 con 0x7fd56c103d70 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.395+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56c1a2690 con 0x7fd56c103d70 2024-05-04T03:29:15.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.396+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56c107880 con 0x7fd56c103d70 2024-05-04T03:29:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.398+0000 7fd568ff9640 1 -- 172.21.15.39:0/2918253825 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd560009a30 con 0x7fd56c103d70 2024-05-04T03:29:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.399+0000 7fd568ff9640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5540b27f0 0x7fd5540b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.399+0000 7fd56b7fe640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5540b27f0 0x7fd5540b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.399+0000 7fd568ff9640 1 -- 172.21.15.39:0/2918253825 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd5600e2370 con 0x7fd56c103d70 2024-05-04T03:29:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.400+0000 7fd56b7fe640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5540b27f0 0x7fd5540b4d00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005f70 tx=0x7fd55c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.402+0000 7fd568ff9640 1 -- 172.21.15.39:0/2918253825 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5600aa0a0 con 0x7fd56c103d70 2024-05-04T03:29:15.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.593+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd56c02ddc0 con 0x7fd56c103d70 2024-05-04T03:29:15.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.594+0000 7fd568ff9640 1 -- 172.21.15.39:0/2918253825 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7fd5600a99a0 con 0x7fd56c103d70 2024-05-04T03:29:15.596 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-05-04T03:29:15.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5540b27f0 msgr2=0x7fd5540b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5540b27f0 0x7fd5540b4d00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005f70 tx=0x7fd55c000bc0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c103d70 msgr2=0x7fd56c19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c103d70 0x7fd56c19a850 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd56000d950 tx=0x7fd56000de10 comp rx=0 tx=0).stop 2024-05-04T03:29:15.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 shutdown_connections 2024-05-04T03:29:15.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd5540b27f0 0x7fd5540b4d00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd56c108a70 0x7fd56c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd56c103d70 0x7fd56c19a850 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 --2- 172.21.15.39:0/2918253825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd56c102b30 0x7fd56c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 >> 172.21.15.39:0/2918253825 conn(0x7fd56c0fdd20 msgr2=0x7fd56c104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:15.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 shutdown_connections 2024-05-04T03:29:15.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.596+0000 7fd571cd1640 1 -- 172.21.15.39:0/2918253825 wait complete. 2024-05-04T03:29:15.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-04T03:29:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 -- 172.21.15.39:0/3201012399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110108a30 msgr2=0x7f511010ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 --2- 172.21.15.39:0/3201012399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110108a30 0x7f511010ae50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5108009ca0 tx=0x7f510802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 -- 172.21.15.39:0/3201012399 shutdown_connections 2024-05-04T03:29:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 --2- 172.21.15.39:0/3201012399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110108a30 0x7f511010ae50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 --2- 172.21.15.39:0/3201012399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5110103d30 0x7f5110104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 --2- 172.21.15.39:0/3201012399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5110102af0 0x7f51101037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 -- 172.21.15.39:0/3201012399 >> 172.21.15.39:0/3201012399 conn(0x7f51100fdd00 msgr2=0x7f5110100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:15.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 -- 172.21.15.39:0/3201012399 shutdown_connections 2024-05-04T03:29:15.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.720+0000 7f5115176640 1 -- 172.21.15.39:0/3201012399 wait complete. 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.721+0000 7f5115176640 1 Processor -- start 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.721+0000 7f5115176640 1 -- start start 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f5115176640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110102af0 0x7f511019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f5115176640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5110103d30 0x7f511019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f5115176640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5110108a30 0x7f51101a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f5115176640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f511010e520 con 0x7f5110102af0 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f5115176640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f511010e3a0 con 0x7f5110103d30 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f5115176640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f511010e6a0 con 0x7f5110108a30 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110102af0 0x7f511019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110102af0 0x7f511019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46990/0 (socket says 172.21.15.39:46990) 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 -- 172.21.15.39:0/2210678501 learned_addr learned my addr 172.21.15.39:0/2210678501 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510f59e640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5110108a30 0x7f51101a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510e59c640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5110103d30 0x7f511019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 -- 172.21.15.39:0/2210678501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5110108a30 msgr2=0x7f51101a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5110108a30 0x7f51101a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 -- 172.21.15.39:0/2210678501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5110103d30 msgr2=0x7f511019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5110103d30 0x7f511019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 -- 172.21.15.39:0/2210678501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51101a1d80 con 0x7f5110102af0 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f510ed9d640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110102af0 0x7f511019a240 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f510000e9f0 tx=0x7f510000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f50f7fff640 1 -- 172.21.15.39:0/2210678501 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510000ccc0 con 0x7f5110102af0 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.722+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51101a2070 con 0x7f5110102af0 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.723+0000 7f50f7fff640 1 -- 172.21.15.39:0/2210678501 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51000035f0 con 0x7f5110102af0 2024-05-04T03:29:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.723+0000 7f50f7fff640 1 -- 172.21.15.39:0/2210678501 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510000ce60 con 0x7f5110102af0 2024-05-04T03:29:15.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.723+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51101a2570 con 0x7f5110102af0 2024-05-04T03:29:15.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.724+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51101a2830 con 0x7f5110102af0 2024-05-04T03:29:15.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.725+0000 7f50f7fff640 1 -- 172.21.15.39:0/2210678501 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f5100004000 con 0x7f5110102af0 2024-05-04T03:29:15.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.726+0000 7f50f7fff640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f50f80b27f0 0x7f50f80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.726+0000 7f510e59c640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f50f80b27f0 0x7f50f80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.726+0000 7f50f7fff640 1 -- 172.21.15.39:0/2210678501 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f51000e2780 con 0x7f5110102af0 2024-05-04T03:29:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.727+0000 7f510e59c640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f50f80b27f0 0x7f50f80b4d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5104005fd0 tx=0x7f5104009490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:15.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.729+0000 7f50f7fff640 1 -- 172.21.15.39:0/2210678501 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51000aa4b0 con 0x7f5110102af0 2024-05-04T03:29:15.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:15 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/379960983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:29:15.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:15 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/3063259502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-04T03:29:15.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:15 smithi039 ceph-mon[29715]: pgmap v940: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.8 MiB/s wr, 515 op/s 2024-05-04T03:29:15.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:15 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2918253825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-04T03:29:15.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.972+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f51101a2a00 con 0x7f5110102af0 2024-05-04T03:29:15.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.982+0000 7f50f7fff640 1 -- 172.21.15.39:0/2210678501 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+815640 (secure 0 0 0) 0x7f51000a9db0 con 0x7f5110102af0 2024-05-04T03:29:15.995 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:29:15.995 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:29:15.995 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 20, 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "active_gid": 14152, 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "active_name": "x", 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addrs": { 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2627629846 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2627629846 2024-05-04T03:29:15.996 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addr": "172.21.15.39:6801/2627629846", 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "active_change": "2024-05-04T03:00:14.875449+0000", 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "active_mgr_features": 4540701547738038271, 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "available": true, 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "standbys": [ 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 24112, 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "z", 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540701547738038271, 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-05-04T03:29:15.997 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:15.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:15.999 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.011 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-05-04T03:29:16.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-04T03:29:16.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-05-04T03:29:16.016 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.018 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.022 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.027 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.031 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.033 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.037 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-05-04T03:29:16.038 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-04T03:29:16.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-05-04T03:29:16.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-04T03:29:16.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.050 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-05-04T03:29:16.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-05-04T03:29:16.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.059 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-05-04T03:29:16.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.061 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-05-04T03:29:16.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-05-04T03:29:16.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-04T03:29:16.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.070 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.071 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.074 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.078 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-04T03:29:16.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.089 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.090 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-05-04T03:29:16.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.093 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.097 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-05-04T03:29:16.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.103 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.106 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.108 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-04T03:29:16.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-04T03:29:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-04T03:29:16.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.114 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.120 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.122 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.124 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.126 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-04T03:29:16.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-04T03:29:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.136 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.148 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.149 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.150 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.151 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.152 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.159 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.164 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.166 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.168 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-05-04T03:29:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.172 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.173 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-05-04T03:29:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-04T03:29:16.183 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.184 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.185 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.186 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.187 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.189 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-05-04T03:29:16.190 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.193 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.194 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.195 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.198 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.200 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.202 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.203 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.204 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.206 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.213 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.214 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-05-04T03:29:16.215 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-05-04T03:29:16.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.218 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.220 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-05-04T03:29:16.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.225 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.226 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.228 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.229 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.231 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.235 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.236 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.238 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.239 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.242 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.243 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.245 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.246 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.247 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.248 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.249 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-05-04T03:29:16.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.254 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.255 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.256 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.257 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.260 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-05-04T03:29:16.261 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.262 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.263 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.267 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.268 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.269 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.271 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-05-04T03:29:16.273 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.274 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.275 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.279 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-05-04T03:29:16.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.283 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.284 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-05-04T03:29:16.290 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.291 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-04T03:29:16.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.293 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.294 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-05-04T03:29:16.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-04T03:29:16.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-05-04T03:29:16.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.303 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-04T03:29:16.305 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.306 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.308 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.310 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.311 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.313 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.317 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.321 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.322 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.323 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.325 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.326 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.329 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.330 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.332 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.333 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.334 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.336 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.337 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.338 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-05-04T03:29:16.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.343 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.345 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.346 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.347 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.348 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.350 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-05-04T03:29:16.352 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.353 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.355 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.357 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-05-04T03:29:16.358 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-05-04T03:29:16.359 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.360 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.361 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.363 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.364 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.365 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-04T03:29:16.367 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.368 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.369 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.370 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.371 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.372 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.373 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.374 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.376 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.377 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.379 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.380 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.381 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.383 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.384 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.385 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.386 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.387 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-05-04T03:29:16.388 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.389 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.390 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.391 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.393 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.394 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.395 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.396 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.398 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.403 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.405 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.407 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.409 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.410 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.412 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-05-04T03:29:16.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.415 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.416 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.417 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.418 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.419 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.420 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.423 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.424 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.425 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.426 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.428 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.432 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.433 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.434 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.435 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.436 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.437 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.438 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.439 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.441 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.442 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.443 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.444 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.445 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 24116, 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "y", 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540701547738038271, 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-05-04T03:29:16.446 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.447 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.448 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.449 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.450 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.451 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-05-04T03:29:16.452 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.453 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.455 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.456 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.457 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.458 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.459 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.460 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.461 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.462 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.463 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-05-04T03:29:16.464 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-05-04T03:29:16.465 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-05-04T03:29:16.466 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.467 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-04T03:29:16.468 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.469 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.470 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.472 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-05-04T03:29:16.473 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.474 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.475 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.476 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.477 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.478 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.479 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.480 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.481 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-05-04T03:29:16.482 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-05-04T03:29:16.483 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.485 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.486 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-05-04T03:29:16.490 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.491 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-04T03:29:16.492 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.493 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.494 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.495 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.496 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.497 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-05-04T03:29:16.498 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.499 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-04T03:29:16.500 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.503 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.504 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.505 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-04T03:29:16.506 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.507 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-05-04T03:29:16.508 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-05-04T03:29:16.509 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.511 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.512 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.513 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-04T03:29:16.514 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.515 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.515 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.515 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.515 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.515 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.517 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.517 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.518 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.519 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.520 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.521 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.522 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-04T03:29:16.523 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-04T03:29:16.524 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.525 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.526 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-05-04T03:29:16.526 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-05-04T03:29:16.526 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.526 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.526 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.526 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.526 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.527 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.527 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.527 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-05-04T03:29:16.527 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.527 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.527 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.527 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.528 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-05-04T03:29:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-04T03:29:16.531 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.532 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.534 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.536 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-05-04T03:29:16.537 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.538 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-05-04T03:29:16.539 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-05-04T03:29:16.540 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.541 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.542 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.543 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.544 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.545 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.546 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.547 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.548 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.550 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.551 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.552 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.553 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.554 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-04T03:29:16.555 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.557 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.559 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.560 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.562 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.563 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.565 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.566 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.567 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.569 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.570 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.571 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-04T03:29:16.572 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-04T03:29:16.573 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-04T03:29:16.574 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-04T03:29:16.575 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-04T03:29:16.576 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-04T03:29:16.577 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-04T03:29:16.578 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-04T03:29:16.579 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-04T03:29:16.580 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-04T03:29:16.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.582 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.584 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-04T03:29:16.586 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.587 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.588 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.589 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.590 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.591 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.592 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.595 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.596 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-05-04T03:29:16.597 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.598 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.599 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.600 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.601 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.602 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.604 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.605 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.606 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.608 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.609 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-05-04T03:29:16.610 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.611 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.612 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.613 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.614 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-04T03:29:16.615 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.616 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.617 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.618 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.619 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.621 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.622 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.623 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.624 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.625 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-05-04T03:29:16.626 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.627 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.629 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.630 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-05-04T03:29:16.631 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-05-04T03:29:16.632 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.633 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.634 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.635 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.636 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.637 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.638 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.639 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.640 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.641 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-05-04T03:29:16.642 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.643 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.644 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-05-04T03:29:16.645 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.646 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.647 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.648 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.649 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.650 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.651 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.652 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.653 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.654 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-04T03:29:16.655 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-05-04T03:29:16.656 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:29:16.657 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.658 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.659 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.660 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.661 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.662 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.663 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-05-04T03:29:16.664 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.665 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.666 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.667 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.668 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.669 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.671 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.672 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.673 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.674 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.675 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.676 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.677 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.678 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-05-04T03:29:16.679 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.680 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.681 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.683 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.684 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-05-04T03:29:16.685 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.686 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.687 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-05-04T03:29:16.688 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.689 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-04T03:29:16.690 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.692 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.694 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.695 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.696 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.697 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.700 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.701 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.702 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.703 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.704 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.705 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.706 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.707 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.708 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.710 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-04T03:29:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.712 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.714 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.715 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.716 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-05-04T03:29:16.717 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.718 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.719 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.720 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.721 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.722 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.723 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.725 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.726 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.727 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.728 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.729 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.730 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.731 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.732 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.733 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.734 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.735 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.737 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.738 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.740 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.741 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.742 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.743 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.744 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.745 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.746 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.748 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.749 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.750 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.751 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.752 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.753 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.753 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.753 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.753 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.753 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.753 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-04T03:29:16.755 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-04T03:29:16.755 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.755 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.755 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.755 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.755 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.756 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.757 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.758 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.759 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.760 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.761 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.762 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.763 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.764 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-04T03:29:16.765 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-05-04T03:29:16.766 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.767 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-04T03:29:16.769 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.771 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.772 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.773 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.774 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.775 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.776 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.778 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.779 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.780 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.781 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.782 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.783 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.785 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.787 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.788 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.789 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.791 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.796 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.798 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.799 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.800 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.802 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.803 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.804 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.806 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-05-04T03:29:16.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.808 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.809 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.810 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.812 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.813 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.815 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.816 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.817 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.818 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.819 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.820 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.822 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.823 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.827 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.828 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.830 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.831 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.834 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-05-04T03:29:16.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-05-04T03:29:16.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.837 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.838 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.839 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.840 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.842 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.843 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.847 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.848 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.849 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-05-04T03:29:16.850 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.851 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.855 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.856 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.857 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.858 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.862 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.864 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.865 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.866 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.870 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.872 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.874 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.877 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.878 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.879 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.882 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.883 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.885 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.886 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.887 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.888 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.890 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.893 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.895 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.897 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.898 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-05-04T03:29:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.907 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.908 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-05-04T03:29:16.909 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.911 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.913 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: "modules": [ 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-05-04T03:29:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard", 2024-05-04T03:29:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: "iostat", 2024-05-04T03:29:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: "nfs", 2024-05-04T03:29:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: "restful" 2024-05-04T03:29:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-05-04T03:29:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-05-04T03:29:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.927 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.928 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.932 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-05-04T03:29:16.934 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-05-04T03:29:16.935 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.937 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-04T03:29:16.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-05-04T03:29:16.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:29:16.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.949 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.955 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-05-04T03:29:16.956 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:16.960 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-05-04T03:29:16.962 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.965 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.967 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.969 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.970 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.971 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-04T03:29:16.972 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-04T03:29:16.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-05-04T03:29:16.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-05-04T03:29:16.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-05-04T03:29:16.981 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.982 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.983 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.984 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-05-04T03:29:16.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-05-04T03:29:16.986 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-05-04T03:29:16.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-05-04T03:29:16.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-05-04T03:29:16.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-05-04T03:29:16.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-05-04T03:29:16.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:16.997 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:16.999 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.001 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.002 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-04T03:29:17.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.005 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-05-04T03:29:17.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-05-04T03:29:17.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-04T03:29:17.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.017 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.021 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-05-04T03:29:17.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-05-04T03:29:17.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-05-04T03:29:17.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.028 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.029 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-05-04T03:29:17.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-05-04T03:29:17.036 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.037 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.042 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.046 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.051 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-05-04T03:29:17.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-04T03:29:17.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.055 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.056 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.061 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-04T03:29:17.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-04T03:29:17.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.066 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.067 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.068 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.071 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-04T03:29:17.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-04T03:29:17.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-04T03:29:17.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-04T03:29:17.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:17.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.086 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.093 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.095 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-05-04T03:29:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-04T03:29:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.106 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.107 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.109 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.110 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.112 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-05-04T03:29:17.113 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-04T03:29:17.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.115 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.117 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.119 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.124 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.128 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-05-04T03:29:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.130 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:17.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:17.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.134 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.137 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.139 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.145 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.150 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.151 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.153 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.155 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.157 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-05-04T03:29:17.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-05-04T03:29:17.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.163 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-05-04T03:29:17.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.169 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.172 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.177 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.179 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.180 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.182 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.185 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.186 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-05-04T03:29:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.203 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-05-04T03:29:17.204 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.214 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.216 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.217 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.219 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.220 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.221 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.222 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.226 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-05-04T03:29:17.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.228 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.231 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.236 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.239 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-04T03:29:17.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.241 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.243 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.244 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-05-04T03:29:17.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-04T03:29:17.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-05-04T03:29:17.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.248 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.250 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-05-04T03:29:17.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-05-04T03:29:17.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.254 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.255 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.259 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-05-04T03:29:17.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.263 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.264 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.266 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.268 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.290 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-05-04T03:29:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-05-04T03:29:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.328 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.329 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.330 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-05-04T03:29:17.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.332 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.334 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.336 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.337 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.338 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-05-04T03:29:17.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.341 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.342 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.343 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.343 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-04T03:29:17.396 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.396 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.396 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.397 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.398 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.401 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.402 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.403 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.404 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.405 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.406 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.407 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.408 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.409 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.412 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.414 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.415 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.416 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.418 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.419 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.421 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.422 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.424 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.425 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.426 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.427 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.428 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.430 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-05-04T03:29:17.431 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-05-04T03:29:17.432 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-05-04T03:29:17.433 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-05-04T03:29:17.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.435 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.436 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-05-04T03:29:17.437 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-05-04T03:29:17.438 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-05-04T03:29:17.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.440 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-05-04T03:29:17.441 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-05-04T03:29:17.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "services": { 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard": "https://172.21.15.39:8443/" 2024-05-04T03:29:17.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "always_on_modules": { 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "octopus": [ 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "pacific": [ 2024-05-04T03:29:17.444 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "quincy": [ 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-04T03:29:17.445 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "reef": [ 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-04T03:29:17.446 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "squid": [ 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-05-04T03:29:17.447 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: "last_failure_osd_epoch": 3, 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: "active_clients": [ 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 573268839 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "libcephsqlite", 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 550966512 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-04T03:29:17.449 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4171673484 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 589610496 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-05-04T03:29:17.450 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:15 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/379960983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:15 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/3063259502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:15 smithi167 ceph-mon[33614]: pgmap v940: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.8 MiB/s wr, 515 op/s 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:15 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2918253825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:16 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2210678501' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:15 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/379960983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:15 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/3063259502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:15 smithi066 ceph-mon[33264]: pgmap v940: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.8 MiB/s wr, 515 op/s 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:15 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2918253825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-04T03:29:17.452 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:16 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2210678501' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-04T03:29:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f50f80b27f0 msgr2=0x7f50f80b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f50f80b27f0 0x7f50f80b4d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5104005fd0 tx=0x7f5104009490 comp rx=0 tx=0).stop 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110102af0 msgr2=0x7f511019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110102af0 0x7f511019a240 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f510000e9f0 tx=0x7f510000eeb0 comp rx=0 tx=0).stop 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 shutdown_connections 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f50f80b27f0 0x7f50f80b4d00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5110108a30 0x7f51101a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5110103d30 0x7f511019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 --2- 172.21.15.39:0/2210678501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5110102af0 0x7f511019a240 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 >> 172.21.15.39:0/2210678501 conn(0x7f51100fdd00 msgr2=0x7f5110109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 shutdown_connections 2024-05-04T03:29:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:15.987+0000 7f5115176640 1 -- 172.21.15.39:0/2210678501 wait complete. 2024-05-04T03:29:17.460 DEBUG:tasks.check_counter:Getting stats from a 2024-05-04T03:29:17.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-04T03:29:17.461 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:16 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2210678501' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-04T03:29:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:17.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:17 smithi039 ceph-mon[29715]: pgmap v941: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.7 MiB/s wr, 296 op/s 2024-05-04T03:29:18.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:17 smithi066 ceph-mon[33264]: pgmap v941: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.7 MiB/s wr, 296 op/s 2024-05-04T03:29:18.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:17 smithi167 ceph-mon[33614]: pgmap v941: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.7 MiB/s wr, 296 op/s 2024-05-04T03:29:19.493 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 606, 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 595, 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 5164340, 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 293614, 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.081127636, 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.030523046, 2024-05-04T03:29:19.494 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.033514396, 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.014786368, 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 594, 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.030869568, 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000051968 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.495 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 909312, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 218560 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1275, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1542, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 20496501, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 1043571, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.147601449, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.046368649, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.047909901, 2024-05-04T03:29:19.496 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.042102979, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1540, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.060126587, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000039043 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 638752 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:19.497 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 2630, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2630, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 17559195, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 835574, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 5, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.269379977, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.089941490, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.076972011, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.082898718, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2629, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.110659926, 2024-05-04T03:29:19.498 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000042092 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-04T03:29:19.499 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 6962, 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.194482756, 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000027934 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-04T03:29:19.500 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-05-04T03:29:19.501 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 1680, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 1680, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 1680, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 3, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-05-04T03:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:19.504 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-05-04T03:29:19.505 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-05-04T03:29:19.506 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 19403, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-04T03:29:19.507 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 23, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 3, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 45340741, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 45339656, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 19403 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1061, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 1311, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 250, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 65, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 69, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 4, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 1680, 2024-05-04T03:29:19.509 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 1684, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 4, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 126332, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 215440 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-05-04T03:29:19.510 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.516 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.517 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:19.521 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 1, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 2, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 1, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 1, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-05-04T03:29:19.522 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 2305, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:19.523 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:19.524 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 216776, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 71, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:19.525 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 159, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:19.526 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 3485158, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 22229, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 4135, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-05-04T03:29:19.527 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 4135, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4135, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2.385956848, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000577014 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 4135, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 4136 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 4135, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 4135, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-05-04T03:29:19.528 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 2447, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1686, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:19.529 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:19.530 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-04T03:29:19.531 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 80, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 2, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 8, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 9, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 1, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-05-04T03:29:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-04T03:29:19.533 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 4507, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 42792342, 2024-05-04T03:29:19.534 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 4507, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 4507, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 42792342, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 4135, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 41152680, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 4135, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 41152680, 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:19.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 4135, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 4135, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 4135, 2024-05-04T03:29:19.537 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 4135, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:19.538 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55c3c570e8f0": { 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:19.539 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:19.540 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:29:20.403 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-04T03:29:20.404 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-04T03:29:20.404 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-04T03:29:20.404 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-04T03:29:20.404 DEBUG:tasks.check_counter:Getting stats from d 2024-05-04T03:29:20.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-04T03:29:20.660 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:20.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:20 smithi039 ceph-mon[29715]: pgmap v942: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 672 KiB/s wr, 76 op/s 2024-05-04T03:29:20.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:20 smithi066 ceph-mon[33264]: pgmap v942: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 672 KiB/s wr, 76 op/s 2024-05-04T03:29:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:20 smithi167 ceph-mon[33614]: pgmap v942: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 672 KiB/s wr, 76 op/s 2024-05-04T03:29:22.464 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:29:22.464 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:22.464 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 0, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 0, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 0, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.001510467, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000000000, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.000328985, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:22.465 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 6, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 276, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3312, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 206208, 2024-05-04T03:29:22.466 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.022693262, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.017099919, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.002786462, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 275, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.018459897, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000067126 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 206208 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 363, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 358, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 854064, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 221216, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.057664092, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.023682973, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.027062310, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.008215616, 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:22.468 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 357, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.016869354, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000047253 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 854064, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 221216 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.469 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:22.470 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:22.471 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 148721, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 30, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-04T03:29:22.472 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 928, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-04T03:29:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 365, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 802742, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 365, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 365, 2024-05-04T03:29:22.474 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 802742, 2024-05-04T03:29:22.475 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:22.475 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:22.475 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:22.475 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:22.475 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:22.475 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:22.475 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:29:22.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:22 smithi066 ceph-mon[33264]: pgmap v943: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-04T03:29:22.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:22 smithi167 ceph-mon[33614]: pgmap v943: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-04T03:29:22.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:22 smithi039 ceph-mon[29715]: pgmap v943: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-05-04T03:29:23.285 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-04T03:29:23.285 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-04T03:29:23.285 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-04T03:29:23.285 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-04T03:29:23.285 DEBUG:tasks.check_counter:Getting stats from g 2024-05-04T03:29:23.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-04T03:29:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:24.684 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:24 smithi039 ceph-mon[29715]: pgmap v944: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-04T03:29:24.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:24 smithi167 ceph-mon[33614]: pgmap v944: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-04T03:29:24.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:24 smithi066 ceph-mon[33264]: pgmap v944: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-04T03:29:25.311 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 2577, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2579, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 771890, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 819487, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.261108176, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.080918682, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.070853477, 2024-05-04T03:29:25.312 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.087462650, 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2577, 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.114803674, 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000044549 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:29:25.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1104, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1373, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 272150, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 903142, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.134426151, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.045256662, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.028998371, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.048301808, 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:25.314 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1371, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.054858756, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000040013 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 552320 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 377, 2024-05-04T03:29:25.315 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 364, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 913116, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 220606, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.058473111, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.022607242, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.029488858, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.008131303, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 364, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.018477903, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000050763 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:25.316 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 912848, 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 220288 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-04T03:29:25.317 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 5950, 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.196763223, 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000033069 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.318 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-05-04T03:29:25.319 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-05-04T03:29:25.320 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-05-04T03:29:25.321 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:25.322 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-05-04T03:29:25.323 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:25.324 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 5, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-05-04T03:29:25.325 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 2, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 4195901, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 4195760, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.326 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 5 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 3, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 3, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 0, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 1, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 1, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 0, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 0, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 0, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 0, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-05-04T03:29:25.327 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 118844, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 215440 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.328 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-05-04T03:29:25.329 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.330 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-05-04T03:29:25.332 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-05-04T03:29:25.333 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.335 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.336 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.338 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.339 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 1, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 2, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 1, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 3525, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-04T03:29:25.340 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:25.341 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:25.342 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 190224, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 54, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:25.343 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 159, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 8512, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 13, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 3681, 2024-05-04T03:29:25.344 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 3681, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3681, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2.114961760, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000574561 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3681, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3682 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 3681, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 3681, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-05-04T03:29:25.345 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 2386, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1293, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:25.346 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-05-04T03:29:25.347 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-05-04T03:29:25.348 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 80, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 2, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 5, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 7, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 2, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-05-04T03:29:25.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.350 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 4054, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1570171, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 4054, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 4054, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1570171, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3681, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 1602, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3681, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1602, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.352 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3681, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3681, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3681, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3681, 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:25.353 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-04T03:29:25.354 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55864d2d68f0": { 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-05-04T03:29:25.355 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:25.356 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:29:26.236 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-04T03:29:26.236 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-04T03:29:26.236 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-04T03:29:26.236 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-04T03:29:26.236 DEBUG:tasks.check_counter:Getting stats from j 2024-05-04T03:29:26.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-04T03:29:26.491 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:26.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:26 smithi039 ceph-mon[29715]: pgmap v945: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-04T03:29:26.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:26 smithi066 ceph-mon[33264]: pgmap v945: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-04T03:29:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:26 smithi167 ceph-mon[33614]: pgmap v945: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 361, 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 358, 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 852400, 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 221200, 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 5, 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-05-04T03:29:28.264 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.064824757, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.026213525, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.030815243, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.008209016, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 356, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.015697588, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000044094 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 852400, 2024-05-04T03:29:28.265 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 221200 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 9, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 102, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 154544, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 97568, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 5, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.015653147, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.006165201, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.009630923, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000033328, 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:28.266 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 99, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.010594349, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000107013 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 154544, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 97568 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 5, 2024-05-04T03:29:28.267 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 185, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3152, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 113424, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 3, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.015285729, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.011440379, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.001687909, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 184, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.009634674, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000052362 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.268 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 113424 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:28.269 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:28.270 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:28.271 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 148712, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 29, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-05-04T03:29:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 928, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:28.273 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 371, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 954635, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 371, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 371, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 954635, 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-05-04T03:29:28.274 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-05-04T03:29:28.275 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-05-04T03:29:28.275 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-05-04T03:29:28.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:28.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-05-04T03:29:28.275 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-05-04T03:29:28.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:28 smithi066 ceph-mon[33264]: pgmap v946: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-04T03:29:28.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:28 smithi039 ceph-mon[29715]: pgmap v946: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-04T03:29:28.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:28 smithi167 ceph-mon[33614]: pgmap v946: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-05-04T03:29:28.954 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-04T03:29:28.955 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-04T03:29:28.955 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-04T03:29:28.955 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-04T03:29:28.955 DEBUG:tasks.check_counter:Getting stats from b 2024-05-04T03:29:28.955 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-04T03:29:29.219 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:29:30.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:30 smithi066 ceph-mon[33264]: pgmap v947: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-04T03:29:30.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:30 smithi167 ceph-mon[33614]: pgmap v947: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-04T03:29:30.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:30 smithi039 ceph-mon[29715]: pgmap v947: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-04T03:29:31.123 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 10000, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 9733, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3066996, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 4416675, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 34, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.865356407, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.262212528, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.608811821, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.102736288, 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:31.124 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 9734, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.535386750, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000055001 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 17919, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.003812437, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000212 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 62336, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 29856 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2181, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2207, 2024-05-04T03:29:31.125 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1516387, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 882516, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 10, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 6, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.215631812, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.074336766, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.133382854, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.021915266, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2207, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.135275063, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000061293 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.126 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2669, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000808847, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000303 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 946672, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 226256 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6718, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 5813, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2133397, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 4614211, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 211, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 6, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.565603228, 2024-05-04T03:29:31.127 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.199061255, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.430384885, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.017659866, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 5808, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.474400473, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000081680 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 11010, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.002823982, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000256 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 33376, 2024-05-04T03:29:31.128 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 564672 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1660, 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.470741762, 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000283579 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:31.129 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 17, 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000331992, 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000019528 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 6332, 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 647, 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 647, 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 7.765855235, 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.012002867 2024-05-04T03:29:31.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 5685, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 451, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 1, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 43, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 32, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 11, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 36, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 295, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 23, 2024-05-04T03:29:31.131 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 23, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 4, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 6730, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 1012, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 5682, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 15, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 1, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 296, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-05-04T03:29:31.132 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 1020, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 1603606212, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:31.133 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 208, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 288, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 253, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 41, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 1, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 321 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 32, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 73, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 44, 2024-05-04T03:29:31.134 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 44, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 64, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 3, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 2, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 1, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 13, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 402, 2024-05-04T03:29:31.135 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 1, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 1, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 586, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 584, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 584, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 2, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-05-04T03:29:31.136 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 3, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 84, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 83, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 83, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 1, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 1, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 1, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 6055542, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 6067642, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 240, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.601099149, 2024-05-04T03:29:31.137 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002504579 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 47, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 339, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 292, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 58, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 82, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 24, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 43, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 341, 2024-05-04T03:29:31.138 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 298, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 23, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 332, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 309, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 129964, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 231824 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 6499, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 426, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 6332, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 25, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 1217, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 278, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.316865965, 2024-05-04T03:29:31.139 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001139805 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 162, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.046168427, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000284990 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.003305260, 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.003305260 2024-05-04T03:29:31.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 153, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 7.228439150, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.047244700 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.141 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.142 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-05-04T03:29:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 31, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.041734795, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001346283 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.002713091, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000678272 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.004121763, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001373921 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.093550585, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.031183528 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.145 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.146 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1, 2024-05-04T03:29:31.147 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001164875, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001164875 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.148 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 8, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.023577238, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.002947154 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.004214086, 2024-05-04T03:29:31.149 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001404695 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 3, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 2, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2024-05-04T03:29:31.150 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 64, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 64, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 1102, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 360, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 360, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:31.151 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:31.152 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 311191, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 105, 2024-05-04T03:29:31.153 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 159, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-05-04T03:29:31.154 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 226261, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 1249, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 1595, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 2476608, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 37, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 1595, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1595, 2024-05-04T03:29:31.155 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 23.436853931, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.014693952 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1595, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 2540 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 1595, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 1, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 1594, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 423, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 26, 2024-05-04T03:29:31.156 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 1, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 241, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 253, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 1, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 514, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 52, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:31.157 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-05-04T03:29:31.158 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 80, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 12, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 13, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 1, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 1015, 2024-05-04T03:29:31.159 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 0, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 14 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 765 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 2, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-05-04T03:29:31.160 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 2, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 2, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 2, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 18895, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 4981799, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 18895, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 18895, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 4981799, 2024-05-04T03:29:31.161 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:31.162 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 1595, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 2476810, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1595, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 2476810, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:31.163 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 1595, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 1595, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1595, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1595, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:31.164 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 2, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 202, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 2, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 1, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 202, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x564c14ab6680": { 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 586, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1873338, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 586, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 240, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1873338, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:31.166 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.167 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:31.167 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-04T03:29:31.930 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-05-04T03:29:31.930 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-04T03:29:31.930 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=32 2024-05-04T03:29:31.930 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=73 2024-05-04T03:29:31.930 DEBUG:tasks.check_counter:Getting stats from e 2024-05-04T03:29:31.930 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-04T03:29:32.180 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:29:32.569 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:32 smithi066 ceph-mon[33264]: pgmap v948: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-04T03:29:32.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:32 smithi167 ceph-mon[33614]: pgmap v948: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-04T03:29:32.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:32 smithi039 ceph-mon[29715]: pgmap v948: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 612, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 603, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 983183, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 407237, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 14, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.079491404, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.027664685, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.045482291, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.009165743, 2024-05-04T03:29:34.051 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 598, 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.028944730, 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000048402 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 382, 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000091906, 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000240 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 923728, 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 228640 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:34.052 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 1346, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1621, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 353425, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 1007670, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 9, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.153801514, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.076182172, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.075352288, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.006094376, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1624, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.073452040, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000045229 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2576, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000744048, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000288 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 555088 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2808, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1734, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 927604, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 353781, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 9, 2024-05-04T03:29:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 5, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.158146537, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.057198806, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.135171247, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000495743, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1732, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.061829282, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000035698 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4338, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001181553, 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000272 2024-05-04T03:29:34.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 21, 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001009476, 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000048070 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000012532, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000006266 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 92, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 70, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 70, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.928178002, 2024-05-04T03:29:34.057 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.013259685 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 22, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 11, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 34, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 34, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 23, 2024-05-04T03:29:34.058 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 4, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 8, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 8, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 4, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 255, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 210, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 22, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 23, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-05-04T03:29:34.059 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 4, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 234, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:34.060 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 82, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 82, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 113, 2024-05-04T03:29:34.061 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 68, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 69, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-05-04T03:29:34.062 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-05-04T03:29:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 13, 2024-05-04T03:29:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 285, 2024-05-04T03:29:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-05-04T03:29:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:29:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:34.063 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 3, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 3, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 2, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 2, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 2, 2024-05-04T03:29:34.064 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 4195369, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 0, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.002987794, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.001493897 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 0 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 38, 2024-05-04T03:29:34.065 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 40, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 2, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 58, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 62, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 4, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 34, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 38, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 4, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 8, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 8, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 123732, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 223632 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.066 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 170, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 92, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 14, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 324, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 70, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.928178002, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.013259685 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.068 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:34.069 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.070 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-05-04T03:29:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-05-04T03:29:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.071 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.072 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.073 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.075 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-05-04T03:29:34.077 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 1, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 0, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 13, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 13, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 1109, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:34.078 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:34.079 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:34.080 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 246823, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 79, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 159, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-05-04T03:29:34.081 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 188989, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 668, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 38, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 11760, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 30, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 38, 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-05-04T03:29:34.082 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 38, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.211998681, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.005578912 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 38, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 61 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 38, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 0, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 38, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 0, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 10, 2024-05-04T03:29:34.083 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 2, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 6, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 6, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 20, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:34.084 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-05-04T03:29:34.085 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 80, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 9, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 13, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 4, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 16, 2024-05-04T03:29:34.086 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 0, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 1 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 1, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 273 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:29:34.087 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 4762, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1811809, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 4762, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:34.088 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 4762, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1811809, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:34.089 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 38, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 11760, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 38, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 11760, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:34.090 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 38, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 38, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 38, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 38, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-04T03:29:34.091 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x5597bef3a680": { 2024-05-04T03:29:34.092 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 3, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1065, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 3, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 2, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1065, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.093 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:34.094 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-04T03:29:34.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:34 smithi066 ceph-mon[33264]: pgmap v949: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-04T03:29:34.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:34 smithi167 ceph-mon[33614]: pgmap v949: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-04T03:29:34.934 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-04T03:29:34.934 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-04T03:29:34.934 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-04T03:29:34.934 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=113 2024-05-04T03:29:34.934 DEBUG:tasks.check_counter:Getting stats from h 2024-05-04T03:29:34.935 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-04T03:29:34.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:34 smithi039 ceph-mon[29715]: pgmap v949: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-04T03:29:35.181 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:29:36.715 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:36 smithi066 ceph-mon[33264]: pgmap v950: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-04T03:29:36.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:36 smithi167 ceph-mon[33614]: pgmap v950: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-04T03:29:36.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:36 smithi039 ceph-mon[29715]: pgmap v950: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-04T03:29:37.014 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-04T03:29:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 1298, 2024-05-04T03:29:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1300, 2024-05-04T03:29:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 436073, 2024-05-04T03:29:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 412765, 2024-05-04T03:29:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 7, 2024-05-04T03:29:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 2, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.148818693, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.045627473, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.040097652, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.047722843, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1298, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.061479836, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000047365 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2416, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2686, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 618139, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 1321667, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.186770926, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.068320108, 2024-05-04T03:29:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.042252404, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.060088388, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2685, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.101896343, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000037950 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 554912 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 380, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 369, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 912564, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 224005, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.056209449, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.017890821, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.029869298, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.008669111, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 368, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.017440577, 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000047392 2024-05-04T03:29:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 911216, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 223040 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-04T03:29:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 6293, 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.177575034, 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000028217 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-05-04T03:29:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 1, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-05-04T03:29:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2024-05-04T03:29:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-05-04T03:29:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 3, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 2, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 2, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-05-04T03:29:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 4195369, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 4194892, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 3 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 3, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 3, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 0, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 1, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 1, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 0, 2024-05-04T03:29:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 121724, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 215440 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-05-04T03:29:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-05-04T03:29:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-05-04T03:29:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 1, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 0, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 1, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 2330, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 199295, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 58, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 159, 2024-05-04T03:29:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 8512, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 13, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 3712, 2024-05-04T03:29:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 21, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 3, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 3712, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3712, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 1.952988229, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000526128 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3712, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 3713 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 3712, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 3712, 2024-05-04T03:29:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 2408, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 1302, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 1, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-05-04T03:29:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 80, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-05-04T03:29:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 6, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 8, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 2, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-05-04T03:29:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 4090, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1576291, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 4090, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 4090, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1576291, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 3712, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 1070, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 3712, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1070, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 3712, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 3712, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 3712, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 3712, 2024-05-04T03:29:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:37.055 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x55cfd498e8f0": { 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:37.057 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-04T03:29:37.828 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-04T03:29:37.828 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-04T03:29:37.828 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-04T03:29:37.829 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-04T03:29:37.829 DEBUG:tasks.check_counter:Getting stats from k 2024-05-04T03:29:37.829 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-04T03:29:38.077 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:29:38.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:38 smithi066 ceph-mon[33264]: pgmap v951: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-04T03:29:38.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:38 smithi167 ceph-mon[33614]: pgmap v951: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-04T03:29:38.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:38 smithi039 ceph-mon[29715]: pgmap v951: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-04T03:29:39.848 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-05-04T03:29:39.848 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:39.848 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 3180, 2024-05-04T03:29:39.848 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 3167, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3614556, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 1115348, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.303847429, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.111125861, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.090886121, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.081851544, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3167, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.102542119, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000032378 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.849 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 916000, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 227488 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 283, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1976, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 646698, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 8, 2024-05-04T03:29:39.850 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.016640997, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.011310843, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.002479951, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000089665, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 281, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.015063289, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000053606 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.851 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 645360 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 1295, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1297, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 318676, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 411817, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.108902051, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.030654493, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.025403539, 2024-05-04T03:29:39.852 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.042855464, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1295, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.034773777, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000026852 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.853 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 6825, 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.181632594, 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000026612 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.854 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.855 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 18, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 4, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 14, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 8, 2024-05-04T03:29:39.856 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 3, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-05-04T03:29:39.857 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 1020, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 1603606212, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:39.858 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.859 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:39.860 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.861 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 586, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-05-04T03:29:39.862 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 2, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 84, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 6055542, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 6067642, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 6061643, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 586 2024-05-04T03:29:39.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 16, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 313, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 297, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 11, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 42, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 31, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 18, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 314, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 296, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-05-04T03:29:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 122660, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 215440 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.869 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-05-04T03:29:39.869 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.869 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.869 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.869 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.870 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.872 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.873 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.874 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.875 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-05-04T03:29:39.876 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 1, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 3, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 2, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 2, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 3550, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:39.877 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:39.878 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:39.879 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 191198, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 53, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:39.880 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 159, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 62538, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 284, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.881 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 4091, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 26, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 4, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 4091, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4091, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 1.702989681, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000416277 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 4091, 2024-05-04T03:29:39.882 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 4092 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 4091, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 4091, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 2592, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 1496, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2024-05-04T03:29:39.883 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 2, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-05-04T03:29:39.884 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-05-04T03:29:39.885 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 80, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 2, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 5, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 7, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 2, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:39.886 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:39.887 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 4477, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 3509492, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 4477, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 4477, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 3509492, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.888 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 4091, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 1873348, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 4091, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1873348, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.889 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 4091, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 4091, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 4091, 2024-05-04T03:29:39.890 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 4091, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:39.891 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x55e273fc68f0": { 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:39.892 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-05-04T03:29:39.893 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-05-04T03:29:40.678 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-04T03:29:40.678 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-04T03:29:40.678 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-04T03:29:40.678 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-04T03:29:40.678 DEBUG:tasks.check_counter:Getting stats from c 2024-05-04T03:29:40.679 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-04T03:29:40.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:40 smithi066 ceph-mon[33264]: pgmap v952: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:40.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:40 smithi167 ceph-mon[33614]: pgmap v952: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:40.929 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:29:40.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:40 smithi039 ceph-mon[29715]: pgmap v952: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:42.641 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:42 smithi167 ceph-mon[33614]: pgmap v953: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:29:42.790 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 6477, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 8636, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2540098, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 15177139, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 11, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.487306476, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.203128129, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.270650298, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.037788394, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 8640, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.277644261, 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000032134 2024-05-04T03:29:42.791 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 12458, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.002464885, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000197 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 938816, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 229568 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 3837, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 5175, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 890098, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 21986944, 2024-05-04T03:29:42.792 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 9, 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 7, 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.380741239, 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.185313529, 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.139704526, 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.058193875, 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5173, 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.163210082, 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000031550 2024-05-04T03:29:42.793 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4453, 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001020201, 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000229 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 558048 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 40338, 2024-05-04T03:29:42.794 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 36349, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 13106928, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 25822559, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 14, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 7, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 2.422629048, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.963321713, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 1.904348253, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.032886111, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 36343, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.760598437, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000020928 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.795 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 72935, 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.014076216, 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000192 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 1440 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-05-04T03:29:42.796 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5433, 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 7.730920617, 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001422956 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 942, 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.039088844, 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000041495 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 10029, 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 10028, 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 10028, 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 76.433998708, 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.007622058 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 1, 2024-05-04T03:29:42.798 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 29, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 2, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 1432, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 1432, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 1431, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 1045, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 1045, 2024-05-04T03:29:42.799 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 3, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 17558, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 14434, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 9, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 41, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-05-04T03:29:42.800 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 1, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 296, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-05-04T03:29:42.801 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 3941, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 3646, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:42.802 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:42.803 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 27335, 2024-05-04T03:29:42.803 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 13138, 2024-05-04T03:29:42.803 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 60, 2024-05-04T03:29:42.803 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 2467 2024-05-04T03:29:42.803 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.803 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-05-04T03:29:42.803 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 140, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 23, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 15, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 16, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 203, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 435, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 250, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 185, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:42.804 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 13, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 13495, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 2, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:42.805 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 19404, 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 19404, 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-05-04T03:29:42.806 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 23, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 23, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 3, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 45341315, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2747, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 28.389670493, 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.010334790 2024-05-04T03:29:42.807 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1063, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 1313, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 250, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 67, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 72, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 5, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 1432, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 1762, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 330, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 1045, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 1296, 2024-05-04T03:29:42.808 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 251, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 140088, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 223632 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 11126, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 1032, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 10029, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 344, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 2177, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 992, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 67.903799676, 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.068451410 2024-05-04T03:29:42.809 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 303, 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.271147466, 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.004195206 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.810 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 185, 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.468462246, 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002532228 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5984, 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2.929300797, 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000489522 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:42.811 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.812 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.813 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-05-04T03:29:42.814 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.814 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.814 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.814 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.814 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-05-04T03:29:42.814 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6, 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.007952164, 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001325360 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.815 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1820, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.365025028, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000750013 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 106, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.196423325, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001853050 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 415, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.870437736, 2024-05-04T03:29:42.816 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.004507078 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.011208489, 2024-05-04T03:29:42.817 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.005604244 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.818 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.819 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 215, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.410241781, 2024-05-04T03:29:42.820 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001908101 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-05-04T03:29:42.821 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 2209, 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 2209, 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 1115, 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:42.822 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:42.823 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:42.824 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 303959, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 105, 2024-05-04T03:29:42.825 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 159, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:42.826 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 3819878, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 36795, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 3850, 2024-05-04T03:29:42.827 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 41554393, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 33, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 3850, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3850, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 10.007819445, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002599433 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3850, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 4375 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 3850, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 115, 2024-05-04T03:29:42.828 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 3735, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 115, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 2862, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 145, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 2, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 307, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-05-04T03:29:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 80, 2024-05-04T03:29:42.831 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 696, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 218 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-05-04T03:29:42.832 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 13226 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 21, 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 6, 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 286, 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 250, 2024-05-04T03:29:42.833 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 50648, 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 11954400, 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 50648, 2024-05-04T03:29:42.834 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:42.835 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:42.835 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 50648, 2024-05-04T03:29:42.835 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 11954400, 2024-05-04T03:29:42.835 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:42.835 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.835 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.835 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.836 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.836 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.836 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:42.836 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:42.836 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-05-04T03:29:42.836 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:42.836 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:42.837 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:42.837 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:42.837 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:42.837 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 3850, 2024-05-04T03:29:42.837 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 41579643, 2024-05-04T03:29:42.837 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3850, 2024-05-04T03:29:42.838 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 41579643, 2024-05-04T03:29:42.838 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:42.838 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.838 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.839 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.839 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.839 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.839 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:42.839 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:42.839 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-05-04T03:29:42.840 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:42.840 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:42.840 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:42.840 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:42.840 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:42.840 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 3850, 2024-05-04T03:29:42.841 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 3850, 2024-05-04T03:29:42.841 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3850, 2024-05-04T03:29:42.841 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 3850, 2024-05-04T03:29:42.841 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:42.841 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.841 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.841 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.842 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.842 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.842 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:42.842 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:42.842 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-05-04T03:29:42.842 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:42.842 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 250, 2024-05-04T03:29:42.843 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 25250, 2024-05-04T03:29:42.843 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:42.843 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 250, 2024-05-04T03:29:42.843 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:42.843 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:42.843 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 115, 2024-05-04T03:29:42.844 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 25250, 2024-05-04T03:29:42.844 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:42.844 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.844 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.844 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.844 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.845 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:42.845 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55a76cacc410": { 2024-05-04T03:29:42.845 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:42.845 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-05-04T03:29:42.845 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:42.845 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 19404, 2024-05-04T03:29:42.846 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 41147011, 2024-05-04T03:29:42.846 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:42.846 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 19404, 2024-05-04T03:29:42.846 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:42.846 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:42.846 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2747, 2024-05-04T03:29:42.846 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 41147011, 2024-05-04T03:29:42.846 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:42.847 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:42.847 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:42.847 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:42.847 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.847 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:42.847 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-04T03:29:42.849 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:42 smithi066 ceph-mon[33264]: pgmap v953: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:29:42.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:42 smithi039 ceph-mon[29715]: pgmap v953: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:29:43.491 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-04T03:29:43.491 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-05-04T03:29:43.491 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=140 2024-05-04T03:29:43.491 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=23 2024-05-04T03:29:43.491 DEBUG:tasks.check_counter:Getting stats from f 2024-05-04T03:29:43.492 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-04T03:29:43.730 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:29:44.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:44 smithi066 ceph-mon[33264]: pgmap v954: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:44.919 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:44 smithi167 ceph-mon[33614]: pgmap v954: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:44.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:44 smithi039 ceph-mon[29715]: pgmap v954: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:45.486 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 4208, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 3086, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1292272, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 810272, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 9, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.297094541, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.128323923, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.197930734, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.007871367, 2024-05-04T03:29:45.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3089, 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.123879505, 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000040103 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6978, 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001590458, 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000227 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:29:45.488 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 441, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 519, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 938281, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 519826, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 11, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.066163080, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.025164064, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.034584284, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.008852386, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 518, 2024-05-04T03:29:45.489 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.031607214, 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000061017 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 23, 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000004075, 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000177 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 934704, 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 489984 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 210, 2024-05-04T03:29:45.490 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 396, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 61640, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 559334, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 14, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.027775987, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.012461531, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.014626836, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000295987, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 390, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.019839485, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000050870 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 396, 2024-05-04T03:29:45.491 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000075167, 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000189 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 393184 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 31, 2024-05-04T03:29:45.492 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001792541, 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000057823 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000006895, 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000003447 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 100, 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 79, 2024-05-04T03:29:45.493 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 79, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 5.260460576, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.066588108 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 21, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 11, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 34, 2024-05-04T03:29:45.494 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 34, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 24, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 4, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 9, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 9, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 4, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 271, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 228, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 20, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 23, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-05-04T03:29:45.495 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:45.496 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 89, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 89, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.497 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 113, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 68, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 69, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-05-04T03:29:45.498 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 13, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 285, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:45.499 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 5, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 5, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 2, 2024-05-04T03:29:45.500 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 4195901, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.007906415, 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001976603 2024-05-04T03:29:45.501 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 38, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 40, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 2, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 58, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 62, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 4, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 34, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 38, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 4, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 9, 2024-05-04T03:29:45.502 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 12, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 3, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 122508, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 223632 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 177, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 100, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 18, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 324, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.503 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000278788, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000278788 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-05-04T03:29:45.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 78, 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 5.260181788, 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.067438228 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.505 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.506 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.507 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.507 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-05-04T03:29:45.507 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.507 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.507 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.507 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.507 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.508 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.509 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-05-04T03:29:45.510 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:45.511 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.512 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.514 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-05-04T03:29:45.515 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 16, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 16, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 1117, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:45.516 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:45.518 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 247607, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 81, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 159, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:45.519 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 189445, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 681, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 43, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 12587, 2024-05-04T03:29:45.520 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 33, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 43, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 43, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.258995273, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006023145 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 43, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 68 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 43, 2024-05-04T03:29:45.521 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 43, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 4, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 7, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 6, 2024-05-04T03:29:45.522 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-05-04T03:29:45.523 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-05-04T03:29:45.524 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 80, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 9, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 13, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 4, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 20, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 1 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-05-04T03:29:45.525 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 273 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:45.526 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 4855, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1830542, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 4855, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4855, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1830542, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 43, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 12587, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 43, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 12587, 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:45.528 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 43, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 43, 2024-05-04T03:29:45.529 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 43, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 43, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:45.530 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55cc52c244e0": { 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:45.531 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 5, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1597, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 5, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1597, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.532 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:45.533 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-04T03:29:46.399 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-04T03:29:46.399 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-04T03:29:46.400 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-04T03:29:46.400 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=113 2024-05-04T03:29:46.400 DEBUG:tasks.check_counter:Getting stats from i 2024-05-04T03:29:46.400 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-04T03:29:46.638 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:29:46.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:46 smithi167 ceph-mon[33614]: pgmap v955: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:29:46.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:46 smithi066 ceph-mon[33264]: pgmap v955: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:29:46.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:46 smithi039 ceph-mon[29715]: pgmap v955: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 4635, 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 3512, 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2230554, 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 1028219, 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 9, 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.357774146, 2024-05-04T03:29:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.150637047, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.228100882, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.017115965, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3511, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.144465034, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000041146 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7019, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001884091, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000268 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.313 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 940064, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 229760 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 224, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 498, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 65011, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 831436, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 17, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.034918287, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.016966769, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.017039259, 2024-05-04T03:29:48.314 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000272353, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 496, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.030891116, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000062280 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 416, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000092096, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000221 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 649680 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:48.315 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 20, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 20, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 7447, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 13756, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 9, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.004597133, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000300108, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.003538012, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000507378, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 19, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.020506922, 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001079311 2024-05-04T03:29:48.316 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-05-04T03:29:48.317 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 44, 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001510990, 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000034340 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000010324, 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000005162 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.318 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 109, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 73, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 73, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.663084186, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.022781975 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 36, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 11, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-05-04T03:29:48.319 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 34, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 34, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 23, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 4, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 10, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 10, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 4, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 272, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 213, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 36, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 23, 2024-05-04T03:29:48.320 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-05-04T03:29:48.321 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 83, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:48.322 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 83, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 113, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 68, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 69, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:48.323 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 13, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 285, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-05-04T03:29:48.324 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 3, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 3, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 2, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-05-04T03:29:48.325 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 4195369, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.003235893, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001617946 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.326 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 38, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 40, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 2, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 58, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 62, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 4, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 34, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 38, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 4, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 10, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 10, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 123712, 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 223632 2024-05-04T03:29:48.327 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 202, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 109, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 14, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 332, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.003528843, 2024-05-04T03:29:48.328 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001764421 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 71, 2024-05-04T03:29:48.329 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.659555343, 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.023374018 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-05-04T03:29:48.330 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.331 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.332 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.332 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.332 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-05-04T03:29:48.332 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.333 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.334 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.335 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-05-04T03:29:48.336 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-05-04T03:29:48.337 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.338 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-05-04T03:29:48.339 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 1, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 14, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 14, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 1120, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:48.340 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:48.341 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:48.342 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 263751, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 89, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 159, 2024-05-04T03:29:48.343 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 189765, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 677, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 39, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 11810, 2024-05-04T03:29:48.344 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 33, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 39, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 39, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.259995266, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006666545 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 39, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 63 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 39, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 0, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 39, 2024-05-04T03:29:48.345 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 2, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 6, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 6, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:48.346 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-05-04T03:29:48.347 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 80, 2024-05-04T03:29:48.348 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 10, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 15, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 5, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 18, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 1 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-05-04T03:29:48.349 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 273 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.350 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 4875, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1839934, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 4875, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4875, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1839934, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.351 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 39, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 11810, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 39, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 11810, 2024-05-04T03:29:48.352 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:48.353 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 39, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 39, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 39, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 39, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:48.354 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x5568c075a5b0": { 2024-05-04T03:29:48.355 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 3, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1065, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 3, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1065, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:48.356 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.357 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:48.357 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-04T03:29:48.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:48 smithi167 ceph-mon[33614]: pgmap v956: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-04T03:29:48.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:48 smithi066 ceph-mon[33264]: pgmap v956: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-04T03:29:48.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:48 smithi039 ceph-mon[29715]: pgmap v956: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-04T03:29:49.093 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-04T03:29:49.093 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-04T03:29:49.093 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-04T03:29:49.093 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=113 2024-05-04T03:29:49.093 DEBUG:tasks.check_counter:Getting stats from l 2024-05-04T03:29:49.094 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-04T03:29:49.332 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:29:50.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:50 smithi066 ceph-mon[33264]: pgmap v957: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:50.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:50 smithi167 ceph-mon[33614]: pgmap v957: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:50.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:50 smithi039 ceph-mon[29715]: pgmap v957: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:51.155 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-04T03:29:51.155 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:29:51.155 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 3754, 2024-05-04T03:29:51.155 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 3756, 2024-05-04T03:29:51.155 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1064244, 2024-05-04T03:29:51.155 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 1193779, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.308399353, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.098931428, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.078467451, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.108479366, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3754, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.122275358, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000032572 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.156 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 392, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 379, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 917212, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 229946, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 3, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-05-04T03:29:51.157 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.061919201, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.022126605, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.033768630, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.008467733, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 379, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.017647809, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000046564 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.158 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 916656, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 229248 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 3, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 283, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 588, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 651054, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.015441944, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.010745931, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.002206478, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000026471, 2024-05-04T03:29:51.159 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 281, 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.014601593, 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000051962 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 650736 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.160 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 6370, 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.175263783, 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000027513 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-05-04T03:29:51.161 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-05-04T03:29:51.162 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 1, 2024-05-04T03:29:51.163 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-05-04T03:29:51.164 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:29:51.165 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-05-04T03:29:51.166 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-05-04T03:29:51.167 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 3, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-05-04T03:29:51.168 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 4195369, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 4194892, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 3 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-05-04T03:29:51.169 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 3, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 3, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 0, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 1, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 1, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 0, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 0, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 0, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 0, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 121436, 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 215440 2024-05-04T03:29:51.170 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.171 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-05-04T03:29:51.172 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.173 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.174 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.175 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-05-04T03:29:51.175 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.175 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.175 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.175 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.175 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-05-04T03:29:51.175 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.176 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.177 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.178 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.179 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-05-04T03:29:51.180 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-05-04T03:29:51.181 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-05-04T03:29:51.182 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 1, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 2386, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:29:51.183 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:29:51.184 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:29:51.185 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 182855, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 49, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 7536, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 159, 2024-05-04T03:29:51.186 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 8512, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 13, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 3757, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2024-05-04T03:29:51.187 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 3757, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3757, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1.886965932, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000502253 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3757, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 3758 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 3757, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 3757, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-05-04T03:29:51.188 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 2438, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 1317, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-05-04T03:29:51.189 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-05-04T03:29:51.190 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 80, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 2, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 4, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 5, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 1, 2024-05-04T03:29:51.191 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-05-04T03:29:51.192 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 4145, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 1586098, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 4145, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:51.193 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4145, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1586098, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:51.194 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 3757, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1070, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3757, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1070, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-05-04T03:29:51.195 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 3757, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 3757, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3757, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 3757, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.196 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:51.197 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55d61e1768f0": { 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-05-04T03:29:51.198 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-04T03:29:51.199 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-04T03:29:51.983 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-04T03:29:51.983 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-04T03:29:51.984 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-04T03:29:51.984 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-04T03:29:51.984 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-04T03:29:51.994 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-04T03:29:51.995 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:29:51.995 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T03:29:52.023 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:29:52.023 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T03:29:52.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-04T03:29:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:52.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:52 smithi066 ceph-mon[33264]: pgmap v958: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:29:52.871 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:52 smithi039 ceph-mon[29715]: pgmap v958: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:29:52.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:52 smithi167 ceph-mon[33614]: pgmap v958: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-04T03:29:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 -- 172.21.15.39:0/3949118657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c1039d0 msgr2=0x7f7a8c1106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 --2- 172.21.15.39:0/3949118657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c1039d0 0x7f7a8c1106f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009990 tx=0x7f7a80009ca0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 -- 172.21.15.39:0/3949118657 shutdown_connections 2024-05-04T03:29:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 --2- 172.21.15.39:0/3949118657 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a8c110c30 0x7f7a8c113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 --2- 172.21.15.39:0/3949118657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c103080 0x7f7a8c103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 --2- 172.21.15.39:0/3949118657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c1039d0 0x7f7a8c1106f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 -- 172.21.15.39:0/3949118657 >> 172.21.15.39:0/3949118657 conn(0x7f7a8c0fedf0 msgr2=0x7f7a8c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 -- 172.21.15.39:0/3949118657 shutdown_connections 2024-05-04T03:29:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.087+0000 7f7a921cb700 1 -- 172.21.15.39:0/3949118657 wait complete. 2024-05-04T03:29:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a921cb700 1 Processor -- start 2024-05-04T03:29:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a921cb700 1 -- start start 2024-05-04T03:29:54.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a921cb700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a8c103080 0x7f7a8c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:54.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a921cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c1039d0 0x7f7a8c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:54.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a921cb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c110c30 0x7f7a8c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:54.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a921cb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c116510 con 0x7f7a8c110c30 2024-05-04T03:29:54.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a921cb700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c116390 con 0x7f7a8c103080 2024-05-04T03:29:54.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a921cb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c116690 con 0x7f7a8c1039d0 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a8bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c110c30 0x7f7a8c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a8bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c110c30 0x7f7a8c1a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60572/0 (socket says 172.21.15.39:60572) 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.088+0000 7f7a8bfff700 1 -- 172.21.15.39:0/256016041 learned_addr learned my addr 172.21.15.39:0/256016041 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a8b7fe700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a8c103080 0x7f7a8c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a8affd700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c1039d0 0x7f7a8c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a8bfff700 1 -- 172.21.15.39:0/256016041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c1039d0 msgr2=0x7f7a8c19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a8bfff700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c1039d0 0x7f7a8c19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a8bfff700 1 -- 172.21.15.39:0/256016041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a8c103080 msgr2=0x7f7a8c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a8bfff700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a8c103080 0x7f7a8c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a8bfff700 1 -- 172.21.15.39:0/256016041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a1eb0 con 0x7f7a8c110c30 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a8bfff700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c110c30 0x7f7a8c1a1810 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007ee0 tx=0x7f7a8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a88ff9700 1 -- 172.21.15.39:0/256016041 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400dbd0 con 0x7f7a8c110c30 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a21a0 con 0x7f7a8c110c30 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a88ff9700 1 -- 172.21.15.39:0/256016041 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a8400dd70 con 0x7f7a8c110c30 2024-05-04T03:29:54.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a9ac0 con 0x7f7a8c110c30 2024-05-04T03:29:54.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.089+0000 7f7a88ff9700 1 -- 172.21.15.39:0/256016041 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a840058d0 con 0x7f7a8c110c30 2024-05-04T03:29:54.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.090+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a78005330 con 0x7f7a8c110c30 2024-05-04T03:29:54.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.092+0000 7f7a88ff9700 1 -- 172.21.15.39:0/256016041 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7a8401a020 con 0x7f7a8c110c30 2024-05-04T03:29:54.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.093+0000 7f7a88ff9700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a740b2880 0x7f7a740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:54.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.093+0000 7f7a88ff9700 1 -- 172.21.15.39:0/256016041 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7f7a840e1930 con 0x7f7a8c110c30 2024-05-04T03:29:54.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.093+0000 7f7a8b7fe700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a740b2880 0x7f7a740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:54.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.093+0000 7f7a8b7fe700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a740b2880 0x7f7a740b4d90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c005fd0 tx=0x7f7a7c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:54.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.099+0000 7f7a88ff9700 1 -- 172.21.15.39:0/256016041 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a84010070 con 0x7f7a8c110c30 2024-05-04T03:29:54.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.329+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a78005620 con 0x7f7a8c110c30 2024-05-04T03:29:54.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.330+0000 7f7a88ff9700 1 -- 172.21.15.39:0/256016041 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v80) ==== 81+0+589 (secure 0 0 0) 0x7f7a840a9660 con 0x7f7a8c110c30 2024-05-04T03:29:54.332 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3594679658 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:54.332 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3124453164 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:54.332 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3124453164 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:54.332 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3594679658 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:54.333 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/163870032 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:54.333 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/518372945 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:54.333 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1251709695 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:54.333 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/402196413 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:54.333 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1831352319 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:54.333 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1379489577 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a740b2880 msgr2=0x7f7a740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a740b2880 0x7f7a740b4d90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c005fd0 tx=0x7f7a7c005f60 comp rx=0 tx=0).stop 2024-05-04T03:29:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c110c30 msgr2=0x7f7a8c1a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c110c30 0x7f7a8c1a1810 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007ee0 tx=0x7f7a8400d3b0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 shutdown_connections 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f7a740b2880 0x7f7a740b4d90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a8c110c30 0x7f7a8c1a1810 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7a8c103080 0x7f7a8c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 --2- 172.21.15.39:0/256016041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a8c1039d0 0x7f7a8c19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 >> 172.21.15.39:0/256016041 conn(0x7f7a8c0fedf0 msgr2=0x7f7a8c1129c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 shutdown_connections 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:54.332+0000 7f7a921cb700 1 -- 172.21.15.39:0/256016041 wait complete. 2024-05-04T03:29:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:listed 10 entries 2024-05-04T03:29:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:54 smithi039 ceph-mon[29715]: pgmap v959: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:54.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:54 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/256016041' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:29:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:54 smithi066 ceph-mon[33264]: pgmap v959: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:54.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:54 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/256016041' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:29:54.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:54 smithi167 ceph-mon[33614]: pgmap v959: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-04T03:29:54.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:54 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/256016041' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:29:55.049 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-05-04T03:29:55.049 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T03:29:55.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-04T03:29:55.356 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:29:56.802 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:56 smithi039 ceph-mon[29715]: pgmap v960: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:29:56.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:56 smithi066 ceph-mon[33264]: pgmap v960: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:29:56.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:56 smithi167 ceph-mon[33614]: pgmap v960: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-04T03:29:57.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 -- 172.21.15.39:0/1327821360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd58410e970 msgr2=0x7fd584110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:57.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 --2- 172.21.15.39:0/1327821360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd58410e970 0x7fd584110e10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00b0a0 tx=0x7fd57c00b3b0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 -- 172.21.15.39:0/1327821360 shutdown_connections 2024-05-04T03:29:57.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 --2- 172.21.15.39:0/1327821360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd58410e970 0x7fd584110e10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 --2- 172.21.15.39:0/1327821360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd584105590 0x7fd5841059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 --2- 172.21.15.39:0/1327821360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584105ee0 0x7fd58410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 -- 172.21.15.39:0/1327821360 >> 172.21.15.39:0/1327821360 conn(0x7fd584078e00 msgr2=0x7fd584079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:57.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 -- 172.21.15.39:0/1327821360 shutdown_connections 2024-05-04T03:29:57.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.134+0000 7fd589de9700 1 -- 172.21.15.39:0/1327821360 wait complete. 2024-05-04T03:29:57.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd589de9700 1 Processor -- start 2024-05-04T03:29:57.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd589de9700 1 -- start start 2024-05-04T03:29:57.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd589de9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd584105590 0x7fd58419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:57.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd589de9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd584105ee0 0x7fd58419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:57.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd589de9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd58410e970 0x7fd58419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:57.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd589de9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd584114390 con 0x7fd584105ee0 2024-05-04T03:29:57.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd589de9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd584114210 con 0x7fd584105590 2024-05-04T03:29:57.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd589de9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd584114510 con 0x7fd58410e970 2024-05-04T03:29:57.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd582ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd584105ee0 0x7fd58419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:57.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.135+0000 7fd5837fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd584105590 0x7fd58419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd5837fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd584105590 0x7fd58419ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:40772/0 (socket says 172.21.15.39:40772) 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd582ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd584105ee0 0x7fd58419b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51020/0 (socket says 172.21.15.39:51020) 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd5837fe700 1 -- 172.21.15.39:0/539039776 learned_addr learned my addr 172.21.15.39:0/539039776 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd583fff700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd58410e970 0x7fd58419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd5837fe700 1 -- 172.21.15.39:0/539039776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd58410e970 msgr2=0x7fd58419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd5837fe700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd58410e970 0x7fd58419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd5837fe700 1 -- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd584105ee0 msgr2=0x7fd58419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd5837fe700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd584105ee0 0x7fd58419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd5837fe700 1 -- 172.21.15.39:0/539039776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58419fbf0 con 0x7fd584105590 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd5837fe700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd584105590 0x7fd58419ad20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd57400e9a0 tx=0x7fd57400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd580ff9700 1 -- 172.21.15.39:0/539039776 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57400cbd0 con 0x7fd584105590 2024-05-04T03:29:57.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58419fee0 con 0x7fd584105590 2024-05-04T03:29:57.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd580ff9700 1 -- 172.21.15.39:0/539039776 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd57400cd70 con 0x7fd584105590 2024-05-04T03:29:57.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd580ff9700 1 -- 172.21.15.39:0/539039776 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574018680 con 0x7fd584105590 2024-05-04T03:29:57.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.136+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5841a7800 con 0x7fd584105590 2024-05-04T03:29:57.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.137+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5840512c0 con 0x7fd584105590 2024-05-04T03:29:57.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.139+0000 7fd580ff9700 1 -- 172.21.15.39:0/539039776 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fd574010430 con 0x7fd584105590 2024-05-04T03:29:57.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.140+0000 7fd580ff9700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56c0b2880 0x7fd56c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:29:57.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.140+0000 7fd582ffd700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56c0b2880 0x7fd56c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:29:57.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.140+0000 7fd580ff9700 1 -- 172.21.15.39:0/539039776 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fd5740e27c0 con 0x7fd584105590 2024-05-04T03:29:57.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.140+0000 7fd582ffd700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56c0b2880 0x7fd56c0b4d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd584110780 tx=0x7fd57800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:29:57.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.143+0000 7fd580ff9700 1 -- 172.21.15.39:0/539039776 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5740aa4f0 con 0x7fd584105590 2024-05-04T03:29:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.375+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd58402e5f0 con 0x7fd584105590 2024-05-04T03:29:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.376+0000 7fd580ff9700 1 -- 172.21.15.39:0/539039776 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v80) ==== 81+0+589 (secure 0 0 0) 0x7fd5740a9df0 con 0x7fd584105590 2024-05-04T03:29:57.378 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3594679658 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3124453164 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3124453164 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3594679658 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/163870032 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/518372945 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1251709695 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/402196413 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1831352319 2024-05-05T02:59:47.622763+0000 2024-05-04T03:29:57.379 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1379489577 2024-05-05T03:00:14.875322+0000 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56c0b2880 msgr2=0x7fd56c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56c0b2880 0x7fd56c0b4d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd584110780 tx=0x7fd57800b410 comp rx=0 tx=0).stop 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd584105590 msgr2=0x7fd58419ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd584105590 0x7fd58419ad20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd57400e9a0 tx=0x7fd57400ecb0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 shutdown_connections 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fd56c0b2880 0x7fd56c0b4d90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd58410e970 0x7fd58419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd584105590 0x7fd58419ad20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 --2- 172.21.15.39:0/539039776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd584105ee0 0x7fd58419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 >> 172.21.15.39:0/539039776 conn(0x7fd584078e00 msgr2=0x7fd584100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:29:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 shutdown_connections 2024-05-04T03:29:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:29:57.378+0000 7fd589de9700 1 -- 172.21.15.39:0/539039776 wait complete. 2024-05-04T03:29:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:listed 10 entries 2024-05-04T03:29:57.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:57 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/539039776' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:29:57.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:57 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/539039776' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:29:57.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:57 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/539039776' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:29:58.180 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi039.front.sepia.ceph.com... 2024-05-04T03:29:58.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:29:58.181 DEBUG:teuthology.orchestra.run.smithi039:> sudo fusermount -u /tmp/cephfs 2024-05-04T03:29:58.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[85071]: fuse finished with error 0 and tester_r 0 2024-05-04T03:29:58.313 INFO:teuthology.orchestra.run:waiting for 300 2024-05-04T03:29:58.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:29:58 smithi167 ceph-mon[33614]: pgmap v961: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-04T03:29:58.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:29:58 smithi039 ceph-mon[29715]: pgmap v961: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-04T03:29:59.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:29:58 smithi066 ceph-mon[33264]: pgmap v961: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-04T03:30:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:00 smithi167 ceph-mon[33614]: pgmap v962: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-04T03:30:00.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:00 smithi167 ceph-mon[33614]: overall HEALTH_OK 2024-05-04T03:30:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:00 smithi039 ceph-mon[29715]: pgmap v962: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-04T03:30:00.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:00 smithi039 ceph-mon[29715]: overall HEALTH_OK 2024-05-04T03:30:01.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:00 smithi066 ceph-mon[33264]: pgmap v962: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-04T03:30:01.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:00 smithi066 ceph-mon[33264]: overall HEALTH_OK 2024-05-04T03:30:02.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:02 smithi167 ceph-mon[33614]: pgmap v963: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-04T03:30:02.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:02 smithi039 ceph-mon[29715]: pgmap v963: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-04T03:30:03.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:02 smithi066 ceph-mon[33264]: pgmap v963: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-04T03:30:04.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T03:30:04.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:30:04.416 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-04T03:30:04.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:30:04.463 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-05-04T03:30:04.529 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-04T03:30:04.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:30:04.530 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-04T03:30:04.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:30:04.600 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-05-04T03:30:04.695 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-04T03:30:04.706 INFO:tasks.cephadm:Teardown begin 2024-05-04T03:30:04.706 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-04T03:30:04.706 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:30:04.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:04 smithi167 ceph-mon[33614]: pgmap v964: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-04T03:30:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:04 smithi039 ceph-mon[29715]: pgmap v964: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-04T03:30:04.959 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:30:05.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:04 smithi066 ceph-mon[33264]: pgmap v964: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-04T03:30:06.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:06 smithi039 ceph-mon[29715]: pgmap v965: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-04T03:30:06.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:06 smithi167 ceph-mon[33614]: pgmap v965: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-04T03:30:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.967+0000 7efee76f2700 1 -- 172.21.15.39:0/2745943489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 msgr2=0x7efee0103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.967+0000 7efee76f2700 1 --2- 172.21.15.39:0/2745943489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 0x7efee0103d70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7efed0009990 tx=0x7efed0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 -- 172.21.15.39:0/2745943489 shutdown_connections 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 --2- 172.21.15.39:0/2745943489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee01042b0 0x7efee0110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 --2- 172.21.15.39:0/2745943489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee0108200 0x7efee0108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 --2- 172.21.15.39:0/2745943489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 0x7efee0103d70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 -- 172.21.15.39:0/2745943489 >> 172.21.15.39:0/2745943489 conn(0x7efee00fe520 msgr2=0x7efee0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 -- 172.21.15.39:0/2745943489 shutdown_connections 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 -- 172.21.15.39:0/2745943489 wait complete. 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 Processor -- start 2024-05-04T03:30:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.968+0000 7efee76f2700 1 -- start start 2024-05-04T03:30:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee76f2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 0x7efee01a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee76f2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee01042b0 0x7efee01a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee76f2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee0108200 0x7efee019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee76f2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efee0113230 con 0x7efee0103960 2024-05-04T03:30:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee76f2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efee01130b0 con 0x7efee01042b0 2024-05-04T03:30:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee76f2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efee01133b0 con 0x7efee0108200 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 0x7efee01a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 0x7efee01a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38470/0 (socket says 172.21.15.39:38470) 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 -- 172.21.15.39:0/336562593 learned_addr learned my addr 172.21.15.39:0/336562593 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee5c8f700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee0108200 0x7efee019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee4c8d700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee01042b0 0x7efee01a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 -- 172.21.15.39:0/336562593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee0108200 msgr2=0x7efee019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee0108200 0x7efee019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 -- 172.21.15.39:0/336562593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee01042b0 msgr2=0x7efee01a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee01042b0 0x7efee01a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 -- 172.21.15.39:0/336562593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee019b1e0 con 0x7efee0103960 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee4c8d700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee01042b0 0x7efee01a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.969+0000 7efee548e700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 0x7efee01a09b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7efedc00d950 tx=0x7efedc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.970+0000 7efed67fc700 1 -- 172.21.15.39:0/336562593 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc017070 con 0x7efee0103960 2024-05-04T03:30:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.970+0000 7efed67fc700 1 -- 172.21.15.39:0/336562593 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efedc009730 con 0x7efee0103960 2024-05-04T03:30:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.970+0000 7efed67fc700 1 -- 172.21.15.39:0/336562593 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc003a00 con 0x7efee0103960 2024-05-04T03:30:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.970+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee019b4d0 con 0x7efee0103960 2024-05-04T03:30:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.970+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a7760 con 0x7efee0103960 2024-05-04T03:30:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.971+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee0104b80 con 0x7efee0103960 2024-05-04T03:30:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.973+0000 7efed67fc700 1 -- 172.21.15.39:0/336562593 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7efedc003ba0 con 0x7efee0103960 2024-05-04T03:30:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.974+0000 7efed67fc700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efecc0b2880 0x7efecc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.974+0000 7efee4c8d700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efecc0b2880 0x7efecc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.974+0000 7efed67fc700 1 -- 172.21.15.39:0/336562593 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7efedc0e2570 con 0x7efee0103960 2024-05-04T03:30:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.975+0000 7efee4c8d700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efecc0b2880 0x7efecc0b4d90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efee019c270 tx=0x7efed001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:06.978+0000 7efed67fc700 1 -- 172.21.15.39:0/336562593 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efedc020030 con 0x7efee0103960 2024-05-04T03:30:07.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:06 smithi066 ceph-mon[33264]: pgmap v965: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-04T03:30:07.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.241+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efee019bfe0 con 0x7efee0103960 2024-05-04T03:30:07.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.242+0000 7efed67fc700 1 -- 172.21.15.39:0/336562593 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7efedc0aa240 con 0x7efee0103960 2024-05-04T03:30:07.245 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:30:07.245 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efecc0b2880 msgr2=0x7efecc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efecc0b2880 0x7efecc0b4d90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efee019c270 tx=0x7efed001a040 comp rx=0 tx=0).stop 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 msgr2=0x7efee01a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 0x7efee01a09b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7efedc00d950 tx=0x7efedc00dc60 comp rx=0 tx=0).stop 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 shutdown_connections 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7efecc0b2880 0x7efecc0b4d90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee0103960 0x7efee01a09b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efee0108200 0x7efee019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 --2- 172.21.15.39:0/336562593 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efee01042b0 0x7efee01a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.244+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 >> 172.21.15.39:0/336562593 conn(0x7efee00fe520 msgr2=0x7efee0102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.245+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 shutdown_connections 2024-05-04T03:30:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:07.245+0000 7efee76f2700 1 -- 172.21.15.39:0/336562593 wait complete. 2024-05-04T03:30:07.248 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:30:07.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:07 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/336562593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:30:07.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:07 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/336562593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:30:07.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:07 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/336562593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:30:07.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:30:08.217 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:30:08.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:08 smithi167 ceph-mon[33614]: pgmap v966: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-05-04T03:30:08.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:08 smithi039 ceph-mon[29715]: pgmap v966: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-05-04T03:30:09.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:08 smithi066 ceph-mon[33264]: pgmap v966: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 13 op/s 2024-05-04T03:30:09.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 -- 172.21.15.39:0/2867520824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 msgr2=0x7fe3741038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:09.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 --2- 172.21.15.39:0/2867520824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 0x7fe3741038d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00b0d0 tx=0x7fe36c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:30:09.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 -- 172.21.15.39:0/2867520824 shutdown_connections 2024-05-04T03:30:09.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 --2- 172.21.15.39:0/2867520824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 0x7fe3741038d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:09.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 --2- 172.21.15.39:0/2867520824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe374108740 0x7fe3741100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:09.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 --2- 172.21.15.39:0/2867520824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374107df0 0x7fe374108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:09.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 -- 172.21.15.39:0/2867520824 >> 172.21.15.39:0/2867520824 conn(0x7fe3740fedf0 msgr2=0x7fe374101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:09.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 -- 172.21.15.39:0/2867520824 shutdown_connections 2024-05-04T03:30:09.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.924+0000 7fe379a5c700 1 -- 172.21.15.39:0/2867520824 wait complete. 2024-05-04T03:30:09.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.925+0000 7fe379a5c700 1 Processor -- start 2024-05-04T03:30:09.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.925+0000 7fe379a5c700 1 -- start start 2024-05-04T03:30:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.925+0000 7fe379a5c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 0x7fe37419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.925+0000 7fe379a5c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374107df0 0x7fe37419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.925+0000 7fe379a5c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe374108740 0x7fe37419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.925+0000 7fe379a5c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe374113220 con 0x7fe3741034c0 2024-05-04T03:30:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.925+0000 7fe379a5c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741130a0 con 0x7fe374107df0 2024-05-04T03:30:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.925+0000 7fe379a5c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741133a0 con 0x7fe374108740 2024-05-04T03:30:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 0x7fe37419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 0x7fe37419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38488/0 (socket says 172.21.15.39:38488) 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 -- 172.21.15.39:0/1686649422 learned_addr learned my addr 172.21.15.39:0/1686649422 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe3727fc700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374107df0 0x7fe37419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe3737fe700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe374108740 0x7fe37419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 -- 172.21.15.39:0/1686649422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe374108740 msgr2=0x7fe37419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe374108740 0x7fe37419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 -- 172.21.15.39:0/1686649422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374107df0 msgr2=0x7fe37419c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374107df0 0x7fe37419c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 -- 172.21.15.39:0/1686649422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3741a6f10 con 0x7fe3741034c0 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe3737fe700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe374108740 0x7fe37419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe372ffd700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 0x7fe37419bba0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe36c007dc0 tx=0x7fe36c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe378a5a700 1 -- 172.21.15.39:0/1686649422 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c01f030 con 0x7fe3741034c0 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe378a5a700 1 -- 172.21.15.39:0/1686649422 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe36c01bb90 con 0x7fe3741034c0 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3741a71a0 con 0x7fe3741034c0 2024-05-04T03:30:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe378a5a700 1 -- 172.21.15.39:0/1686649422 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c012bb0 con 0x7fe3741034c0 2024-05-04T03:30:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.926+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3741a76a0 con 0x7fe3741034c0 2024-05-04T03:30:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.928+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3740512c0 con 0x7fe3741034c0 2024-05-04T03:30:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.929+0000 7fe378a5a700 1 -- 172.21.15.39:0/1686649422 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fe36c022030 con 0x7fe3741034c0 2024-05-04T03:30:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.930+0000 7fe378a5a700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe35c0b2900 0x7fe35c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.930+0000 7fe3727fc700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe35c0b2900 0x7fe35c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.930+0000 7fe378a5a700 1 -- 172.21.15.39:0/1686649422 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fe36c00f020 con 0x7fe3741034c0 2024-05-04T03:30:09.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.931+0000 7fe3727fc700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe35c0b2900 0x7fe35c0b4e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe368005fd0 tx=0x7fe36800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:09.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:09.935+0000 7fe378a5a700 1 -- 172.21.15.39:0/1686649422 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe36c029020 con 0x7fe3741034c0 2024-05-04T03:30:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.199+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe37402e5f0 con 0x7fe3741034c0 2024-05-04T03:30:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.200+0000 7fe378a5a700 1 -- 172.21.15.39:0/1686649422 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11930 (secure 0 0 0) 0x7fe36c0ab200 con 0x7fe3741034c0 2024-05-04T03:30:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:30:10.202 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3587796449","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3587796449},{"type":"v1","addr":"172.21.15.39:6835","nonce":3587796449}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2949312289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2949312289},{"type":"v1","addr":"172.21.15.39:6837","nonce":2949312289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:07:39.087961+0000","modified":"2024-05-04T03:14:06.227518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24472,"mds_3":24466,"mds_4":24455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14607":{"gid":14607,"name":"h","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6837/2860496376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":2860496376},{"type":"v1","addr":"172.21.15.66:6837","nonce":2860496376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6839/4175892611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":4175892611},{"type":"v1","addr":"172.21.15.39:6839","nonce":4175892611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14634":{"gid":14634,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6841/1534369514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1534369514},{"type":"v1","addr":"172.21.15.39:6841","nonce":1534369514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/1077677237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1077677237},{"type":"v1","addr":"172.21.15.167:6833","nonce":1077677237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6833/462065299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":462065299},{"type":"v1","addr":"172.21.15.66:6833","nonce":462065299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.66:6835/3463909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":3463909305},{"type":"v1","addr":"172.21.15.66:6835","nonce":3463909305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.167:6835/2699315372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2699315372},{"type":"v1","addr":"172.21.15.167:6835","nonce":2699315372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.167:6837/2270685849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270685849},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270685849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6839/975309166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":975309166},{"type":"v1","addr":"172.21.15.167:6839","nonce":975309166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.66:6839/4050747247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":4050747247},{"type":"v1","addr":"172.21.15.66:6839","nonce":4050747247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24472,24466,24455,24490]},"id":1}]} 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe35c0b2900 msgr2=0x7fe35c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe35c0b2900 0x7fe35c0b4e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe368005fd0 tx=0x7fe36800a560 comp rx=0 tx=0).stop 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 msgr2=0x7fe37419bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 0x7fe37419bba0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe36c007dc0 tx=0x7fe36c007ea0 comp rx=0 tx=0).stop 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 shutdown_connections 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fe35c0b2900 0x7fe35c0b4e10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3741034c0 0x7fe37419bba0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe374108740 0x7fe37419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 --2- 172.21.15.39:0/1686649422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe374107df0 0x7fe37419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 >> 172.21.15.39:0/1686649422 conn(0x7fe3740fedf0 msgr2=0x7fe374101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.202+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 shutdown_connections 2024-05-04T03:30:10.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:10.203+0000 7fe379a5c700 1 -- 172.21.15.39:0/1686649422 wait complete. 2024-05-04T03:30:10.206 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-05-04T03:30:10.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:10 smithi039 ceph-mon[29715]: pgmap v967: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-04T03:30:10.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:10 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1686649422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:30:10.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:30:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:10 smithi167 ceph-mon[33614]: pgmap v967: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-04T03:30:10.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:10 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1686649422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:30:11.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:10 smithi066 ceph-mon[33264]: pgmap v967: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-04T03:30:11.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:10 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1686649422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:30:11.123 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:30:11.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:11 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:30:11.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:11 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:30:12.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:11 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:30:12.814 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:12 smithi039 ceph-mon[29715]: pgmap v968: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-04T03:30:12.822 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:12 smithi066 ceph-mon[33264]: pgmap v968: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-04T03:30:12.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.811+0000 7fa04cd58700 1 -- 172.21.15.39:0/521150552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 msgr2=0x7fa048109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:12.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.811+0000 7fa04cd58700 1 --2- 172.21.15.39:0/521150552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 0x7fa048109310 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa0300099c0 tx=0x7fa030009cd0 comp rx=0 tx=0).stop 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.811+0000 7fa04cd58700 1 -- 172.21.15.39:0/521150552 shutdown_connections 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.811+0000 7fa04cd58700 1 --2- 172.21.15.39:0/521150552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048109850 0x7fa0481111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.811+0000 7fa04cd58700 1 --2- 172.21.15.39:0/521150552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0481045d0 0x7fa0481049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.811+0000 7fa04cd58700 1 --2- 172.21.15.39:0/521150552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 0x7fa048109310 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.811+0000 7fa04cd58700 1 -- 172.21.15.39:0/521150552 >> 172.21.15.39:0/521150552 conn(0x7fa0480ffea0 msgr2=0x7fa0481022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.811+0000 7fa04cd58700 1 -- 172.21.15.39:0/521150552 shutdown_connections 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.812+0000 7fa04cd58700 1 -- 172.21.15.39:0/521150552 wait complete. 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04cd58700 1 Processor -- start 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04cd58700 1 -- start start 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04cd58700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0481045d0 0x7fa04819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04cd58700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 0x7fa04819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04cd58700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048109850 0x7fa04819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04cd58700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa048114360 con 0x7fa048108f00 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04cd58700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481141e0 con 0x7fa0481045d0 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04cd58700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481144e0 con 0x7fa048109850 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa045d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 0x7fa04819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa045d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 0x7fa04819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38506/0 (socket says 172.21.15.39:38506) 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa045d9b700 1 -- 172.21.15.39:0/1018123 learned_addr learned my addr 172.21.15.39:0/1018123 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa04659c700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0481045d0 0x7fa04819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.813+0000 7fa046d9d700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048109850 0x7fa04819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa045d9b700 1 -- 172.21.15.39:0/1018123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048109850 msgr2=0x7fa04819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa045d9b700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048109850 0x7fa04819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa045d9b700 1 -- 172.21.15.39:0/1018123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0481045d0 msgr2=0x7fa04819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa045d9b700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0481045d0 0x7fa04819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa045d9b700 1 -- 172.21.15.39:0/1018123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04819fca0 con 0x7fa048108f00 2024-05-04T03:30:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa045d9b700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 0x7fa04819b370 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa03000b580 tx=0x7fa030004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa03f7fe700 1 -- 172.21.15.39:0/1018123 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030009e60 con 0x7fa048108f00 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04819ff30 con 0x7fa048108f00 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa03f7fe700 1 -- 172.21.15.39:0/1018123 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa030003c50 con 0x7fa048108f00 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa03f7fe700 1 -- 172.21.15.39:0/1018123 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030004910 con 0x7fa048108f00 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.814+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0481a03b0 con 0x7fa048108f00 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.815+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0480512c0 con 0x7fa048108f00 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.817+0000 7fa03f7fe700 1 -- 172.21.15.39:0/1018123 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fa0300037c0 con 0x7fa048108f00 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.817+0000 7fa03f7fe700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0340b2880 0x7fa0340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.818+0000 7fa04659c700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0340b2880 0x7fa0340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.818+0000 7fa03f7fe700 1 -- 172.21.15.39:0/1018123 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fa0300e2f00 con 0x7fa048108f00 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.818+0000 7fa04659c700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0340b2880 0x7fa0340b4d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa04000f540 tx=0x7fa04000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:12.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:12.821+0000 7fa03f7fe700 1 -- 172.21.15.39:0/1018123 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa030013070 con 0x7fa048108f00 2024-05-04T03:30:12.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:12 smithi167 ceph-mon[33614]: pgmap v968: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-04T03:30:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.050+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa04802e5f0 con 0x7fa048108f00 2024-05-04T03:30:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.052+0000 7fa03f7fe700 1 -- 172.21.15.39:0/1018123 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22541 (secure 0 0 0) 0x7fa0300c4e20 con 0x7fa048108f00 2024-05-04T03:30:13.054 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-05-04T03:30:13.055 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"fsid":"1e913ae0-09c2-11ef-bc97-c7b262605968","created":"2024-05-04T02:59:20.440407+0000","modified":"2024-05-04T03:08:16.237923+0000","last_up_change":"2024-05-04T03:06:30.980271+0000","last_in_change":"2024-05-04T03:06:10.921685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:02:42.983535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:07:32.727181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:07:36.009986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9940446-c2ae-4eb5-b6fe-313baf8466d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6803","nonce":837850170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6805","nonce":837850170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6809","nonce":837850170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":837850170},{"type":"v1","addr":"172.21.15.39:6807","nonce":837850170}]},"public_addr":"172.21.15.39:6803/837850170","cluster_addr":"172.21.15.39:6805/837850170","heartbeat_back_addr":"172.21.15.39:6809/837850170","heartbeat_front_addr":"172.21.15.39:6807/837850170","state":["exists","up"]},{"osd":1,"uuid":"6411bd68-dcd4-4e32-98d7-c1f89c6c6b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6801","nonce":2695940383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6803","nonce":2695940383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6807","nonce":2695940383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2695940383},{"type":"v1","addr":"172.21.15.66:6805","nonce":2695940383}]},"public_addr":"172.21.15.66:6801/2695940383","cluster_addr":"172.21.15.66:6803/2695940383","heartbeat_back_addr":"172.21.15.66:6807/2695940383","heartbeat_front_addr":"172.21.15.66:6805/2695940383","state":["exists","up"]},{"osd":2,"uuid":"339eb4b8-1e7f-490b-9d54-63886da97018","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6801","nonce":2785227057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6803","nonce":2785227057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6807","nonce":2785227057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":2785227057},{"type":"v1","addr":"172.21.15.167:6805","nonce":2785227057}]},"public_addr":"172.21.15.167:6801/2785227057","cluster_addr":"172.21.15.167:6803/2785227057","heartbeat_back_addr":"172.21.15.167:6807/2785227057","heartbeat_front_addr":"172.21.15.167:6805/2785227057","state":["exists","up"]},{"osd":3,"uuid":"118295e4-04a9-4997-bfa5-a589e4849058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6811","nonce":747919958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6813","nonce":747919958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6817","nonce":747919958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":747919958},{"type":"v1","addr":"172.21.15.39:6815","nonce":747919958}]},"public_addr":"172.21.15.39:6811/747919958","cluster_addr":"172.21.15.39:6813/747919958","heartbeat_back_addr":"172.21.15.39:6817/747919958","heartbeat_front_addr":"172.21.15.39:6815/747919958","state":["exists","up"]},{"osd":4,"uuid":"14d54715-138c-4f72-837d-ecea4f0ef3e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6809","nonce":1509400752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6811","nonce":1509400752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6815","nonce":1509400752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1509400752},{"type":"v1","addr":"172.21.15.66:6813","nonce":1509400752}]},"public_addr":"172.21.15.66:6809/1509400752","cluster_addr":"172.21.15.66:6811/1509400752","heartbeat_back_addr":"172.21.15.66:6815/1509400752","heartbeat_front_addr":"172.21.15.66:6813/1509400752","state":["exists","up"]},{"osd":5,"uuid":"3aa01070-d35f-4964-9ccd-f956e1c9e03c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6809","nonce":3200709973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6811","nonce":3200709973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6815","nonce":3200709973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3200709973},{"type":"v1","addr":"172.21.15.167:6813","nonce":3200709973}]},"public_addr":"172.21.15.167:6809/3200709973","cluster_addr":"172.21.15.167:6811/3200709973","heartbeat_back_addr":"172.21.15.167:6815/3200709973","heartbeat_front_addr":"172.21.15.167:6813/3200709973","state":["exists","up"]},{"osd":6,"uuid":"e73a0efc-8e1e-42dc-9c24-f45d08af466c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6819","nonce":519893434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6821","nonce":519893434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6825","nonce":519893434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":519893434},{"type":"v1","addr":"172.21.15.39:6823","nonce":519893434}]},"public_addr":"172.21.15.39:6819/519893434","cluster_addr":"172.21.15.39:6821/519893434","heartbeat_back_addr":"172.21.15.39:6825/519893434","heartbeat_front_addr":"172.21.15.39:6823/519893434","state":["exists","up"]},{"osd":7,"uuid":"56289675-217d-4d0e-9875-fb5e48254898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6817","nonce":931306967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6819","nonce":931306967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6823","nonce":931306967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":931306967},{"type":"v1","addr":"172.21.15.66:6821","nonce":931306967}]},"public_addr":"172.21.15.66:6817/931306967","cluster_addr":"172.21.15.66:6819/931306967","heartbeat_back_addr":"172.21.15.66:6823/931306967","heartbeat_front_addr":"172.21.15.66:6821/931306967","state":["exists","up"]},{"osd":8,"uuid":"755cbc15-ebe2-408f-8009-518e6f31b975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6817","nonce":948600298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6819","nonce":948600298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6823","nonce":948600298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":948600298},{"type":"v1","addr":"172.21.15.167:6821","nonce":948600298}]},"public_addr":"172.21.15.167:6817/948600298","cluster_addr":"172.21.15.167:6819/948600298","heartbeat_back_addr":"172.21.15.167:6823/948600298","heartbeat_front_addr":"172.21.15.167:6821/948600298","state":["exists","up"]},{"osd":9,"uuid":"956d274d-166b-4474-92d3-d31bda097f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6827","nonce":1483154283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6829","nonce":1483154283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6833","nonce":1483154283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1483154283},{"type":"v1","addr":"172.21.15.39:6831","nonce":1483154283}]},"public_addr":"172.21.15.39:6827/1483154283","cluster_addr":"172.21.15.39:6829/1483154283","heartbeat_back_addr":"172.21.15.39:6833/1483154283","heartbeat_front_addr":"172.21.15.39:6831/1483154283","state":["exists","up"]},{"osd":10,"uuid":"2c1ff6cd-7475-4b79-83bb-1f671fe132cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6825","nonce":3028550986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6827","nonce":3028550986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6831","nonce":3028550986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3028550986},{"type":"v1","addr":"172.21.15.66:6829","nonce":3028550986}]},"public_addr":"172.21.15.66:6825/3028550986","cluster_addr":"172.21.15.66:6827/3028550986","heartbeat_back_addr":"172.21.15.66:6831/3028550986","heartbeat_front_addr":"172.21.15.66:6829/3028550986","state":["exists","up"]},{"osd":11,"uuid":"d8940073-e9c7-4d49-af48-0a4c9a4b6f6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6825","nonce":3994242029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6827","nonce":3994242029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6831","nonce":3994242029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3994242029},{"type":"v1","addr":"172.21.15.167:6829","nonce":3994242029}]},"public_addr":"172.21.15.167:6825/3994242029","cluster_addr":"172.21.15.167:6827/3994242029","heartbeat_back_addr":"172.21.15.167:6831/3994242029","heartbeat_front_addr":"172.21.15.167:6829/3994242029","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:01:51.840672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:16.238721+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:02:39.616199+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:04.641411+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:29.872530+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:03:54.412763+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:19.862154+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:04:45.941035+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:10.179329+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:05:36.757484+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:03.263958+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:06:29.231699+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:6801/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6800/3124453164":"2024-05-05T02:59:47.622763+0000","172.21.15.39:6801/3594679658":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/163870032":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/518372945":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1251709695":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/402196413":"2024-05-05T03:00:14.875322+0000","172.21.15.39:0/1831352319":"2024-05-05T02:59:47.622763+0000","172.21.15.39:0/1379489577":"2024-05-05T03:00:14.875322+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0340b2880 msgr2=0x7fa0340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0340b2880 0x7fa0340b4d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa04000f540 tx=0x7fa04000b040 comp rx=0 tx=0).stop 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 msgr2=0x7fa04819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 0x7fa04819b370 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa03000b580 tx=0x7fa030004270 comp rx=0 tx=0).stop 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 shutdown_connections 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fa0340b2880 0x7fa0340b4d90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa048109850 0x7fa04819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0481045d0 0x7fa04819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 --2- 172.21.15.39:0/1018123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa048108f00 0x7fa04819b370 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.054+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 >> 172.21.15.39:0/1018123 conn(0x7fa0480ffea0 msgr2=0x7fa04810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.055+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 shutdown_connections 2024-05-04T03:30:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:13.055+0000 7fa04cd58700 1 -- 172.21.15.39:0/1018123 wait complete. 2024-05-04T03:30:13.918 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-04T03:30:14.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:13 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1018123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:30:14.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:13 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:13 smithi039 ceph-mon[29715]: pgmap v969: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-04T03:30:14.319 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.319 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.319 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:13 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1018123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:30:14.319 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.319 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:13 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.319 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:13 smithi066 ceph-mon[33264]: pgmap v969: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-04T03:30:14.409 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:30:14.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:13 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1018123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:30:14.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:13 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:14.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:13 smithi167 ceph-mon[33614]: pgmap v969: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 -- 172.21.15.39:0/244358850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864108740 msgr2=0x7fb8641100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 --2- 172.21.15.39:0/244358850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864108740 0x7fb8641100a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00b0d0 tx=0x7fb85c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 -- 172.21.15.39:0/244358850 shutdown_connections 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 --2- 172.21.15.39:0/244358850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864108740 0x7fb8641100a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 --2- 172.21.15.39:0/244358850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8641034c0 0x7fb8641038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 --2- 172.21.15.39:0/244358850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb864107df0 0x7fb864108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 -- 172.21.15.39:0/244358850 >> 172.21.15.39:0/244358850 conn(0x7fb8640fedc0 msgr2=0x7fb8641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 -- 172.21.15.39:0/244358850 shutdown_connections 2024-05-04T03:30:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.135+0000 7fb86a41a700 1 -- 172.21.15.39:0/244358850 wait complete. 2024-05-04T03:30:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.136+0000 7fb86a41a700 1 Processor -- start 2024-05-04T03:30:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.136+0000 7fb86a41a700 1 -- start start 2024-05-04T03:30:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.136+0000 7fb86a41a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8641034c0 0x7fb864199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.136+0000 7fb86a41a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864107df0 0x7fb864199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.136+0000 7fb86a41a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb864108740 0x7fb86419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.136+0000 7fb86a41a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb864113220 con 0x7fb864107df0 2024-05-04T03:30:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.136+0000 7fb86a41a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641130a0 con 0x7fb8641034c0 2024-05-04T03:30:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.136+0000 7fb86a41a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641133a0 con 0x7fb864108740 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864107df0 0x7fb864199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb863fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8641034c0 0x7fb864199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864107df0 0x7fb864199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43534/0 (socket says 172.21.15.39:43534) 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb863fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8641034c0 0x7fb864199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.39:43292/0 (socket says 172.21.15.39:43292) 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8637fe700 1 -- 172.21.15.39:0/2672609695 learned_addr learned my addr 172.21.15.39:0/2672609695 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8689b7700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb864108740 0x7fb86419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8637fe700 1 -- 172.21.15.39:0/2672609695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb864108740 msgr2=0x7fb86419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8637fe700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb864108740 0x7fb86419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8637fe700 1 -- 172.21.15.39:0/2672609695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8641034c0 msgr2=0x7fb864199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8637fe700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8641034c0 0x7fb864199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.137+0000 7fb8637fe700 1 -- 172.21.15.39:0/2672609695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8641a8f90 con 0x7fb864107df0 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.138+0000 7fb8637fe700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864107df0 0x7fb864199e50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb85800ca10 tx=0x7fb85800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.138+0000 7fb8617fa700 1 -- 172.21.15.39:0/2672609695 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858007a60 con 0x7fb864107df0 2024-05-04T03:30:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.138+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8641a9280 con 0x7fb864107df0 2024-05-04T03:30:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.138+0000 7fb8617fa700 1 -- 172.21.15.39:0/2672609695 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb858007c00 con 0x7fb864107df0 2024-05-04T03:30:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.138+0000 7fb8617fa700 1 -- 172.21.15.39:0/2672609695 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858018400 con 0x7fb864107df0 2024-05-04T03:30:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.138+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8641a97c0 con 0x7fb864107df0 2024-05-04T03:30:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.140+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb864104970 con 0x7fb864107df0 2024-05-04T03:30:16.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.141+0000 7fb8617fa700 1 -- 172.21.15.39:0/2672609695 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7fb85800f450 con 0x7fb864107df0 2024-05-04T03:30:16.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.142+0000 7fb8617fa700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb84c0b2880 0x7fb84c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:16.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.142+0000 7fb8617fa700 1 -- 172.21.15.39:0/2672609695 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8219+0+0 (secure 0 0 0) 0x7fb8580e2090 con 0x7fb864107df0 2024-05-04T03:30:16.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.142+0000 7fb863fff700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb84c0b2880 0x7fb84c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:16.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.143+0000 7fb863fff700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb84c0b2880 0x7fb84c0b4d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb854005950 tx=0x7fb85400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:16.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.151+0000 7fb8617fa700 1 -- 172.21.15.39:0/2672609695 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb858014030 con 0x7fb864107df0 2024-05-04T03:30:16.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.432+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb86402e5f0 con 0x7fb864107df0 2024-05-04T03:30:16.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.435+0000 7fb8617fa700 1 -- 172.21.15.39:0/2672609695 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7fb8580a9dc0 con 0x7fb864107df0 2024-05-04T03:30:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.441+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb84c0b2880 msgr2=0x7fb84c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.441+0000 7fb86a41a700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb84c0b2880 0x7fb84c0b4d90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb854005950 tx=0x7fb85400b410 comp rx=0 tx=0).stop 2024-05-04T03:30:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.441+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864107df0 msgr2=0x7fb864199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:16.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.441+0000 7fb86a41a700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864107df0 0x7fb864199e50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb85800ca10 tx=0x7fb85800cd20 comp rx=0 tx=0).stop 2024-05-04T03:30:16.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.443+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 shutdown_connections 2024-05-04T03:30:16.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.443+0000 7fb86a41a700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7fb84c0b2880 0x7fb84c0b4d90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.443+0000 7fb86a41a700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb864108740 0x7fb86419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.443+0000 7fb86a41a700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb8641034c0 0x7fb864199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.443+0000 7fb86a41a700 1 --2- 172.21.15.39:0/2672609695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb864107df0 0x7fb864199e50 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.443+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 >> 172.21.15.39:0/2672609695 conn(0x7fb8640fedc0 msgr2=0x7fb8641096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.443+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 shutdown_connections 2024-05-04T03:30:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:16.443+0000 7fb86a41a700 1 -- 172.21.15.39:0/2672609695 wait complete. 2024-05-04T03:30:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-04T03:30:16.692 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a[70069]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.693 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a[70069]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.693 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mds[70073]: -- 172.21.15.39:0/2026230169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba76b2da40 con 0x55ba76b75600 2024-05-04T03:30:16.693 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a[70069]: starting mds.a at 2024-05-04T03:30:16.693 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:30:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g[69599]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.693 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:30:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g[69599]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.693 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mds[69603]: -- 172.21.15.39:0/2260797944 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56186ee4da40 con 0x56186ee95180 2024-05-04T03:30:16.693 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:30:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g[69599]: starting mds.g at 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a[29711]: 2024-05-04T03:30:16.433+0000 7f03e5994700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: pgmap v970: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2672609695' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/2672609695' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: osdmap e81: 12 total, 12 up, 12 in 2024-05-04T03:30:16.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:16 smithi039 ceph-mon[29715]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-04T03:30:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: pgmap v970: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-04T03:30:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2672609695' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:30:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-04T03:30:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T03:30:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/2672609695' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-04T03:30:16.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: osdmap e81: 12 total, 12 up, 12 in 2024-05-04T03:30:16.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mon[33264]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-04T03:30:16.820 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e[63231]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.820 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e[63231]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.820 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mds[63235]: -- 172.21.15.66:0/343702741 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555af0ea9a40 con 0x555af0ef1a80 2024-05-04T03:30:16.820 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e[63231]: starting mds.e at 2024-05-04T03:30:16.820 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b[63850]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.820 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b[63850]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.821 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mds[63854]: -- 172.21.15.66:0/732951930 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56059a12da40 con 0x56059a175180 2024-05-04T03:30:16.821 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b[63850]: starting mds.b at 2024-05-04T03:30:16.821 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k[62910]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.821 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k[62910]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.821 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mds[62914]: -- 172.21.15.66:0/892888273 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad9aa03a40 con 0x55ad9aa4b180 2024-05-04T03:30:16.821 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k[62910]: starting mds.k at 2024-05-04T03:30:16.822 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h[63541]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.822 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h[63541]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.822 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:30:16 smithi066 ceph-mds[63545]: -- 172.21.15.66:0/1334802859 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffcd5c7a40 con 0x55ffcd60f600 2024-05-04T03:30:16.822 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:30:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h[63541]: starting mds.h at 2024-05-04T03:30:16.928 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f[64463]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.928 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f[64463]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.929 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mds[64467]: -- 172.21.15.167:0/2586637504 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4e5a2ba40 con 0x55d4e5a73600 2024-05-04T03:30:16.929 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f[64463]: starting mds.f at 2024-05-04T03:30:16.929 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c[64779]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.929 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c[64779]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.929 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mds[64783]: -- 172.21.15.167:0/2220608657 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620eb6dfa40 con 0x5620eb727180 2024-05-04T03:30:16.929 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c[64779]: starting mds.c at 2024-05-04T03:30:16.929 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i[65089]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i[65089]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mds[65093]: -- 172.21.15.167:0/2145469636 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56456e591a40 con 0x56456e5d9600 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i[65089]: starting mds.i at 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l[64143]: ignoring --setuser ceph since I am not root 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l[64143]: ignoring --setgroup ceph since I am not root 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mds[64147]: -- 172.21.15.167:0/2828451778 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e64e8ba40 con 0x561e64ed3a80 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:30:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l[64143]: starting mds.l at 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: pgmap v970: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-04T03:30:16.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:16.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:16.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2672609695' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:30:16.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-04T03:30:16.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T03:30:16.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/2672609695' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-04T03:30:16.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: osdmap e81: 12 total, 12 up, 12 in 2024-05-04T03:30:16.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:16 smithi167 ceph-mon[33614]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-04T03:30:17.327 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-04T03:30:17.597 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:30:18.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: pgmap v972: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6832/167599548,v1:172.21.15.167:6833/167599548] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6834/4275567249,v1:172.21.15.167:6835/4275567249] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6836/2016743649,v1:172.21.15.167:6837/2016743649] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6832/1806915115,v1:172.21.15.66:6833/1806915115] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6834/2049355414,v1:172.21.15.66:6835/2049355414] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.167:6838/2846406291,v1:172.21.15.167:6839/2846406291] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6836/2324693985,v1:172.21.15.66:6837/2324693985] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.66:6838/2568949642,v1:172.21.15.66:6839/2568949642] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.39:6838/809350271,v1:172.21.15.39:6839/809350271] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: mds.? [v2:172.21.15.39:6840/2969493238,v1:172.21.15.39:6841/2969493238] up:boot 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-04T03:30:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:30:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:18 smithi039 ceph-mon[29715]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:18.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: pgmap v972: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-04T03:30:18.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6832/167599548,v1:172.21.15.167:6833/167599548] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6834/4275567249,v1:172.21.15.167:6835/4275567249] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6836/2016743649,v1:172.21.15.167:6837/2016743649] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6832/1806915115,v1:172.21.15.66:6833/1806915115] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6834/2049355414,v1:172.21.15.66:6835/2049355414] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.167:6838/2846406291,v1:172.21.15.167:6839/2846406291] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6836/2324693985,v1:172.21.15.66:6837/2324693985] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.66:6838/2568949642,v1:172.21.15.66:6839/2568949642] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.39:6838/809350271,v1:172.21.15.39:6839/809350271] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: mds.? [v2:172.21.15.39:6840/2969493238,v1:172.21.15.39:6841/2969493238] up:boot 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:30:18.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:30:18.821 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:18 smithi066 ceph-mon[33264]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:18.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: pgmap v972: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6832/167599548,v1:172.21.15.167:6833/167599548] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6834/4275567249,v1:172.21.15.167:6835/4275567249] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6836/2016743649,v1:172.21.15.167:6837/2016743649] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6832/1806915115,v1:172.21.15.66:6833/1806915115] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6834/2049355414,v1:172.21.15.66:6835/2049355414] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.167:6838/2846406291,v1:172.21.15.167:6839/2846406291] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6836/2324693985,v1:172.21.15.66:6837/2324693985] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.66:6838/2568949642,v1:172.21.15.66:6839/2568949642] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.39:6838/809350271,v1:172.21.15.39:6839/809350271] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: mds.? [v2:172.21.15.39:6840/2969493238,v1:172.21.15.39:6841/2969493238] up:boot 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:30:18.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:30:18.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:18 smithi167 ceph-mon[33614]: from='mgr.14152 172.21.15.39:0/4277355805' entity='mgr.x' 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 -- 172.21.15.39:0/2705710878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 msgr2=0x7f33c0104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 --2- 172.21.15.39:0/2705710878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 0x7f33c0104050 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f33b00099c0 tx=0x7f33b0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 -- 172.21.15.39:0/2705710878 shutdown_connections 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 --2- 172.21.15.39:0/2705710878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 0x7f33c0104050 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 --2- 172.21.15.39:0/2705710878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c0108790 0x7f33c01100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 --2- 172.21.15.39:0/2705710878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c0107e40 0x7f33c0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 -- 172.21.15.39:0/2705710878 >> 172.21.15.39:0/2705710878 conn(0x7f33c00fe6f0 msgr2=0x7f33c0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 -- 172.21.15.39:0/2705710878 shutdown_connections 2024-05-04T03:30:19.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.514+0000 7f33c612a700 1 -- 172.21.15.39:0/2705710878 wait complete. 2024-05-04T03:30:19.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33c612a700 1 Processor -- start 2024-05-04T03:30:19.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33c612a700 1 -- start start 2024-05-04T03:30:19.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33c612a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 0x7f33c019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:19.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33c612a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c0107e40 0x7f33c019c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:19.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33c612a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c0108790 0x7f33c019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:19.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33c612a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01132e0 con 0x7f33c0103c40 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33c612a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0113160 con 0x7f33c0107e40 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33c612a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0113460 con 0x7f33c0108790 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33bf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 0x7f33c019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33bf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 0x7f33c019bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43610/0 (socket says 172.21.15.39:43610) 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33bf7fe700 1 -- 172.21.15.39:0/915123747 learned_addr learned my addr 172.21.15.39:0/915123747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33beffd700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c0107e40 0x7f33c019c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.515+0000 7f33bffff700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c0108790 0x7f33c019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bf7fe700 1 -- 172.21.15.39:0/915123747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c0108790 msgr2=0x7f33c019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bf7fe700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c0108790 0x7f33c019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bf7fe700 1 -- 172.21.15.39:0/915123747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c0107e40 msgr2=0x7f33c019c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bf7fe700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c0107e40 0x7f33c019c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bf7fe700 1 -- 172.21.15.39:0/915123747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c01a7010 con 0x7f33c0103c40 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bf7fe700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 0x7f33c019bcc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f33b000b580 tx=0x7f33b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bcff9700 1 -- 172.21.15.39:0/915123747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0027070 con 0x7f33c0103c40 2024-05-04T03:30:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c01a72a0 con 0x7f33c0103c40 2024-05-04T03:30:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bcff9700 1 -- 172.21.15.39:0/915123747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33b0022470 con 0x7f33c0103c40 2024-05-04T03:30:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33bcff9700 1 -- 172.21.15.39:0/915123747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0017650 con 0x7f33c0103c40 2024-05-04T03:30:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.516+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c01a7750 con 0x7f33c0103c40 2024-05-04T03:30:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.517+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c00512c0 con 0x7f33c0103c40 2024-05-04T03:30:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.519+0000 7f33bcff9700 1 -- 172.21.15.39:0/915123747 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f33b002a070 con 0x7f33c0103c40 2024-05-04T03:30:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.519+0000 7f33bcff9700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f33a80b2880 0x7f33a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.520+0000 7f33beffd700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f33a80b2880 0x7f33a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.520+0000 7f33bcff9700 1 -- 172.21.15.39:0/915123747 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8649+0+0 (secure 0 0 0) 0x7f33b00e3960 con 0x7f33c0103c40 2024-05-04T03:30:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.520+0000 7f33beffd700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f33a80b2880 0x7f33a80b4d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f33c01069a0 tx=0x7f33b4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:19.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.523+0000 7f33bcff9700 1 -- 172.21.15.39:0/915123747 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33b0031020 con 0x7f33c0103c40 2024-05-04T03:30:19.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:19.791+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f33c002e5f0 con 0x7f33c0103c40 2024-05-04T03:30:20.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.497+0000 7f33bcff9700 1 -- 172.21.15.39:0/915123747 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f33b0013070 con 0x7f33c0103c40 2024-05-04T03:30:20.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.500+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f33a80b2880 msgr2=0x7f33a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:20.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.500+0000 7f33c612a700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f33a80b2880 0x7f33a80b4d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f33c01069a0 tx=0x7f33b4009190 comp rx=0 tx=0).stop 2024-05-04T03:30:20.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.500+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 msgr2=0x7f33c019bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:20.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.500+0000 7f33c612a700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 0x7f33c019bcc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f33b000b580 tx=0x7f33b000f690 comp rx=0 tx=0).stop 2024-05-04T03:30:20.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.501+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 shutdown_connections 2024-05-04T03:30:20.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.501+0000 7f33c612a700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f33a80b2880 0x7f33a80b4d90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:20.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.501+0000 7f33c612a700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c0103c40 0x7f33c019bcc0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:20.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.501+0000 7f33c612a700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33c0108790 0x7f33c019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:20.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.501+0000 7f33c612a700 1 --2- 172.21.15.39:0/915123747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33c0107e40 0x7f33c019c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:20.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.501+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 >> 172.21.15.39:0/915123747 conn(0x7f33c00fe6f0 msgr2=0x7f33c0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:20.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.501+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 shutdown_connections 2024-05-04T03:30:20.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:20.501+0000 7f33c612a700 1 -- 172.21.15.39:0/915123747 wait complete. 2024-05-04T03:30:20.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:20 smithi066 ceph-mon[33264]: pgmap v973: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-04T03:30:20.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:20 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/915123747' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:30:20.892 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:20 smithi039 ceph-mon[29715]: pgmap v973: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-04T03:30:20.892 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:20 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/915123747' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:30:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:20 smithi167 ceph-mon[33614]: pgmap v973: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-04T03:30:20.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:20 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/915123747' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:30:21.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-04T03:30:21.514 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:30:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:21 smithi039 ceph-mon[29715]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-04T03:30:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:21 smithi039 ceph-mon[29715]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-04T03:30:21.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:21 smithi039 ceph-mon[29715]: Cluster is now healthy 2024-05-04T03:30:21.778 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:21 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/915123747' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:30:21.778 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:21 smithi039 ceph-mon[29715]: fsmap 12 up:standby 2024-05-04T03:30:21.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:21 smithi066 ceph-mon[33264]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-04T03:30:21.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:21 smithi066 ceph-mon[33264]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-04T03:30:21.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:21 smithi066 ceph-mon[33264]: Cluster is now healthy 2024-05-04T03:30:21.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:21 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/915123747' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:30:21.820 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:21 smithi066 ceph-mon[33264]: fsmap 12 up:standby 2024-05-04T03:30:21.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:21 smithi167 ceph-mon[33614]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-04T03:30:21.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:21 smithi167 ceph-mon[33614]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-04T03:30:21.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:21 smithi167 ceph-mon[33614]: Cluster is now healthy 2024-05-04T03:30:21.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:21 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/915123747' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:30:21.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:21 smithi167 ceph-mon[33614]: fsmap 12 up:standby 2024-05-04T03:30:22.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:22 smithi039 ceph-mon[29715]: pgmap v974: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-04T03:30:22.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:22 smithi066 ceph-mon[33264]: pgmap v974: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-04T03:30:22.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:22 smithi167 ceph-mon[33614]: pgmap v974: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-05-04T03:30:23.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 -- 172.21.15.39:0/920391292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0780109860 msgr2=0x7f07801111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:23.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 --2- 172.21.15.39:0/920391292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0780109860 0x7f07801111c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f077800b0d0 tx=0x7f077800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 -- 172.21.15.39:0/920391292 shutdown_connections 2024-05-04T03:30:23.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 --2- 172.21.15.39:0/920391292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0780109860 0x7f07801111c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 --2- 172.21.15.39:0/920391292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07801045e0 0x7f07801049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 --2- 172.21.15.39:0/920391292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780108f10 0x7f0780109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 -- 172.21.15.39:0/920391292 >> 172.21.15.39:0/920391292 conn(0x7f07800ffe70 msgr2=0x7f07801022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 -- 172.21.15.39:0/920391292 shutdown_connections 2024-05-04T03:30:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.303+0000 7f0784def700 1 -- 172.21.15.39:0/920391292 wait complete. 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.304+0000 7f0784def700 1 Processor -- start 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.304+0000 7f0784def700 1 -- start start 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.304+0000 7f0784def700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07801045e0 0x7f078019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.304+0000 7f0784def700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0780108f10 0x7f078019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.304+0000 7f0784def700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780109860 0x7f078019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.304+0000 7f0784def700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f07801143b0 con 0x7f07801045e0 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.304+0000 7f0784def700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0780114230 con 0x7f0780109860 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.304+0000 7f0784def700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0780114530 con 0x7f0780108f10 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07801045e0 0x7f078019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07801045e0 0x7f078019ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43640/0 (socket says 172.21.15.39:43640) 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 -- 172.21.15.39:0/4255362523 learned_addr learned my addr 172.21.15.39:0/4255362523 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:30:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077dd9b700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0780108f10 0x7f078019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 -- 172.21.15.39:0/4255362523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0780108f10 msgr2=0x7f078019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077ed9d700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780109860 0x7f078019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0780108f10 0x7f078019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 -- 172.21.15.39:0/4255362523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780109860 msgr2=0x7f078019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780109860 0x7f078019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 -- 172.21.15.39:0/4255362523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078019fbf0 con 0x7f07801045e0 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077ed9d700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780109860 0x7f078019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f077e59c700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07801045e0 0x7f078019ad20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f076800d950 tx=0x7f076800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f07777fe700 1 -- 172.21.15.39:0/4255362523 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768016070 con 0x7f07801045e0 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.305+0000 7f07777fe700 1 -- 172.21.15.39:0/4255362523 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0768010460 con 0x7f07801045e0 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.306+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078019fee0 con 0x7f07801045e0 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.306+0000 7f07777fe700 1 -- 172.21.15.39:0/4255362523 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076800f620 con 0x7f07801045e0 2024-05-04T03:30:23.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.306+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07801a7800 con 0x7f07801045e0 2024-05-04T03:30:23.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.307+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07800512c0 con 0x7f07801045e0 2024-05-04T03:30:23.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.308+0000 7f07777fe700 1 -- 172.21.15.39:0/4255362523 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f0768019070 con 0x7f07801045e0 2024-05-04T03:30:23.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.309+0000 7f07777fe700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f076c0b2880 0x7f076c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:23.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.309+0000 7f077dd9b700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f076c0b2880 0x7f076c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:23.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.309+0000 7f07777fe700 1 -- 172.21.15.39:0/4255362523 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8649+0+0 (secure 0 0 0) 0x7f07680e2300 con 0x7f07801045e0 2024-05-04T03:30:23.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.309+0000 7f077dd9b700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f076c0b2880 0x7f076c0b4d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0770006fd0 tx=0x7f0770008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:23.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.314+0000 7f07777fe700 1 -- 172.21.15.39:0/4255362523 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07680a9e80 con 0x7f07801045e0 2024-05-04T03:30:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.526+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f078002e5f0 con 0x7f07801045e0 2024-05-04T03:30:23.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.578+0000 7f07777fe700 1 -- 172.21.15.39:0/4255362523 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f07680a9780 con 0x7f07801045e0 2024-05-04T03:30:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f076c0b2880 msgr2=0x7f076c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f076c0b2880 0x7f076c0b4d90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0770006fd0 tx=0x7f0770008040 comp rx=0 tx=0).stop 2024-05-04T03:30:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07801045e0 msgr2=0x7f078019ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07801045e0 0x7f078019ad20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f076800d950 tx=0x7f076800dd10 comp rx=0 tx=0).stop 2024-05-04T03:30:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 shutdown_connections 2024-05-04T03:30:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f076c0b2880 0x7f076c0b4d90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07801045e0 0x7f078019ad20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0780109860 0x7f078019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 --2- 172.21.15.39:0/4255362523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0780108f10 0x7f078019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.582+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 >> 172.21.15.39:0/4255362523 conn(0x7f07800ffe70 msgr2=0x7f0780108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.583+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 shutdown_connections 2024-05-04T03:30:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:23.583+0000 7f0784def700 1 -- 172.21.15.39:0/4255362523 wait complete. 2024-05-04T03:30:23.587 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-05-04T03:30:23.915 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:23 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4255362523' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:30:23.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:23 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4255362523' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:30:24.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:23 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4255362523' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:30:24.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-04T03:30:24.498 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:30:24.881 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:24 smithi039 ceph-mon[29715]: pgmap v975: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-04T03:30:24.881 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:24 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/4255362523' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:30:24.882 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:24 smithi039 ceph-mon[29715]: osdmap e82: 12 total, 12 up, 12 in 2024-05-04T03:30:24.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:24 smithi167 ceph-mon[33614]: pgmap v975: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-04T03:30:24.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:24 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/4255362523' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:30:24.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:24 smithi167 ceph-mon[33614]: osdmap e82: 12 total, 12 up, 12 in 2024-05-04T03:30:25.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:24 smithi066 ceph-mon[33264]: pgmap v975: 97 pgs: 97 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-04T03:30:25.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:24 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/4255362523' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:30:25.069 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:24 smithi066 ceph-mon[33264]: osdmap e82: 12 total, 12 up, 12 in 2024-05-04T03:30:26.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.225+0000 7f7596b52700 1 -- 172.21.15.39:0/3786464633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590107e10 msgr2=0x7f7590108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.225+0000 7f7596b52700 1 --2- 172.21.15.39:0/3786464633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590107e10 0x7f7590108220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f75780099c0 tx=0x7f7578009cd0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.226+0000 7f7596b52700 1 -- 172.21.15.39:0/3786464633 shutdown_connections 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.226+0000 7f7596b52700 1 --2- 172.21.15.39:0/3786464633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7590108760 0x7f75901100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.226+0000 7f7596b52700 1 --2- 172.21.15.39:0/3786464633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75901034e0 0x7f75901038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.226+0000 7f7596b52700 1 --2- 172.21.15.39:0/3786464633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590107e10 0x7f7590108220 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.226+0000 7f7596b52700 1 -- 172.21.15.39:0/3786464633 >> 172.21.15.39:0/3786464633 conn(0x7f75900fef30 msgr2=0x7f7590101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.226+0000 7f7596b52700 1 -- 172.21.15.39:0/3786464633 shutdown_connections 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.226+0000 7f7596b52700 1 -- 172.21.15.39:0/3786464633 wait complete. 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.226+0000 7f7596b52700 1 Processor -- start 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f7596b52700 1 -- start start 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f7596b52700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75901034e0 0x7f759019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f7596b52700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7590107e10 0x7f759019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:26.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f7596b52700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590108760 0x7f75901a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f7596b52700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7590114390 con 0x7f7590108760 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f7596b52700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7590114210 con 0x7f7590107e10 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f7596b52700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7590114510 con 0x7f75901034e0 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75950ef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590108760 0x7f75901a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75950ef700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590108760 0x7f75901a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49438/0 (socket says 172.21.15.39:49438) 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75950ef700 1 -- 172.21.15.39:0/1339067466 learned_addr learned my addr 172.21.15.39:0/1339067466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75948ee700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75901034e0 0x7f759019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f758ffff700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7590107e10 0x7f759019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75950ef700 1 -- 172.21.15.39:0/1339067466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75901034e0 msgr2=0x7f759019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75950ef700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75901034e0 0x7f759019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75950ef700 1 -- 172.21.15.39:0/1339067466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7590107e10 msgr2=0x7f759019d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75950ef700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7590107e10 0x7f759019d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.227+0000 7f75950ef700 1 -- 172.21.15.39:0/1339067466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75901a1de0 con 0x7f7590108760 2024-05-04T03:30:26.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.228+0000 7f75950ef700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590108760 0x7f75901a17f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f758000c900 tx=0x7f758000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:26.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.228+0000 7f758dffb700 1 -- 172.21.15.39:0/1339067466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75800078d0 con 0x7f7590108760 2024-05-04T03:30:26.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.228+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75901a20d0 con 0x7f7590108760 2024-05-04T03:30:26.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.228+0000 7f758dffb700 1 -- 172.21.15.39:0/1339067466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f758000f450 con 0x7f7590108760 2024-05-04T03:30:26.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.228+0000 7f758dffb700 1 -- 172.21.15.39:0/1339067466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580018660 con 0x7f7590108760 2024-05-04T03:30:26.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.228+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75901a25e0 con 0x7f7590108760 2024-05-04T03:30:26.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.229+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75900512c0 con 0x7f7590108760 2024-05-04T03:30:26.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.231+0000 7f758dffb700 1 -- 172.21.15.39:0/1339067466 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149652+0+0 (secure 0 0 0) 0x7f7580007a70 con 0x7f7590108760 2024-05-04T03:30:26.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.232+0000 7f758dffb700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f757c0b2880 0x7f757c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:30:26.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.232+0000 7f75948ee700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f757c0b2880 0x7f757c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:30:26.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.232+0000 7f758dffb700 1 -- 172.21.15.39:0/1339067466 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8095+0+0 (secure 0 0 0) 0x7f75800e1f10 con 0x7f7590108760 2024-05-04T03:30:26.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.232+0000 7f75948ee700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f757c0b2880 0x7f757c0b4d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7588009550 tx=0x7f75880091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:30:26.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.237+0000 7f758dffb700 1 -- 172.21.15.39:0/1339067466 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7580014030 con 0x7f7590108760 2024-05-04T03:30:26.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.454+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75901a9b60 con 0x7f7590108760 2024-05-04T03:30:26.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.594+0000 7f758dffb700 1 -- 172.21.15.39:0/1339067466 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7f75800a9d10 con 0x7f7590108760 2024-05-04T03:30:26.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f757c0b2880 msgr2=0x7f757c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:26.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f757c0b2880 0x7f757c0b4d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7588009550 tx=0x7f75880091b0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590108760 msgr2=0x7f75901a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590108760 0x7f75901a17f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f758000c900 tx=0x7f758000cc10 comp rx=0 tx=0).stop 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 shutdown_connections 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:6800/2627629846,v1:172.21.15.39:6801/2627629846] conn(0x7f757c0b2880 0x7f757c0b4d90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7590108760 0x7f75901a17f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75901034e0 0x7f759019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 --2- 172.21.15.39:0/1339067466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7590107e10 0x7f759019d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 >> 172.21.15.39:0/1339067466 conn(0x7f75900fef30 msgr2=0x7f75901063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 shutdown_connections 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-05-04T03:30:26.597+0000 7f7596b52700 1 -- 172.21.15.39:0/1339067466 wait complete. 2024-05-04T03:30:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-05-04T03:30:26.871 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:26 smithi066 ceph-mon[33264]: pgmap v977: 65 pgs: 65 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:30:26.871 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:26 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1339067466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:30:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:26 smithi167 ceph-mon[33614]: pgmap v977: 65 pgs: 65 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:30:26.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:26 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1339067466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:30:26.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:26 smithi039 ceph-mon[29715]: pgmap v977: 65 pgs: 65 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:30:26.942 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:26 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1339067466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:30:27.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:30:27.358 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:30:27.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:30:27.444 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-04T03:30:27.444 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-04T03:30:27.467 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-04T03:30:27.495 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-04T03:30:27.510 INFO:tasks.cephadm:Stopping all daemons... 2024-05-04T03:30:27.510 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-04T03:30:27.510 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a 2024-05-04T03:30:27.758 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:27 smithi066 ceph-mon[33264]: from='client.? 172.21.15.39:0/1339067466' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:30:27.759 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:27 smithi066 ceph-mon[33264]: osdmap e83: 12 total, 12 up, 12 in 2024-05-04T03:30:27.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:27 smithi039 systemd[1]: Stopping Ceph mon.a for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:27.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:27 smithi039 ceph-mon[29715]: from='client.? 172.21.15.39:0/1339067466' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:30:27.835 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:27 smithi039 ceph-mon[29715]: osdmap e83: 12 total, 12 up, 12 in 2024-05-04T03:30:27.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:27 smithi167 ceph-mon[33614]: from='client.? 172.21.15.39:0/1339067466' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:30:27.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:27 smithi167 ceph-mon[33614]: osdmap e83: 12 total, 12 up, 12 in 2024-05-04T03:30:28.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a[29711]: 2024-05-04T03:30:27.831+0000 7f03eb19f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:30:28.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a[29711]: 2024-05-04T03:30:27.831+0000 7f03eb19f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-04T03:30:28.192 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:28 smithi039 podman[118571]: 2024-05-04 03:30:28.046358727 +0000 UTC m=+0.326247168 container died 8812331cba60dbf671af7e283147cc4e35070fb6cfbcd8e4a6c644b432482cfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-04T03:30:28.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:28 smithi039 podman[118571]: 2024-05-04 03:30:28.683009644 +0000 UTC m=+0.962898093 container cleanup 8812331cba60dbf671af7e283147cc4e35070fb6cfbcd8e4a6c644b432482cfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-04T03:30:28.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:28 smithi039 bash[118571]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a 2024-05-04T03:30:29.168 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a.service' 2024-05-04T03:30:29.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:28 smithi039 podman[118650]: 2024-05-04 03:30:28.934457096 +0000 UTC m=+0.892436229 container remove 8812331cba60dbf671af7e283147cc4e35070fb6cfbcd8e4a6c644b432482cfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-04T03:30:29.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:29 smithi039 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-04T03:30:29.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:29 smithi039 systemd[1]: Stopped Ceph mon.a for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:30:29.189 INFO:journalctl@ceph.mon.a.smithi039.stdout:May 04 03:30:29 smithi039 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.a.service: Consumed 28.967s CPU time. 2024-05-04T03:30:29.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:30:29.209 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-04T03:30:29.209 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-04T03:30:29.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.b 2024-05-04T03:30:29.530 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:29 smithi066 systemd[1]: Stopping Ceph mon.b for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:29.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-b[33260]: 2024-05-04T03:30:29.527+0000 7f9d548d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:30:29.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-b[33260]: 2024-05-04T03:30:29.527+0000 7f9d548d5700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-04T03:30:29.819 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:29 smithi066 podman[70084]: 2024-05-04 03:30:29.752738443 +0000 UTC m=+0.350277578 container died b24312c8fd4102638fc6e5082939f4894d8e34e95de2e570a98104be360e7a8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-04T03:30:30.490 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:30 smithi066 podman[70084]: 2024-05-04 03:30:30.236927861 +0000 UTC m=+0.834467003 container cleanup b24312c8fd4102638fc6e5082939f4894d8e34e95de2e570a98104be360e7a8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-b, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:30:30.490 INFO:journalctl@ceph.mon.b.smithi066.stdout:May 04 03:30:30 smithi066 bash[70084]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-b 2024-05-04T03:30:30.664 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.b.service' 2024-05-04T03:30:30.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:30:30.707 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-04T03:30:30.707 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-04T03:30:30.707 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.c 2024-05-04T03:30:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:30 smithi167 systemd[1]: Stopping Ceph mon.c for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:31.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-c[33610]: 2024-05-04T03:30:31.020+0000 7ffa09abf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:30:31.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-c[33610]: 2024-05-04T03:30:31.020+0000 7ffa09abf700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-04T03:30:31.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:31 smithi167 podman[71142]: 2024-05-04 03:30:31.225397957 +0000 UTC m=+0.319163342 container died e7adc71bf1ee23361ef06e2bedf82b4c1552dee8ad7ea73b0331450d0be6446f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-04T03:30:31.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:31 smithi167 podman[71142]: 2024-05-04 03:30:31.587851364 +0000 UTC m=+0.681616748 container cleanup e7adc71bf1ee23361ef06e2bedf82b4c1552dee8ad7ea73b0331450d0be6446f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:30:31.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:May 04 03:30:31 smithi167 bash[71142]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mon-c 2024-05-04T03:30:32.298 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mon.c.service' 2024-05-04T03:30:32.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:30:32.344 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-04T03:30:32.344 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-04T03:30:32.345 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.x 2024-05-04T03:30:32.692 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:30:32 smithi039 systemd[1]: Stopping Ceph mgr.x for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:32.982 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:30:32 smithi039 podman[118779]: 2024-05-04 03:30:32.693710572 +0000 UTC m=+0.146785852 container died 4730d304cd2a7bcf36eaae11af669c817c52992f45b6b40d7e321e06951d0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-04T03:30:33.442 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:30:33 smithi039 podman[118779]: 2024-05-04 03:30:33.172426222 +0000 UTC m=+0.625501488 container cleanup 4730d304cd2a7bcf36eaae11af669c817c52992f45b6b40d7e321e06951d0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-04T03:30:33.442 INFO:journalctl@ceph.mgr.x.smithi039.stdout:May 04 03:30:33 smithi039 bash[118779]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-x 2024-05-04T03:30:33.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-04T03:30:33.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:30:33.613 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-04T03:30:33.613 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-04T03:30:33.613 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.y 2024-05-04T03:30:33.956 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:30:33 smithi066 systemd[1]: Stopping Ceph mgr.y for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:34.319 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:30:33 smithi066 podman[70205]: 2024-05-04 03:30:33.954557734 +0000 UTC m=+0.147827766 container died d3214c33ee95aa15ffa415b2abd652edd04a4ae629bcfe2a0aceb9e9c5ce4046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:30:34.705 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:30:34 smithi066 podman[70205]: 2024-05-04 03:30:34.451378161 +0000 UTC m=+0.644648198 container cleanup d3214c33ee95aa15ffa415b2abd652edd04a4ae629bcfe2a0aceb9e9c5ce4046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:30:34.705 INFO:journalctl@ceph.mgr.y.smithi066.stdout:May 04 03:30:34 smithi066 bash[70205]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-y 2024-05-04T03:30:34.843 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-04T03:30:34.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:30:34.878 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-04T03:30:34.878 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-04T03:30:34.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.z 2024-05-04T03:30:35.178 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:30:34 smithi167 systemd[1]: Stopping Ceph mgr.z for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:35.471 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:30:35 smithi167 podman[71266]: 2024-05-04 03:30:35.189043615 +0000 UTC m=+0.119522852 container died fb82acedd078dc481013827781399d1975261d7a80288b90b5fcb7fe7f4d9365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:30:35.928 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:30:35 smithi167 podman[71266]: 2024-05-04 03:30:35.569205349 +0000 UTC m=+0.499684627 container cleanup fb82acedd078dc481013827781399d1975261d7a80288b90b5fcb7fe7f4d9365 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-04T03:30:35.928 INFO:journalctl@ceph.mgr.z.smithi167.stdout:May 04 03:30:35 smithi167 bash[71266]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mgr-z 2024-05-04T03:30:36.235 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-04T03:30:36.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:30:36.268 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-04T03:30:36.268 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-04T03:30:36.268 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.0 2024-05-04T03:30:36.579 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:36 smithi039 systemd[1]: Stopping Ceph osd.0 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:36.942 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:36 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0[42157]: 2024-05-04T03:30:36.577+0000 7fce38818700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:30:36.942 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:36 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0[42157]: 2024-05-04T03:30:36.577+0000 7fce38818700 -1 osd.0 83 *** Got signal Terminated *** 2024-05-04T03:30:36.942 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:36 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0[42157]: 2024-05-04T03:30:36.577+0000 7fce38818700 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:30:42.297 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:41 smithi039 podman[118900]: 2024-05-04 03:30:41.993267756 +0000 UTC m=+5.536053356 container died f34163066569c5e0e98578b1c6e3f05a56474acd0a6d3a9ea4ae82efe5ee6de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:30:42.692 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:42 smithi039 podman[118900]: 2024-05-04 03:30:42.494721694 +0000 UTC m=+6.037507292 container cleanup f34163066569c5e0e98578b1c6e3f05a56474acd0a6d3a9ea4ae82efe5ee6de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-04T03:30:42.692 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:42 smithi039 bash[118900]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0 2024-05-04T03:30:43.130 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:42 smithi039 podman[118919]: 2024-05-04 03:30:42.762976134 +0000 UTC m=+0.762170794 container remove f34163066569c5e0e98578b1c6e3f05a56474acd0a6d3a9ea4ae82efe5ee6de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True) 2024-05-04T03:30:43.402 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:43 smithi039 podman[119015]: 2024-05-04 03:30:43.029726235 +0000 UTC m=+0.018385205 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:30:43.402 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:43 smithi039 podman[119015]: 2024-05-04 03:30:43.156068565 +0000 UTC m=+0.144727579 container create 4f070dc147a9ddbecf3b041e4870eaaa90b662b96e03eeaed423df097f068c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-04T03:30:43.692 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:43 smithi039 podman[119015]: 2024-05-04 03:30:43.400327952 +0000 UTC m=+0.388986907 container init 4f070dc147a9ddbecf3b041e4870eaaa90b662b96e03eeaed423df097f068c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-04T03:30:43.692 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:43 smithi039 podman[119015]: 2024-05-04 03:30:43.403766111 +0000 UTC m=+0.392425066 container start 4f070dc147a9ddbecf3b041e4870eaaa90b662b96e03eeaed423df097f068c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-04T03:30:43.693 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:43 smithi039 podman[119015]: 2024-05-04 03:30:43.517007129 +0000 UTC m=+0.505666088 container attach 4f070dc147a9ddbecf3b041e4870eaaa90b662b96e03eeaed423df097f068c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240430) 2024-05-04T03:30:44.280 INFO:journalctl@ceph.osd.0.smithi039.stdout:May 04 03:30:43 smithi039 podman[119015]: 2024-05-04 03:30:43.974160755 +0000 UTC m=+0.962819722 container died 4f070dc147a9ddbecf3b041e4870eaaa90b662b96e03eeaed423df097f068c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:30:44.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.0.service' 2024-05-04T03:30:44.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:30:44.617 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-04T03:30:44.617 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-04T03:30:44.617 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.1 2024-05-04T03:30:45.050 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:44 smithi066 systemd[1]: Stopping Ceph osd.1 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:45.319 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:45 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1[37875]: 2024-05-04T03:30:45.047+0000 7fa129480700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:30:45.319 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:45 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1[37875]: 2024-05-04T03:30:45.047+0000 7fa129480700 -1 osd.1 83 *** Got signal Terminated *** 2024-05-04T03:30:45.319 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:45 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1[37875]: 2024-05-04T03:30:45.047+0000 7fa129480700 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:30:50.930 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:50 smithi066 podman[70332]: 2024-05-04 03:30:50.598150708 +0000 UTC m=+5.786903802 container died d089e88590e1fd170a0fa6b680e52da51c9f92b04b27f474a0734f93ee2a5572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-04T03:30:51.319 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:51 smithi066 podman[70332]: 2024-05-04 03:30:51.024585494 +0000 UTC m=+6.213338594 container cleanup d089e88590e1fd170a0fa6b680e52da51c9f92b04b27f474a0734f93ee2a5572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-04T03:30:51.319 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:51 smithi066 bash[70332]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1 2024-05-04T03:30:51.819 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:51 smithi066 podman[70349]: 2024-05-04 03:30:51.401929833 +0000 UTC m=+0.803631662 container remove d089e88590e1fd170a0fa6b680e52da51c9f92b04b27f474a0734f93ee2a5572 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD) 2024-05-04T03:30:52.191 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:51 smithi066 podman[70436]: 2024-05-04 03:30:51.809094349 +0000 UTC m=+0.018558547 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:30:52.191 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:51 smithi066 podman[70436]: 2024-05-04 03:30:51.936976623 +0000 UTC m=+0.146440794 container create d9200e0bd3d81759070e764729f53bb763106205f3d68fc71460e8c8c5187502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:30:52.191 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:52 smithi066 podman[70436]: 2024-05-04 03:30:52.189892503 +0000 UTC m=+0.399356677 container init d9200e0bd3d81759070e764729f53bb763106205f3d68fc71460e8c8c5187502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-04T03:30:52.469 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:52 smithi066 podman[70436]: 2024-05-04 03:30:52.193229028 +0000 UTC m=+0.402693202 container start d9200e0bd3d81759070e764729f53bb763106205f3d68fc71460e8c8c5187502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS) 2024-05-04T03:30:52.470 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:52 smithi066 podman[70436]: 2024-05-04 03:30:52.289907263 +0000 UTC m=+0.499371434 container attach d9200e0bd3d81759070e764729f53bb763106205f3d68fc71460e8c8c5187502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:30:52.819 INFO:journalctl@ceph.osd.1.smithi066.stdout:May 04 03:30:52 smithi066 podman[70436]: 2024-05-04 03:30:52.505405626 +0000 UTC m=+0.714869815 container died d9200e0bd3d81759070e764729f53bb763106205f3d68fc71460e8c8c5187502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-04T03:30:53.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.1.service' 2024-05-04T03:30:53.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:30:53.252 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-04T03:30:53.252 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-04T03:30:53.253 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.2 2024-05-04T03:30:53.551 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:30:53 smithi167 systemd[1]: Stopping Ceph osd.2 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:30:53.928 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:30:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2[38939]: 2024-05-04T03:30:53.549+0000 7f01c3836700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:30:53.928 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:30:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2[38939]: 2024-05-04T03:30:53.549+0000 7f01c3836700 -1 osd.2 83 *** Got signal Terminated *** 2024-05-04T03:30:53.928 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:30:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2[38939]: 2024-05-04T03:30:53.549+0000 7f01c3836700 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:30:59.226 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:30:58 smithi167 podman[71392]: 2024-05-04 03:30:58.947470249 +0000 UTC m=+5.511179691 container died eaea622546bf1ed69e74abb48ee2524b81a024e2da01de90382f8291b18e503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:30:59.602 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:30:59 smithi167 podman[71392]: 2024-05-04 03:30:59.357487826 +0000 UTC m=+5.921197268 container cleanup eaea622546bf1ed69e74abb48ee2524b81a024e2da01de90382f8291b18e503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-05-04T03:30:59.602 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:30:59 smithi167 bash[71392]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2 2024-05-04T03:30:59.928 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:30:59 smithi167 podman[71409]: 2024-05-04 03:30:59.600757007 +0000 UTC m=+0.651095254 container remove eaea622546bf1ed69e74abb48ee2524b81a024e2da01de90382f8291b18e503f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-05-04T03:31:00.281 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:31:00 smithi167 podman[71501]: 2024-05-04 03:30:59.925623396 +0000 UTC m=+0.017855792 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:31:00.281 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:31:00 smithi167 podman[71501]: 2024-05-04 03:31:00.035266545 +0000 UTC m=+0.127498964 container create cfe2133a9cfe8df31ab46717578d12e76f2930952f3d4cd9cdb3f804f85a279a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-04T03:31:00.281 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:31:00 smithi167 podman[71501]: 2024-05-04 03:31:00.279929262 +0000 UTC m=+0.372161638 container init cfe2133a9cfe8df31ab46717578d12e76f2930952f3d4cd9cdb3f804f85a279a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-04T03:31:00.580 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:31:00 smithi167 podman[71501]: 2024-05-04 03:31:00.283127939 +0000 UTC m=+0.375360315 container start cfe2133a9cfe8df31ab46717578d12e76f2930952f3d4cd9cdb3f804f85a279a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:31:00.580 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:31:00 smithi167 podman[71501]: 2024-05-04 03:31:00.363258059 +0000 UTC m=+0.455490432 container attach cfe2133a9cfe8df31ab46717578d12e76f2930952f3d4cd9cdb3f804f85a279a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-04T03:31:00.894 INFO:journalctl@ceph.osd.2.smithi167.stdout:May 04 03:31:00 smithi167 podman[71501]: 2024-05-04 03:31:00.579381229 +0000 UTC m=+0.671613614 container died cfe2133a9cfe8df31ab46717578d12e76f2930952f3d4cd9cdb3f804f85a279a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-04T03:31:01.237 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.2.service' 2024-05-04T03:31:01.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:31:01.277 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-04T03:31:01.277 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-04T03:31:01.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.3 2024-05-04T03:31:01.601 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:01 smithi039 systemd[1]: Stopping Ceph osd.3 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:31:01.877 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:01 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3[49136]: 2024-05-04T03:31:01.598+0000 7f2f08ca7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:31:01.877 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:01 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3[49136]: 2024-05-04T03:31:01.598+0000 7f2f08ca7700 -1 osd.3 83 *** Got signal Terminated *** 2024-05-04T03:31:01.877 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:01 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3[49136]: 2024-05-04T03:31:01.598+0000 7f2f08ca7700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:31:03.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:02 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:02.775+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:03.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:03 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:03.230+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:04.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:03 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:03.752+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:04.692 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:04 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3[49136]: 2024-05-04T03:31:04.405+0000 7f2f0525d700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.428704+0000 front 2024-05-04T03:30:38.428672+0000 (oldest deadline 2024-05-04T03:31:04.229541+0000) 2024-05-04T03:31:04.693 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:04 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:04.226+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:05.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:04 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:04.761+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:05.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:04.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:05.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:04 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:04.899+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:05.692 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:05 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3[49136]: 2024-05-04T03:31:05.400+0000 7f2f0525d700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.428704+0000 front 2024-05-04T03:30:38.428672+0000 (oldest deadline 2024-05-04T03:31:04.229541+0000) 2024-05-04T03:31:05.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:05 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:05.245+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:05.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:05.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:05.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:05.606+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:06.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:05 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:05.805+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:06.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:05 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:05.906+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:06.692 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:06 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3[49136]: 2024-05-04T03:31:06.407+0000 7f2f0525d700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.428704+0000 front 2024-05-04T03:30:38.428672+0000 (oldest deadline 2024-05-04T03:31:04.229541+0000) 2024-05-04T03:31:06.693 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:06 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:06.224+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:06.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:06.637+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:06.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:06.701+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:07.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:06 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:06.801+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:07.248 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:07 smithi039 podman[119121]: 2024-05-04 03:31:07.158127581 +0000 UTC m=+5.683169893 container died 56615f8fd63f8e646ff5fa5ae8d4cf5381386a19783ef08e70c18bbdf717d674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, ceph=True) 2024-05-04T03:31:07.249 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:06 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:06.956+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:07.558 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:07 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:07.246+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:07.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:07.625+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:07.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:07.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:07.941 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:07 smithi039 podman[119121]: 2024-05-04 03:31:07.557327993 +0000 UTC m=+6.082370297 container cleanup 56615f8fd63f8e646ff5fa5ae8d4cf5381386a19783ef08e70c18bbdf717d674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:31:07.941 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:07 smithi039 bash[119121]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3 2024-05-04T03:31:07.941 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:07 smithi039 podman[119148]: 2024-05-04 03:31:07.775433973 +0000 UTC m=+0.612624512 container remove 56615f8fd63f8e646ff5fa5ae8d4cf5381386a19783ef08e70c18bbdf717d674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-04T03:31:08.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:07 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:07.801+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:08.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:07 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:07.937+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:08.472 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:08 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:08.199+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:08.473 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:08 smithi039 podman[119237]: 2024-05-04 03:31:08.108851057 +0000 UTC m=+0.018858623 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:31:08.473 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:08 smithi039 podman[119237]: 2024-05-04 03:31:08.218007636 +0000 UTC m=+0.128015185 container create 26e3496a83fc8a0460ef0232f156c6db82b46f5ddc092101fa85e359673af1e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-04T03:31:08.473 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:08 smithi039 podman[119237]: 2024-05-04 03:31:08.470941023 +0000 UTC m=+0.380948585 container init 26e3496a83fc8a0460ef0232f156c6db82b46f5ddc092101fa85e359673af1e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:31:08.785 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:08 smithi039 podman[119237]: 2024-05-04 03:31:08.474363186 +0000 UTC m=+0.384370736 container start 26e3496a83fc8a0460ef0232f156c6db82b46f5ddc092101fa85e359673af1e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-04T03:31:08.785 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:08 smithi039 podman[119237]: 2024-05-04 03:31:08.576953645 +0000 UTC m=+0.486961203 container attach 26e3496a83fc8a0460ef0232f156c6db82b46f5ddc092101fa85e359673af1e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-05-04T03:31:08.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:08.593+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:08.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:08.752+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:09.097 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:08 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:08.961+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:09.098 INFO:journalctl@ceph.osd.3.smithi039.stdout:May 04 03:31:08 smithi039 podman[119262]: 2024-05-04 03:31:08.809080496 +0000 UTC m=+0.019341542 container died 26e3496a83fc8a0460ef0232f156c6db82b46f5ddc092101fa85e359673af1e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-04T03:31:09.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:08 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:08.840+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:09.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:09 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:09.211+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:09.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.3.service' 2024-05-04T03:31:09.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:31:09.531 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-04T03:31:09.532 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-04T03:31:09.532 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.4 2024-05-04T03:31:09.819 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:09 smithi066 systemd[1]: Stopping Ceph osd.4 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:31:09.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:09.566+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:09.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:09.751+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:10.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:09 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:09.854+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:10.319 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:09 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4[44802]: 2024-05-04T03:31:09.863+0000 7f2299e6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:31:10.319 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:09 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4[44802]: 2024-05-04T03:31:09.863+0000 7f2299e6d700 -1 osd.4 83 *** Got signal Terminated *** 2024-05-04T03:31:10.320 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:09 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4[44802]: 2024-05-04T03:31:09.863+0000 7f2299e6d700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:31:10.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:10 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:10.197+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:10.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:09 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:09.994+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:10.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:10.550+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:10.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:10.752+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:11.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:10 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:10.812+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:11.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:11 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:11.203+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:11.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:10 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:10.961+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:11.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:11 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:11.732+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:11.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:11 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:11.525+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:12.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:11 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:11.803+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:12.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:11 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:11.803+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:12.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:12 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:12.218+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:12.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:11 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:11.976+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:12.569 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:12 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:12.255+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:12.678 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:12 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:12.529+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:12.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:12 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:12.424+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:13.069 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:12 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4[44802]: 2024-05-04T03:31:12.757+0000 7f229241b700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.239706+0000 front 2024-05-04T03:30:48.239737+0000 (oldest deadline 2024-05-04T03:31:12.339847+0000) 2024-05-04T03:31:13.070 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:12 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:12.829+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:13.070 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:12 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:12.829+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:13.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:12 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:12.779+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:13.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:13.005+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:13.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:13.224+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:13.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:13 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:13.224+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:13.569 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:13 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:13.258+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:13.817 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:13 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:13.445+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:13.817 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:13 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:13.557+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:14.069 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:13 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4[44802]: 2024-05-04T03:31:13.739+0000 7f229241b700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.239706+0000 front 2024-05-04T03:30:48.239737+0000 (oldest deadline 2024-05-04T03:31:12.339847+0000) 2024-05-04T03:31:14.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:13 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:13.814+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:14.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:13 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:13.814+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:14.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:13 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:13.815+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:14.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:14.214+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:14.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:14.214+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:14.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:14.043+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:14.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:14.043+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:14.569 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:14 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:14.276+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:14.795 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:14 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:14.453+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:14.795 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:14 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:14.524+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:14.795 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:14 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:14.524+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:15.069 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:14 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4[44802]: 2024-05-04T03:31:14.699+0000 7f229241b700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.239706+0000 front 2024-05-04T03:30:48.239737+0000 (oldest deadline 2024-05-04T03:31:12.339847+0000) 2024-05-04T03:31:15.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:14 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:14.792+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:15.070 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:14 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:14.792+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:15.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:14 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:14.792+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:15.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:15 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:15.220+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:15.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:15 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:15.220+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:15.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:14.992+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:15.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:14.992+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:15.569 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:15 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:15.258+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:15.569 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:15 smithi066 podman[70555]: 2024-05-04 03:31:15.360716992 +0000 UTC m=+5.634299539 container died c10393e99505fe3053cc3358e837207961b0e8634398f16c69c2466bb7a662f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-04T03:31:15.792 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:15 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:15.524+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:15.792 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:15 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:15.524+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:15.792 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:15 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:15.498+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:16.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:15 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:15.763+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:16.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:15 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:15.763+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:16.069 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:15 smithi066 podman[70555]: 2024-05-04 03:31:15.88155196 +0000 UTC m=+6.155134508 container cleanup c10393e99505fe3053cc3358e837207961b0e8634398f16c69c2466bb7a662f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-05-04T03:31:16.070 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:15 smithi066 bash[70555]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4 2024-05-04T03:31:16.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:15 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:15.789+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:16.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:15 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:15.789+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:16.273 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:15 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:15.987+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:16.273 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:15 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:15.987+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:16.507 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:16.272+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:16.507 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:16 smithi066 podman[70574]: 2024-05-04 03:31:16.091156927 +0000 UTC m=+0.722181406 container remove c10393e99505fe3053cc3358e837207961b0e8634398f16c69c2466bb7a662f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-04T03:31:16.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:16.270+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:16.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:16.270+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:16.762 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:16 smithi066 podman[70664]: 2024-05-04 03:31:16.40880391 +0000 UTC m=+0.018446180 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:31:16.762 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:16 smithi066 podman[70664]: 2024-05-04 03:31:16.525980319 +0000 UTC m=+0.135622572 container create 474041ea8064fbefe45f0074a713c57e82c05f6a03846bb4f6764df9bbcb92f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-04T03:31:16.812 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:16.528+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:16.813 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:16.528+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:16.813 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:16.533+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:17.014 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:16.760+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:17.014 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:16 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:16.760+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:17.014 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:16 smithi066 podman[70664]: 2024-05-04 03:31:16.911769059 +0000 UTC m=+0.521411327 container init 474041ea8064fbefe45f0074a713c57e82c05f6a03846bb4f6764df9bbcb92f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-04T03:31:17.014 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:16 smithi066 podman[70664]: 2024-05-04 03:31:16.915405458 +0000 UTC m=+0.525047714 container start 474041ea8064fbefe45f0074a713c57e82c05f6a03846bb4f6764df9bbcb92f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:31:17.014 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:17 smithi066 podman[70664]: 2024-05-04 03:31:17.011663824 +0000 UTC m=+0.621306093 container attach 474041ea8064fbefe45f0074a713c57e82c05f6a03846bb4f6764df9bbcb92f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430) 2024-05-04T03:31:17.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:16.810+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:17.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:16 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:16.810+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:17.289 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:16.969+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:17.289 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:16 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:16.969+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:17.314 INFO:journalctl@ceph.osd.4.smithi066.stdout:May 04 03:31:17 smithi066 podman[70664]: 2024-05-04 03:31:17.225658764 +0000 UTC m=+0.835301018 container died 474041ea8064fbefe45f0074a713c57e82c05f6a03846bb4f6764df9bbcb92f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-04T03:31:17.569 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:17 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:17.312+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:17.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:17 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:17.286+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:17.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:17 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:17.286+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:17.814 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:17 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:17.525+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:17.814 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:17 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:17.525+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:17.815 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:17 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:17.513+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:17.843 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.4.service' 2024-05-04T03:31:17.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:31:17.882 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-04T03:31:17.882 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-04T03:31:17.883 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.5 2024-05-04T03:31:18.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:17 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:17.762+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:18.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:17 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:17.762+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:18.178 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:17 smithi167 systemd[1]: Stopping Ceph osd.5 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:31:18.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:17 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:17.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:18.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:17 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:17.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:18.285 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:17 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:17.966+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:18.285 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:17 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:17.966+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:18.544 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:18 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:18.178+0000 7ff2b290f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:31:18.544 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:18 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:18.178+0000 7ff2b290f700 -1 osd.5 83 *** Got signal Terminated *** 2024-05-04T03:31:18.544 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:18 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:18.178+0000 7ff2b290f700 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:31:18.544 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:18 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:18.541+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:18.544 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:18 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:18.541+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:18.569 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:18 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:18.296+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:18.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:18 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:18.282+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:18.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:18 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:18.282+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:18.809 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:18 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:18.555+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:18.809 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:18 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:18.806+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:18.809 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:18 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:18.806+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:19.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:18 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:18.762+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:19.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:18 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:18.762+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:19.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:18 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:18.924+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:19.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:18 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:18.924+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:19.572 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:19 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:19.316+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:19.572 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:19 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:19.316+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:19.788 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:19 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:19.328+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:19.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:19 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:19.563+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:19.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:19 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:19.563+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:19.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:19 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:19.553+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:19.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:19 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:19.553+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:19.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:19 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:19.772+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:19.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:19 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:19.772+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:20.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:19 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:19.786+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:20.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:19 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:19.786+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:20.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:19 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:19.878+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:20.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:19 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:19.878+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:20.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:20.313+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:20.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:20.313+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:20.795 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:20 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:20.370+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:20.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:20 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:20.605+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:20.929 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:20 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:20.605+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:20.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:20 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:20.584+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:20.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:20 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:20.584+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:20.930 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:20 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:20.775+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:20.930 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:20 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:20.775+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:20.930 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:20 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:20.775+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:21.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:20 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:20.793+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:21.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:20 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:20.793+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:21.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:20 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:20.793+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:21.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:20.874+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:21.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:20.874+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:21.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:20.874+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:21.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:21 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:21.281+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:21.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:21 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:21.281+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:21.815 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:21 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:21.342+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:21.815 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:21 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:21.342+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:21.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:21 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:21.654+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:21.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:21 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:21.654+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:21.928 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:21 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:21.654+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:56.911788+0000 front 2024-05-04T03:30:56.911753+0000 (oldest deadline 2024-05-04T03:31:21.011927+0000) 2024-05-04T03:31:21.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:21 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:21.538+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:21.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:21 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:21.538+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:21.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:21 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:21.765+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:21.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:21 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:21.765+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:21.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:21 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:21.765+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:22.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:21 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:21.812+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:22.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:21 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:21.812+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:22.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:21 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:21.812+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:22.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:21 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:21.865+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:22.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:21 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:21.865+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:22.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:21 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:21.865+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:22.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:22 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:22.284+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:22.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:22 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:22.284+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:22.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:22 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:22.284+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:22.786 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:22 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:22.334+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:22.786 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:22 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:22.334+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:22.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:22 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:22.543+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:22.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:22 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:22.543+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:22.929 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:22 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:22.608+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:39.309821+0000 front 2024-05-04T03:30:39.309869+0000 (oldest deadline 2024-05-04T03:31:04.609873+0000) 2024-05-04T03:31:22.929 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:22 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:22.608+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:44.610604+0000 front 2024-05-04T03:30:44.610438+0000 (oldest deadline 2024-05-04T03:31:14.010868+0000) 2024-05-04T03:31:22.929 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:22 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5[45884]: 2024-05-04T03:31:22.608+0000 7ff2aaebd700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:56.911788+0000 front 2024-05-04T03:30:56.911753+0000 (oldest deadline 2024-05-04T03:31:21.011927+0000) 2024-05-04T03:31:22.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:22 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:22.753+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:22.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:22 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:22.753+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:22.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:22 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:22.753+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:23.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:22 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:22.783+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:23.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:22 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:22.783+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:23.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:22 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:22.783+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:23.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:22 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:22.902+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:23.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:22 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:22.902+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:23.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:22 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:22.902+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:23.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:23 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:23.260+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:23.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:23 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:23.260+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:23.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:23 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:23.260+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:23.792 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:23 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:23.360+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:23.792 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:23 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:23.360+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:23.873 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:23 smithi167 podman[71617]: 2024-05-04 03:31:23.630736168 +0000 UTC m=+5.552880288 container died b475e2986cbd683679e04735709f6ba4805e1c19466da76b78f12c104788a038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-05-04T03:31:23.873 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:23 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:23.514+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:23.873 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:23 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:23.514+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:23.874 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:23 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:23.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:23.874 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:23 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:23.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:23.874 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:23 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:23.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:24.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:23 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:23.789+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:24.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:23 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:23.789+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:24.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:23 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:23.789+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:24.178 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:23 smithi167 podman[71617]: 2024-05-04 03:31:23.962528665 +0000 UTC m=+5.884672778 container cleanup b475e2986cbd683679e04735709f6ba4805e1c19466da76b78f12c104788a038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-05-04T03:31:24.178 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:23 smithi167 bash[71617]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5 2024-05-04T03:31:24.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:23 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:23.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:24.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:23 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:23.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:24.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:23 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:23.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:24.550 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:24 smithi167 podman[71636]: 2024-05-04 03:31:24.197239198 +0000 UTC m=+0.563720751 container remove b475e2986cbd683679e04735709f6ba4805e1c19466da76b78f12c104788a038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-05-04T03:31:24.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:24 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:24.219+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:24.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:24 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:24.219+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:24.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:24 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:24.219+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:24.819 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:24 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:24.373+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:24.819 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:24 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:24.373+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:24.852 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:24 smithi167 podman[71726]: 2024-05-04 03:31:24.509171104 +0000 UTC m=+0.017911076 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:31:24.853 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:24 smithi167 podman[71726]: 2024-05-04 03:31:24.614882325 +0000 UTC m=+0.123622288 container create cdd2dd685392d08be9eb6ac3f3d6b08fd79084673a61049ce5aa475764ea7dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-04T03:31:24.853 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:24 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:24.548+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:24.853 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:24 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:24.548+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:24.853 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:24 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:24.748+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:24.853 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:24 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:24.748+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:24.854 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:24 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:24.748+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:25.155 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:24 smithi167 podman[71726]: 2024-05-04 03:31:24.851106546 +0000 UTC m=+0.359846508 container init cdd2dd685392d08be9eb6ac3f3d6b08fd79084673a61049ce5aa475764ea7dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-05-04T03:31:25.155 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:24 smithi167 podman[71726]: 2024-05-04 03:31:24.854166787 +0000 UTC m=+0.362906752 container start cdd2dd685392d08be9eb6ac3f3d6b08fd79084673a61049ce5aa475764ea7dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-04T03:31:25.156 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:24 smithi167 podman[71726]: 2024-05-04 03:31:24.93453137 +0000 UTC m=+0.443271340 container attach cdd2dd685392d08be9eb6ac3f3d6b08fd79084673a61049ce5aa475764ea7dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:31:25.172 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:24 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:24.887+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:25.172 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:24 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:24.887+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:25.172 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:24 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:24.887+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:25.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:24 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:24.826+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:25.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:24 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:24.826+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:25.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:24 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:24.826+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:25.428 INFO:journalctl@ceph.osd.5.smithi167.stdout:May 04 03:31:25 smithi167 podman[71726]: 2024-05-04 03:31:25.15379273 +0000 UTC m=+0.662532701 container died cdd2dd685392d08be9eb6ac3f3d6b08fd79084673a61049ce5aa475764ea7dee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:31:25.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:25 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:25.169+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:25.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:25 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:25.169+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:25.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:25 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:25.169+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:25.765 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:25 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:25.534+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:25.765 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:25 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:25.534+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:25.797 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.5.service' 2024-05-04T03:31:25.819 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:25 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:25.381+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:25.819 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:25 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:25.381+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:25.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:31:25.836 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-04T03:31:25.836 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-04T03:31:25.836 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.6 2024-05-04T03:31:26.142 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:25 smithi039 systemd[1]: Stopping Ceph osd.6 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:31:26.142 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:25 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:25.898+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:26.142 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:25 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:25.898+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:26.142 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:25 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:25.898+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:26.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:25 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:25.798+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:26.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:25 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:25.798+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:26.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:25 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:25.798+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:26.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:25 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:25.820+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:26.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:25 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:25.820+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:26.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:25 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:25.820+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:26.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:26.139+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:26.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:26.139+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:26.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:26.139+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:26.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:26.161+0000 7f2ec4c4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:31:26.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:26.161+0000 7f2ec4c4b700 -1 osd.6 83 *** Got signal Terminated *** 2024-05-04T03:31:26.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:26.161+0000 7f2ec4c4b700 -1 osd.6 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:31:26.789 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:26.488+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:26.789 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:26.488+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:26.789 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:26.786+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:26.789 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:26.786+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:26.789 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:26 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:26.786+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:26.819 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:26 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:26.403+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:26.819 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:26 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:26.403+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:27.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:26 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:26.823+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:27.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:26 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:26.823+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:27.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:26 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:26.823+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:27.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:27.090+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:27.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:27.091+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:27.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:27.091+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:27.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:26.942+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:27.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:26.942+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:27.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:26 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:26.942+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:27.757 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:27 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:27.446+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:27.757 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:27 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:27.446+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:27.788 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:27 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:27.408+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:27.788 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:27 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:27.408+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:27.788 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:27 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:27.408+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:28.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:27 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:27.786+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:28.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:27 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:27.786+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:28.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:27 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:27.786+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:28.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:27 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:27.755+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:28.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:27 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:27.755+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:28.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:27 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:27.755+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:28.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:28.122+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:28.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:28.122+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:28.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:28.122+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:28.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:27.943+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:28.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:27.943+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:28.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:27.943+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:28.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:27 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:27.943+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:28.735 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:28.477+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:28.735 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:28.477+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:28.739 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:28.434+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:28.740 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:28.434+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:28.740 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:28.434+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:29.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:28.737+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:29.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:28.737+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:29.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:28.737+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:29.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:28.733+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:29.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:28.733+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:29.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:28.733+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:29.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:28 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:28.733+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:29.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:29.077+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:29.442 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:29.077+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:29.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:29.077+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:29.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:28.953+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:29.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:28.953+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:29.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:28.953+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:29.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:28 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:28.953+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:29.750 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:29.470+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:29.750 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:29.470+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:29.781 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:29.426+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:29.781 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:29.426+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:29.781 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:29.426+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:30.037 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:29.748+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:30.037 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:29.748+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:30.037 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:29.748+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:30.038 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:29 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:29.748+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:30.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:29.779+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:30.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:29.779+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:30.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:29 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:29.779+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:30.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:29.940+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:30.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:29.940+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:30.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:29.940+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:30.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:29 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:29.940+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:30.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:30.057+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:30.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:30.057+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:30.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:30.057+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:30.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:30.057+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.677805+0000 front 2024-05-04T03:31:01.677824+0000 (oldest deadline 2024-05-04T03:31:29.878488+0000) 2024-05-04T03:31:30.732 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:30 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:30.473+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:30.732 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:30 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:30.473+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:30.732 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:30 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:30.474+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:30.784 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:30.508+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:30.784 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:30.508+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:31.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:30 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:30.729+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:31.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:30 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:30.729+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:31.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:30 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:30.729+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:31.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:30.781+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:31.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:30.781+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:31.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:30.781+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:31.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:30 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:30.781+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:31.344 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:31 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:31.096+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.674572+0000 front 2024-05-04T03:30:40.674542+0000 (oldest deadline 2024-05-04T03:31:02.374760+0000) 2024-05-04T03:31:31.344 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:31 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:31.096+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.976658+0000 front 2024-05-04T03:30:49.976477+0000 (oldest deadline 2024-05-04T03:31:12.276393+0000) 2024-05-04T03:31:31.344 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:31 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:31.096+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.577600+0000 front 2024-05-04T03:30:57.577565+0000 (oldest deadline 2024-05-04T03:31:21.677371+0000) 2024-05-04T03:31:31.344 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:31 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6[55987]: 2024-05-04T03:31:31.096+0000 7f2ec1201700 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.677805+0000 front 2024-05-04T03:31:01.677824+0000 (oldest deadline 2024-05-04T03:31:29.878488+0000) 2024-05-04T03:31:31.344 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:30.973+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:31.344 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:30.973+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:31.344 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:30.973+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:31.344 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:30 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:30.973+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:31.692 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:31 smithi039 podman[119342]: 2024-05-04 03:31:31.568837109 +0000 UTC m=+5.541784144 container died bb46f39f70ef93aa7052dff81afff2a0c49339e9dd7d3acd5e2b0c090381a185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:31:31.746 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:31.469+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:31.746 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:31.469+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:31.746 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:31.469+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:31.747 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:31.485+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:31.747 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:31.485+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:31.747 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:31.485+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:32.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:31.746+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:32.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:31.746+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:32.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:31.746+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:32.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:31 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:31.746+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:32.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:31.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:32.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:31.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:32.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:31.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:32.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:31 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:31.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:32.192 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:32 smithi039 podman[119342]: 2024-05-04 03:31:32.036599872 +0000 UTC m=+6.009546937 container cleanup bb46f39f70ef93aa7052dff81afff2a0c49339e9dd7d3acd5e2b0c090381a185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-04T03:31:32.192 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:32 smithi039 bash[119342]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6 2024-05-04T03:31:32.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:32 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:32.015+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:32.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:32 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:32.015+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:32.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:32 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:32.015+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:32.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:32 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:32.015+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:32.665 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:32 smithi039 podman[119360]: 2024-05-04 03:31:32.338213791 +0000 UTC m=+0.762080463 container remove bb46f39f70ef93aa7052dff81afff2a0c49339e9dd7d3acd5e2b0c090381a185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-04T03:31:32.724 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:32.455+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:32.724 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:32.455+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:32.724 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:32.455+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:32.741 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:32.431+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:32.741 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:32.431+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:32.741 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:32.431+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:32.942 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:32 smithi039 podman[119451]: 2024-05-04 03:31:32.567057508 +0000 UTC m=+0.018568430 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:31:32.942 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:32 smithi039 podman[119451]: 2024-05-04 03:31:32.680645642 +0000 UTC m=+0.132156538 container create eaa4e69a82a2050179c93fc154cbff7e8d1bf40e44af0630f23263d98cfed268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-04T03:31:32.942 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:32 smithi039 podman[119451]: 2024-05-04 03:31:32.908309644 +0000 UTC m=+0.359820541 container init eaa4e69a82a2050179c93fc154cbff7e8d1bf40e44af0630f23263d98cfed268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:31:32.942 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:32 smithi039 podman[119451]: 2024-05-04 03:31:32.911464834 +0000 UTC m=+0.362975735 container start eaa4e69a82a2050179c93fc154cbff7e8d1bf40e44af0630f23263d98cfed268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:31:33.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:32.721+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:33.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:32.721+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:33.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:32.721+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:33.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:32 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:32.721+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:33.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:32.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:33.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:32.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:33.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:32.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:33.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:32 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:32.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:33.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:33 smithi039 podman[119451]: 2024-05-04 03:31:33.000075379 +0000 UTC m=+0.451586284 container attach eaa4e69a82a2050179c93fc154cbff7e8d1bf40e44af0630f23263d98cfed268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-04T03:31:33.443 INFO:journalctl@ceph.osd.6.smithi039.stdout:May 04 03:31:33 smithi039 podman[119451]: 2024-05-04 03:31:33.214203904 +0000 UTC m=+0.665714810 container died eaa4e69a82a2050179c93fc154cbff7e8d1bf40e44af0630f23263d98cfed268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:31:33.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:33.037+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:33.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:33.037+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:33.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:33.037+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:33.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:33 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:33.037+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:33.740 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:33 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:33.481+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:33.740 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:33 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:33.481+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:33.740 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:33 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:33.481+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:33.743 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:33.476+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:33.743 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:33.476+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:33.744 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:33.476+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:33.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.6.service' 2024-05-04T03:31:33.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:31:33.906 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-04T03:31:33.906 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-04T03:31:33.907 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.7 2024-05-04T03:31:34.069 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:33 smithi066 systemd[1]: Stopping Ceph osd.7 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:31:34.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:33.740+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:34.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:33.740+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:34.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:33.740+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:34.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:33.740+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:34.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:33 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:33.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:34.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:33 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:33.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:34.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:33 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:33.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:34.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:33 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:33.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:34.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:34.007+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:34.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:34.007+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:34.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:34.007+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:34.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:34.007+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:34.748 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:34.342+0000 7f5df754d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:31:34.748 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:34.342+0000 7f5df754d700 -1 osd.7 83 *** Got signal Terminated *** 2024-05-04T03:31:34.748 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:34.342+0000 7f5df754d700 -1 osd.7 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:31:34.749 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:34.456+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:34.749 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:34.456+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:34.749 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:34.456+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:34.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:34 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:34.527+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:34.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:34 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:34.527+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:34.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:34 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:34.527+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:34.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:34 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:34.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:34.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:34 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:34.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:34.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:34 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:34.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:34.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:34 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:34.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:35.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:34.746+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:35.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:34.746+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:35.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:34.746+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:35.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:34 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:34.746+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:35.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:34.990+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:35.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:34.990+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:35.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:34.990+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:35.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:34 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:34.990+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:35.796 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:35.499+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:35.796 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:35.499+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:35.796 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:35.499+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:35.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:35 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:35.532+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:35.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:35 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:35.532+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:35.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:35 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:35.532+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:35.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:35 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:35.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:35.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:35 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:35.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:35.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:35 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:35.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:35.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:35 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:35.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:36.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:35.794+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:36.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:35.794+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:36.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:35.794+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:36.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:35 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:35.794+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:36.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:36 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:36.019+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:36.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:36 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:36.019+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:36.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:36 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:36.019+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:36.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:36 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:36.019+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:36.797 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:36 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:36.458+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:36.797 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:36 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:36.458+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:36.797 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:36 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:36.458+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:36.798 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:36 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:36.458+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:10.698649+0000 front 2024-05-04T03:31:10.698624+0000 (oldest deadline 2024-05-04T03:31:35.998703+0000) 2024-05-04T03:31:36.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:36 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:36.523+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:36.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:36 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:36.523+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:36.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:36 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:36.523+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:36.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:36 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:36.523+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:36.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:36 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:36.695+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:36.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:36 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:36.695+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:36.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:36 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:36.695+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:36.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:36 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:36.695+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:37.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:36 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:36.794+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:37.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:36 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:36.794+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:37.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:36 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:36.794+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:37.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:36 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:36.794+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:37.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:37 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:37.069+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:37.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:37 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:37.069+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:37.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:37 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:37.069+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:37.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:37 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:37.069+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:37.787 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:37.495+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:37.787 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:37.495+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:37.788 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:37.495+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:37.788 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:37.495+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:10.698649+0000 front 2024-05-04T03:31:10.698624+0000 (oldest deadline 2024-05-04T03:31:35.998703+0000) 2024-05-04T03:31:37.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:37 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:37.512+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:37.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:37 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:37.512+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:37.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:37 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:37.512+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:37.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:37 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:37.512+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:37.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:37 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:37.693+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:37.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:37 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:37.693+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:37.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:37 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:37.693+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:37.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:37 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:37.693+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:38.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:37.784+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:38.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:37.784+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:38.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:37.784+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:38.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:37.784+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:38.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:38 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:38.057+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:38.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:38 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:38.057+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:38.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:38 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:38.057+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:38.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:38 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:38.057+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:38.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:38 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:38.057+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:38.798 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:38 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:38.479+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.793440+0000 front 2024-05-04T03:30:48.793638+0000 (oldest deadline 2024-05-04T03:31:11.693561+0000) 2024-05-04T03:31:38.798 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:38 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:38.479+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.995685+0000 front 2024-05-04T03:30:57.995578+0000 (oldest deadline 2024-05-04T03:31:20.895711+0000) 2024-05-04T03:31:38.798 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:38 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:38.479+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:06.097458+0000 front 2024-05-04T03:31:06.097652+0000 (oldest deadline 2024-05-04T03:31:27.197536+0000) 2024-05-04T03:31:38.798 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:38 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7[51637]: 2024-05-04T03:31:38.479+0000 7f5defafb700 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:10.698649+0000 front 2024-05-04T03:31:10.698624+0000 (oldest deadline 2024-05-04T03:31:35.998703+0000) 2024-05-04T03:31:38.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:38.480+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:38.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:38.480+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:38.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:38.480+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:38.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:38.480+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:38.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:38.681+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:38.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:38.681+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:38.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:38.681+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:38.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:38 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:38.681+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:39.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:38 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:38.796+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:39.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:38 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:38.796+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:39.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:38 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:38.796+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:39.069 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:38 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:38.796+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:39.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:39 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:39.047+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:39.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:39 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:39.047+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:39.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:39 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:39.047+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:39.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:39 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:39.047+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:39.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:39 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:39.047+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:39.711 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:39.458+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:39.711 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:39.458+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:39.711 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:39.458+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:39.712 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:39.458+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:40.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:39.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:40.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:39.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:40.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:39.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:40.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:39.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:40.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:39 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:39.708+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:40.319 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:39 smithi066 podman[70781]: 2024-05-04 03:31:39.947844516 +0000 UTC m=+5.846578820 container died 55669f7d9140e38f0124f0f08814357d4752dbdfec46d7b7a1614be2d95ae46e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-04T03:31:40.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:39 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:39.820+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:40.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:39 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:39.820+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:40.320 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:39 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:39.820+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:40.320 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:39 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:39.820+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:40.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.008+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:40.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.008+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:40.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.008+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:40.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.008+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:40.443 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.008+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:40.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:40.424+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:40.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:40.424+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:40.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:40.424+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:40.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:40.424+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:40.704 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:40 smithi066 podman[70781]: 2024-05-04 03:31:40.443466355 +0000 UTC m=+6.342200652 container cleanup 55669f7d9140e38f0124f0f08814357d4752dbdfec46d7b7a1614be2d95ae46e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:31:40.704 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:40 smithi066 bash[70781]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7 2024-05-04T03:31:41.030 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:40 smithi066 podman[70799]: 2024-05-04 03:31:40.702766503 +0000 UTC m=+0.751188944 container remove 55669f7d9140e38f0124f0f08814357d4752dbdfec46d7b7a1614be2d95ae46e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-04T03:31:41.030 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:40 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:40.846+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:41.030 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:40 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:40.846+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:41.030 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:40 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:40.846+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:41.030 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:40 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:40.846+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:41.030 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:40 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:40.846+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:41.044 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:40.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:41.044 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:40.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:41.044 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:40.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:41.044 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:40.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:41.045 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:40 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:40.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:41.319 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:41 smithi066 podman[70891]: 2024-05-04 03:31:40.930926799 +0000 UTC m=+0.017952036 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:31:41.319 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:41 smithi066 podman[70891]: 2024-05-04 03:31:41.028634708 +0000 UTC m=+0.115659940 container create 037becbe269042fae2702b9188ff705ae3b25dd18cec62b6e2a7b70a7a36bbd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-04T03:31:41.319 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:41 smithi066 podman[70891]: 2024-05-04 03:31:41.256134007 +0000 UTC m=+0.343159236 container init 037becbe269042fae2702b9188ff705ae3b25dd18cec62b6e2a7b70a7a36bbd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS) 2024-05-04T03:31:41.319 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:41 smithi066 podman[70891]: 2024-05-04 03:31:41.259326492 +0000 UTC m=+0.346351731 container start 037becbe269042fae2702b9188ff705ae3b25dd18cec62b6e2a7b70a7a36bbd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-04T03:31:41.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.959+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:41.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.959+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:41.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.959+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:41.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.959+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:41.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:40 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:40.959+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:41.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:41.388+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:41.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:41.388+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:41.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:41.388+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:41.679 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:41.388+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:41.819 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:41 smithi066 podman[70891]: 2024-05-04 03:31:41.339296938 +0000 UTC m=+0.426322184 container attach 037becbe269042fae2702b9188ff705ae3b25dd18cec62b6e2a7b70a7a36bbd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:31:41.819 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:41 smithi066 podman[70891]: 2024-05-04 03:31:41.56529181 +0000 UTC m=+0.652317053 container died 037becbe269042fae2702b9188ff705ae3b25dd18cec62b6e2a7b70a7a36bbd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-04T03:31:42.169 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:41 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:41.896+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:42.170 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:41 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:41.896+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:42.170 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:41 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:41.896+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:42.170 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:41 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:41.896+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:42.170 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:41 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:41.896+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:42.170 INFO:journalctl@ceph.osd.7.smithi066.stdout:May 04 03:31:42 smithi066 podman[70917]: 2024-05-04 03:31:42.168141353 +0000 UTC m=+0.597447509 container remove 037becbe269042fae2702b9188ff705ae3b25dd18cec62b6e2a7b70a7a36bbd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-04T03:31:42.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:41.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:42.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:41.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:42.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:41.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:42.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:41.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:42.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:41 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:41.738+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:42.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.7.service' 2024-05-04T03:31:42.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:41 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:41.915+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:42.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:41 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:41.915+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:42.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:41 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:41.915+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:42.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:41 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:41.915+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:42.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:41 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:41.915+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:42.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:31:42.233 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-04T03:31:42.233 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-04T03:31:42.233 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.8 2024-05-04T03:31:42.535 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:42 smithi167 systemd[1]: Stopping Ceph osd.8 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:31:42.535 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:42.420+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:42.535 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:42.420+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:42.535 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:42.420+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:42.535 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:42.420+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:42.536 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:42.532+0000 7fd9f0a7d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:31:42.536 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:42.532+0000 7fd9f0a7d700 -1 osd.8 83 *** Got signal Terminated *** 2024-05-04T03:31:42.536 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:42.532+0000 7fd9f0a7d700 -1 osd.8 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:31:42.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:42.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:42.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:42.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:42.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:42.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:42.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:42.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:42.928 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:42 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:42.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:43.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:42 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:42.874+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:43.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:42 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:42.874+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:43.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:42 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:42.874+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:43.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:42 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:42.874+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:43.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:42 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:42.874+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:43.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:42 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:42.941+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:43.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:42 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:42.941+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:43.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:42 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:42.941+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:43.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:42 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:42.941+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:43.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:42 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:42.941+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:43.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:43.436+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:43.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:43.436+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:43.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:43.436+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:43.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:43.436+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:44.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:43.713+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:44.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:43.713+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:44.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:43.713+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:44.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:43.713+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:44.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:43 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:43.713+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:44.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:43.834+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:44.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:43.834+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:44.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:43.834+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:44.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:43.834+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:44.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:43 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:43.834+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:44.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:43 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:43.925+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:44.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:43 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:43.925+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:44.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:43 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:43.925+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:44.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:43 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:43.925+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:44.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:43 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:43.925+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:44.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:44.413+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:44.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:44.413+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:44.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:44.413+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:44.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:44.413+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:45.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:44.723+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:45.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:44.723+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:45.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:44.723+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:45.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:44.723+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:45.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:44 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:44.723+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:45.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:44.830+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:45.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:44.830+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:45.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:44.830+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:45.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:44.830+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:45.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:44.830+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:45.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:44 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:44.830+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:45.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:44 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:44.908+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:45.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:44 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:44.908+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:45.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:44 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:44.908+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:45.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:44 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:44.908+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:45.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:44 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:44.908+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:45.740 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:45.432+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:45.741 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:45.432+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:45.741 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:45.432+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:45.741 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:45.432+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:46.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:45.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:46.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:45.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:46.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:45.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:46.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:45.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:46.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:45.739+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:46.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:45.860+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:46.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:45.860+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:46.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:45.860+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:46.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:45.860+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:46.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:45.860+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:46.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:45 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:45.860+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:46.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:45 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:45.931+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:46.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:45 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:45.931+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:46.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:45 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:45.931+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:46.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:45 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:45.931+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:46.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:45 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:45.931+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:46.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:46.479+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:46.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:46.479+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:46.928 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:46.479+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:46.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:46.479+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:46.929 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:46.479+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:18.871831+0000 front 2024-05-04T03:31:18.871858+0000 (oldest deadline 2024-05-04T03:31:46.472574+0000) 2024-05-04T03:31:46.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:46.719+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:46.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:46.719+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:46.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:46.719+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:46.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:46.719+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:46.929 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:46 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:46.719+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:47.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:46.864+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:47.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:46.864+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:47.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:46.864+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:47.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:46.864+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:47.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:46.864+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:47.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:46 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:46.864+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:47.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:46 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:46.928+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:47.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:46 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:46.928+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:47.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:46 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:46.928+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:47.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:46 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:46.928+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:47.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:46 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:46.928+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:47.726 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:47.430+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:47.466340+0000 front 2024-05-04T03:30:47.466373+0000 (oldest deadline 2024-05-04T03:31:12.166411+0000) 2024-05-04T03:31:47.726 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:47.430+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:53.267400+0000 front 2024-05-04T03:30:53.267594+0000 (oldest deadline 2024-05-04T03:31:19.167523+0000) 2024-05-04T03:31:47.726 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:47.430+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.068939+0000 front 2024-05-04T03:31:05.068966+0000 (oldest deadline 2024-05-04T03:31:30.968551+0000) 2024-05-04T03:31:47.726 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:47.430+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.770477+0000 front 2024-05-04T03:31:13.770440+0000 (oldest deadline 2024-05-04T03:31:36.070245+0000) 2024-05-04T03:31:47.726 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8[52752]: 2024-05-04T03:31:47.430+0000 7fd9e902b700 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:18.871831+0000 front 2024-05-04T03:31:18.871858+0000 (oldest deadline 2024-05-04T03:31:46.472574+0000) 2024-05-04T03:31:48.178 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:47 smithi167 podman[71846]: 2024-05-04 03:31:47.928645597 +0000 UTC m=+5.499952953 container died 719dedb0ab04ec69ef8a4c89a267feda3982e1667e7e5a534dab061a8695d2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-04T03:31:48.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:47.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:48.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:47.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:48.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:47.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:48.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:47.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:48.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:47.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:48.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:47 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:47.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:48.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:47 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:47.847+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:48.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:47 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:47.847+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:48.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:47 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:47.847+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:48.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:47 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:47.847+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:48.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:47 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:47.847+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:48.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:47 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:47.847+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:48.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:47 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:47.927+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:48.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:47 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:47.927+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:48.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:47 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:47.927+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:48.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:47 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:47.927+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:48.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:47 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:47.927+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:48.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:47 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:47.927+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:48.552 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:48 smithi167 podman[71846]: 2024-05-04 03:31:48.308289296 +0000 UTC m=+5.879596643 container cleanup 719dedb0ab04ec69ef8a4c89a267feda3982e1667e7e5a534dab061a8695d2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-04T03:31:48.552 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:48 smithi167 bash[71846]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8 2024-05-04T03:31:48.882 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:48 smithi167 podman[71865]: 2024-05-04 03:31:48.551052485 +0000 UTC m=+0.623744966 container remove 719dedb0ab04ec69ef8a4c89a267feda3982e1667e7e5a534dab061a8695d2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-04T03:31:48.882 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:48 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:48.687+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:48.882 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:48 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:48.687+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:48.882 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:48 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:48.687+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:48.882 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:48 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:48.687+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:48.882 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:48 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:48.687+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:48.882 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:48 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:48.687+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:49.178 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:48 smithi167 podman[71952]: 2024-05-04 03:31:48.783599881 +0000 UTC m=+0.017982075 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:31:49.179 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:48 smithi167 podman[71952]: 2024-05-04 03:31:48.885307902 +0000 UTC m=+0.119690139 container create 9e243ca5eee9eaee827cfa3d7df9aaa63d1a21372a9ef0699248fe1f2acd0da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:31:49.179 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:49 smithi167 podman[71952]: 2024-05-04 03:31:49.121537648 +0000 UTC m=+0.355919851 container init 9e243ca5eee9eaee827cfa3d7df9aaa63d1a21372a9ef0699248fe1f2acd0da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-04T03:31:49.179 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:49 smithi167 podman[71952]: 2024-05-04 03:31:49.124873758 +0000 UTC m=+0.359255956 container start 9e243ca5eee9eaee827cfa3d7df9aaa63d1a21372a9ef0699248fe1f2acd0da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:31:49.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:48 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:48.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:49.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:48 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:48.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:49.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:48 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:48.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:49.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:48 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:48.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:49.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:48 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:48.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:49.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:48 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:48.886+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:49.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:48 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:48.968+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:49.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:48 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:48.968+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:49.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:48 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:48.968+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:49.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:48 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:48.968+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:49.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:48 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:48.968+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:49.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:48 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:48.968+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:49.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:49 smithi167 podman[71952]: 2024-05-04 03:31:49.212993602 +0000 UTC m=+0.447375815 container attach 9e243ca5eee9eaee827cfa3d7df9aaa63d1a21372a9ef0699248fe1f2acd0da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-04T03:31:49.678 INFO:journalctl@ceph.osd.8.smithi167.stdout:May 04 03:31:49 smithi167 podman[71952]: 2024-05-04 03:31:49.417236477 +0000 UTC m=+0.651618673 container died 9e243ca5eee9eaee827cfa3d7df9aaa63d1a21372a9ef0699248fe1f2acd0da2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-04T03:31:50.020 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:49 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:49.683+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:50.021 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:49 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:49.683+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:50.021 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:49 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:49.683+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:50.021 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:49 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:49.683+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:50.021 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:49 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:49.683+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:50.021 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:49 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:49.683+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:50.039 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.8.service' 2024-05-04T03:31:50.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:31:50.079 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-04T03:31:50.079 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-04T03:31:50.079 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.9 2024-05-04T03:31:50.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:49 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:49.926+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:50.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:49 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:49.926+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:50.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:49 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:49.926+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:50.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:49 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:49.926+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:50.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:49 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:49.926+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:50.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:49 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:49.926+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:50.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 systemd[1]: Stopping Ceph osd.9 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:31:50.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.000+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:50.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.000+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:50.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.000+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:50.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.000+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:50.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.000+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:50.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.000+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:50.692 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.388+0000 7fab5bcf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:31:50.692 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.388+0000 7fab5bcf7700 -1 osd.9 83 *** Got signal Terminated *** 2024-05-04T03:31:50.692 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.388+0000 7fab5bcf7700 -1 osd.9 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:31:51.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:50 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:50.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:51.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:50 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:50.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:51.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:50 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:50.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:51.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:50 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:50.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:51.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:50 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:50.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:51.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:50 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:50.729+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:51.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.896+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:51.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.896+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:51.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.896+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:51.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.896+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:51.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.896+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:51.193 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:50 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:50.896+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:51.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.954+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:51.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.954+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:51.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.954+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:51.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.954+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:51.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.954+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:51.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:50 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:50.954+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:52.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:51 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:51.778+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:52.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:51 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:51.778+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:52.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:51 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:51.778+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:52.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:51 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:51.778+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:52.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:51 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:51.778+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:52.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:51 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:51.778+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:52.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:51 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:51.935+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:52.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:51 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:51.935+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:52.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:51 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:51.935+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:52.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:51 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:51.935+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:52.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:51 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:51.935+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:52.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:51 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:51.935+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:52.252 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:51 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:51.958+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:52.252 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:51 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:51.958+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:52.252 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:51 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:51.958+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:52.252 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:51 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:51.959+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:52.252 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:51 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:51.959+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:52.252 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:51 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:51.959+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:52.252 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:51 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:51.959+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:31:53.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:52 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:52.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:53.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:52 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:52.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:53.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:52 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:52.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:53.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:52 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:52.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:53.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:52 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:52.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:53.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:52 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:52.744+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:53.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:52.904+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:53.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:52.904+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:53.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:52.904+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:53.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:52.904+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:53.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:52.904+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:53.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:52 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:52.904+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:53.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:52 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:52.943+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:53.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:52 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:52.943+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:53.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:52 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:52.943+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:53.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:52 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:52.943+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:53.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:52 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:52.943+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:53.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:52 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:52.943+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:53.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:52 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:52.943+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:31:54.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:53.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:54.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:53.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:54.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:53.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:54.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:53.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:54.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:53.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:54.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:53.714+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:54.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:53 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:53.916+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:54.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:53 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:53.916+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:54.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:53 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:53.916+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:54.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:53 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:53.916+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:54.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:53 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:53.916+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:54.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:53 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:53.916+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:54.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:53 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:53.916+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:31:54.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:53 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:53.955+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:54.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:53 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:53.955+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:54.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:53 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:53.955+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:54.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:53 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:53.955+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:54.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:53 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:53.955+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:54.442 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:53 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:53.955+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:55.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:54 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:54.758+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:55.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:54 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:54.758+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:55.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:54 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:54.758+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:55.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:54 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:54.758+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:55.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:54 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:54.758+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:55.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:54 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:54.758+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:55.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:54 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:54.933+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.715035+0000 front 2024-05-04T03:30:38.715067+0000 (oldest deadline 2024-05-04T03:31:04.615146+0000) 2024-05-04T03:31:55.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:54 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:54.933+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:49.916428+0000 front 2024-05-04T03:30:49.916366+0000 (oldest deadline 2024-05-04T03:31:13.416520+0000) 2024-05-04T03:31:55.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:54 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:54.933+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:58.117819+0000 front 2024-05-04T03:30:58.117887+0000 (oldest deadline 2024-05-04T03:31:20.417895+0000) 2024-05-04T03:31:55.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:54 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:54.933+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.619509+0000 front 2024-05-04T03:31:05.619478+0000 (oldest deadline 2024-05-04T03:31:27.319652+0000) 2024-05-04T03:31:55.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:54 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:54.933+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:11.821822+0000 front 2024-05-04T03:31:11.821876+0000 (oldest deadline 2024-05-04T03:31:37.721715+0000) 2024-05-04T03:31:55.192 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:54 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9[62717]: 2024-05-04T03:31:54.933+0000 7fab542a5700 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.223004+0000 front 2024-05-04T03:31:21.222935+0000 (oldest deadline 2024-05-04T03:31:44.122759+0000) 2024-05-04T03:31:55.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:54 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:54.923+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:55.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:54 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:54.924+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:55.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:54 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:54.924+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:55.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:54 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:54.924+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:55.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:54 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:54.924+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:55.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:54 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:54.924+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:55.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:54 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:54.924+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:31:56.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:55 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:55.747+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:56.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:55 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:55.747+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:56.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:55 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:55.747+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:56.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:55 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:55.747+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:56.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:55 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:55.747+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:56.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:55 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:55.747+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:56.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:55 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:55.940+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:56.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:55 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:55.940+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:56.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:55 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:55.940+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:56.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:55 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:55.940+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:56.320 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:55 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:55.940+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:56.320 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:55 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:55.940+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:56.320 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:55 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:55.940+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:31:56.333 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:55 smithi039 podman[119555]: 2024-05-04 03:31:55.969270888 +0000 UTC m=+5.692573506 container died c0d76dc23384014db5b69c6d103e7b1f867f5370d828ec02b2c557f9c310e1d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-04T03:31:56.666 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:56 smithi039 podman[119555]: 2024-05-04 03:31:56.480172587 +0000 UTC m=+6.203475217 container cleanup c0d76dc23384014db5b69c6d103e7b1f867f5370d828ec02b2c557f9c310e1d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-04T03:31:56.666 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:56 smithi039 bash[119555]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9 2024-05-04T03:31:56.942 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:56 smithi039 podman[119573]: 2024-05-04 03:31:56.664927844 +0000 UTC m=+0.698318588 container remove c0d76dc23384014db5b69c6d103e7b1f867f5370d828ec02b2c557f9c310e1d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-04T03:31:57.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:56 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:56.767+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:57.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:56 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:56.767+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:57.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:56 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:56.767+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:57.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:56 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:56.767+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:57.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:56 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:56.767+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:57.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:56 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:56.767+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:57.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:56 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:56.961+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:57.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:56 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:56.961+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:57.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:56 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:56.961+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:57.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:56 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:56.961+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:57.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:56 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:56.961+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:57.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:56 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:56.961+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:57.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:56 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:56.961+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:31:57.418 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:57 smithi039 podman[119661]: 2024-05-04 03:31:57.10720672 +0000 UTC m=+0.114501626 container create fee62d02031db84e14862188b4499fc0cdb02013be7418f365b16e9a51eac067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-04T03:31:57.418 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:57 smithi039 podman[119661]: 2024-05-04 03:31:57.0112478 +0000 UTC m=+0.018542734 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:31:57.418 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:57 smithi039 podman[119661]: 2024-05-04 03:31:57.309551708 +0000 UTC m=+0.316846617 container init fee62d02031db84e14862188b4499fc0cdb02013be7418f365b16e9a51eac067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:31:57.418 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:57 smithi039 podman[119661]: 2024-05-04 03:31:57.312937227 +0000 UTC m=+0.320232142 container start fee62d02031db84e14862188b4499fc0cdb02013be7418f365b16e9a51eac067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-04T03:31:57.418 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:57 smithi039 podman[119661]: 2024-05-04 03:31:57.41597829 +0000 UTC m=+0.423273196 container attach fee62d02031db84e14862188b4499fc0cdb02013be7418f365b16e9a51eac067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-04T03:31:57.692 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:57 smithi039 podman[119661]: 2024-05-04 03:31:57.614312608 +0000 UTC m=+0.621607530 container died fee62d02031db84e14862188b4499fc0cdb02013be7418f365b16e9a51eac067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:31:58.144 INFO:journalctl@ceph.osd.9.smithi039.stdout:May 04 03:31:58 smithi039 podman[119687]: 2024-05-04 03:31:58.129713741 +0000 UTC m=+0.509118904 container remove fee62d02031db84e14862188b4499fc0cdb02013be7418f365b16e9a51eac067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2) 2024-05-04T03:31:58.150 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.9.service' 2024-05-04T03:31:58.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:57 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:57.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:58.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:57 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:57.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:58.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:57 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:57.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:58.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:57 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:57.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:58.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:57 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:57.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:58.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:57 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:57.724+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:58.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:31:58.190 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-04T03:31:58.190 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-04T03:31:58.190 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.10 2024-05-04T03:31:58.217 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:57 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:57.937+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:58.217 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:57 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:57.937+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:58.217 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:57 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:57.937+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:58.217 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:57 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:57.937+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:58.218 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:57 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:57.937+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:58.218 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:57 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:57.937+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:58.218 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:57 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:57.937+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:31:58.489 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 systemd[1]: Stopping Ceph osd.10 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:31:58.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.487+0000 7f011ef3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:31:58.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.487+0000 7f011ef3b700 -1 osd.10 83 *** Got signal Terminated *** 2024-05-04T03:31:58.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.487+0000 7f011ef3b700 -1 osd.10 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:31:59.080 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:58 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:58.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:31:59.080 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:58 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:58.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:31:59.080 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:58 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:58.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:31:59.080 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:58 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:58.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:31:59.080 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:58 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:58.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:31:59.080 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:58 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:58.742+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:31:59.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.948+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:31:59.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.949+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:31:59.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.949+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:31:59.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.949+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:31:59.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.949+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:31:59.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.949+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:31:59.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:58 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:58.949+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:32:00.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:59 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:59.789+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:00.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:59 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:59.790+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:00.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:59 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:59.790+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:00.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:59 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:59.790+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:00.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:59 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:59.790+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:00.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:31:59 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:31:59.790+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:00.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:59 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:59.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:32:00.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:59 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:59.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:32:00.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:59 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:59.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:32:00.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:59 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:59.946+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:32:00.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:59 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:59.946+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:32:00.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:59 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:59.946+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:32:00.320 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:31:59 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:31:59.946+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:32:01.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:00 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:00.829+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:01.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:00 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:00.829+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:01.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:00 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:00.829+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:01.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:00 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:00.829+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:01.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:00 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:00.829+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:01.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:00 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:00.829+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:01.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:00 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:00.992+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:32:01.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:00 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:00.992+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:32:01.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:00 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:00.992+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:32:01.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:00 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:00.992+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:32:01.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:00 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:00.992+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:32:01.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:00 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:00.992+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:32:01.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:00 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:00.992+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:32:02.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:01 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:01.847+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:02.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:01 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:01.847+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:02.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:01 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:01.847+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:02.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:01 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:01.847+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:02.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:01 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:01.847+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:02.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:01 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:01.847+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:02.220 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:01 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:01.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:32:02.221 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:01 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:01.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:32:02.221 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:01 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:01.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:32:02.221 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:01 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:01.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:32:02.221 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:01 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:01.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:32:02.221 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:01 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:01.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:32:02.221 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:01 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:01.945+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:32:03.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:02 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:02.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:03.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:02 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:02.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:03.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:02 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:02.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:03.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:02 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:02.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:03.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:02 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:02.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:03.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:02 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:02.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:03.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:02 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:02.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:03.236 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:02 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:02.972+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:38.638628+0000 front 2024-05-04T03:30:38.638763+0000 (oldest deadline 2024-05-04T03:31:02.738560+0000) 2024-05-04T03:32:03.236 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:02 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:02.972+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:46.740347+0000 front 2024-05-04T03:30:46.740259+0000 (oldest deadline 2024-05-04T03:31:10.840408+0000) 2024-05-04T03:32:03.236 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:02 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:02.972+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:57.742542+0000 front 2024-05-04T03:30:57.742723+0000 (oldest deadline 2024-05-04T03:31:20.642630+0000) 2024-05-04T03:32:03.236 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:02 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:02.972+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:05.343629+0000 front 2024-05-04T03:31:05.343734+0000 (oldest deadline 2024-05-04T03:31:31.144190+0000) 2024-05-04T03:32:03.236 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:02 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:02.972+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:14.645163+0000 front 2024-05-04T03:31:14.645126+0000 (oldest deadline 2024-05-04T03:31:40.545286+0000) 2024-05-04T03:32:03.236 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:02 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:02.972+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:20.546007+0000 front 2024-05-04T03:31:20.546080+0000 (oldest deadline 2024-05-04T03:31:47.546397+0000) 2024-05-04T03:32:03.236 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:02 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10[58210]: 2024-05-04T03:32:02.972+0000 7f011b4f1700 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-05-04T03:31:27.547112+0000 front 2024-05-04T03:31:27.547020+0000 (oldest deadline 2024-05-04T03:31:51.646816+0000) 2024-05-04T03:32:04.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:03 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:03.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:04.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:03 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:03.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:04.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:03 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:03.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:04.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:03 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:03.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:04.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:03 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:03.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:04.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:03 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:03.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:04.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:03 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:03.886+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:04.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:04 smithi066 podman[70996]: 2024-05-04 03:32:04.051326815 +0000 UTC m=+5.657864477 container died 92d584f108a22a4e9a970d5022b16d1fe35f1f63f24d961f1f114ec79c9a85aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-04T03:32:04.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:04 smithi066 podman[70996]: 2024-05-04 03:32:04.38713892 +0000 UTC m=+5.993676569 container cleanup 92d584f108a22a4e9a970d5022b16d1fe35f1f63f24d961f1f114ec79c9a85aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:32:04.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:04 smithi066 bash[70996]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10 2024-05-04T03:32:04.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:04 smithi066 podman[71024]: 2024-05-04 03:32:04.605454737 +0000 UTC m=+0.551326444 container remove 92d584f108a22a4e9a970d5022b16d1fe35f1f63f24d961f1f114ec79c9a85aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-04T03:32:05.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:04.908+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:05.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:04.908+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:05.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:04.908+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:05.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:04.908+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:05.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:04.908+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:05.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:04.908+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:05.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:04 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:04.908+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:05.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:05 smithi066 podman[71115]: 2024-05-04 03:32:04.951043435 +0000 UTC m=+0.018310357 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:32:05.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:05 smithi066 podman[71115]: 2024-05-04 03:32:05.056384944 +0000 UTC m=+0.123651851 container create 89b12319ee2b8771c0c2c5728b2f10b0b773429a6f8be59b44581e904f7ddebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:32:05.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:05 smithi066 podman[71115]: 2024-05-04 03:32:05.258660365 +0000 UTC m=+0.325927272 container init 89b12319ee2b8771c0c2c5728b2f10b0b773429a6f8be59b44581e904f7ddebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:32:05.319 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:05 smithi066 podman[71115]: 2024-05-04 03:32:05.261814951 +0000 UTC m=+0.329081864 container start 89b12319ee2b8771c0c2c5728b2f10b0b773429a6f8be59b44581e904f7ddebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-04T03:32:05.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:05 smithi066 podman[71115]: 2024-05-04 03:32:05.333382921 +0000 UTC m=+0.400649831 container attach 89b12319ee2b8771c0c2c5728b2f10b0b773429a6f8be59b44581e904f7ddebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-04T03:32:05.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:05 smithi066 podman[71115]: 2024-05-04 03:32:05.567419897 +0000 UTC m=+0.634686813 container died 89b12319ee2b8771c0c2c5728b2f10b0b773429a6f8be59b44581e904f7ddebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240430, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:32:06.071 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.10.service' 2024-05-04T03:32:06.135 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:06 smithi066 podman[71141]: 2024-05-04 03:32:06.04481093 +0000 UTC m=+0.472171636 container remove 89b12319ee2b8771c0c2c5728b2f10b0b773429a6f8be59b44581e904f7ddebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-04T03:32:06.135 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:06 smithi066 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.10.service: Deactivated successfully. 2024-05-04T03:32:06.135 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:06 smithi066 systemd[1]: Stopped Ceph osd.10 for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:32:06.135 INFO:journalctl@ceph.osd.10.smithi066.stdout:May 04 03:32:06 smithi066 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.10.service: Consumed 5min 4.649s CPU time. 2024-05-04T03:32:06.150 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:06.151 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-04T03:32:06.151 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-04T03:32:06.151 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.11 2024-05-04T03:32:06.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:05.871+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:06.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:05.871+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:06.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:05.871+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:06.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:05.871+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:06.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:05.871+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:06.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:05.871+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:06.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:05 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:05.871+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:06.428 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 systemd[1]: Stopping Ceph osd.11 for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:06.838 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.462+0000 7f77d88df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:06.838 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.462+0000 7f77d88df700 -1 osd.11 83 *** Got signal Terminated *** 2024-05-04T03:32:06.838 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.462+0000 7f77d88df700 -1 osd.11 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:32:06.838 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.835+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:06.838 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.835+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:06.838 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.835+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:06.838 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.835+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:06.839 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.835+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:06.839 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.835+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:06.839 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:06 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:06.835+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:08.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:07.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:08.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:07.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:08.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:07.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:08.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:07.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:08.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:07.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:08.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:07.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:08.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:07 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:07.812+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:08.858+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:08.858+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:08.858+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:08.858+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:08.858+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:08.858+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:09.136 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:08 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:08.858+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:10.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:09.906+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:10.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:09.906+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:10.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:09.906+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:10.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:09.906+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:10.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:09.906+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:10.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:09.906+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:10.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:09.906+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:10.179 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:09 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:09.906+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-05-04T03:31:43.878746+0000 front 2024-05-04T03:31:43.878802+0000 (oldest deadline 2024-05-04T03:32:09.778767+0000) 2024-05-04T03:32:11.428 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:10.940+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-05-04T03:30:40.269791+0000 front 2024-05-04T03:30:40.269761+0000 (oldest deadline 2024-05-04T03:31:03.769796+0000) 2024-05-04T03:32:11.428 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:10.940+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-05-04T03:30:48.971595+0000 front 2024-05-04T03:30:48.971571+0000 (oldest deadline 2024-05-04T03:31:14.871621+0000) 2024-05-04T03:32:11.428 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:10.940+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-05-04T03:30:54.872013+0000 front 2024-05-04T03:30:54.872060+0000 (oldest deadline 2024-05-04T03:31:20.772221+0000) 2024-05-04T03:32:11.428 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:10.940+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-05-04T03:31:01.873231+0000 front 2024-05-04T03:31:01.873252+0000 (oldest deadline 2024-05-04T03:31:27.773270+0000) 2024-05-04T03:32:11.428 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:10.940+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-05-04T03:31:13.674537+0000 front 2024-05-04T03:31:13.674443+0000 (oldest deadline 2024-05-04T03:31:38.974377+0000) 2024-05-04T03:32:11.429 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:10.940+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-05-04T03:31:21.275303+0000 front 2024-05-04T03:31:21.275229+0000 (oldest deadline 2024-05-04T03:31:47.675769+0000) 2024-05-04T03:32:11.429 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:10.940+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-05-04T03:31:36.878167+0000 front 2024-05-04T03:31:36.878060+0000 (oldest deadline 2024-05-04T03:32:02.177908+0000) 2024-05-04T03:32:11.429 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:10 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11[59312]: 2024-05-04T03:32:10.940+0000 7f77d0e8d700 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-05-04T03:31:43.878746+0000 front 2024-05-04T03:31:43.878802+0000 (oldest deadline 2024-05-04T03:32:09.778767+0000) 2024-05-04T03:32:12.178 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:11 smithi167 podman[72069]: 2024-05-04 03:32:11.86913388 +0000 UTC m=+5.519526879 container died 0978b4fcde11f680baebb29a414eb0ed9304792eb7f499808b09fd959232dfbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-04T03:32:12.678 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:12 smithi167 podman[72069]: 2024-05-04 03:32:12.179905535 +0000 UTC m=+5.830298534 container cleanup 0978b4fcde11f680baebb29a414eb0ed9304792eb7f499808b09fd959232dfbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:32:12.678 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:12 smithi167 bash[72069]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11 2024-05-04T03:32:12.678 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:12 smithi167 podman[72086]: 2024-05-04 03:32:12.406051308 +0000 UTC m=+0.537638856 container remove 0978b4fcde11f680baebb29a414eb0ed9304792eb7f499808b09fd959232dfbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-04T03:32:12.994 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:12 smithi167 podman[72177]: 2024-05-04 03:32:12.644903229 +0000 UTC m=+0.018365134 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:32:12.994 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:12 smithi167 podman[72177]: 2024-05-04 03:32:12.765431982 +0000 UTC m=+0.138893902 container create b0bf341b8272021a82bf982d484c2326bc6c5c8a5019771da0a67a17119a522b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:32:12.994 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:12 smithi167 podman[72177]: 2024-05-04 03:32:12.992770161 +0000 UTC m=+0.366232041 container init b0bf341b8272021a82bf982d484c2326bc6c5c8a5019771da0a67a17119a522b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True) 2024-05-04T03:32:13.287 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:12 smithi167 podman[72177]: 2024-05-04 03:32:12.995943353 +0000 UTC m=+0.369405232 container start b0bf341b8272021a82bf982d484c2326bc6c5c8a5019771da0a67a17119a522b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-04T03:32:13.287 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:13 smithi167 podman[72177]: 2024-05-04 03:32:13.059450508 +0000 UTC m=+0.432912389 container attach b0bf341b8272021a82bf982d484c2326bc6c5c8a5019771da0a67a17119a522b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-05-04T03:32:13.555 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:13 smithi167 podman[72177]: 2024-05-04 03:32:13.286197264 +0000 UTC m=+0.659659149 container died b0bf341b8272021a82bf982d484c2326bc6c5c8a5019771da0a67a17119a522b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-04T03:32:13.812 INFO:journalctl@ceph.osd.11.smithi167.stdout:May 04 03:32:13 smithi167 podman[72203]: 2024-05-04 03:32:13.796340902 +0000 UTC m=+0.500188980 container remove b0bf341b8272021a82bf982d484c2326bc6c5c8a5019771da0a67a17119a522b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-04T03:32:13.818 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@osd.11.service' 2024-05-04T03:32:13.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:13.896 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-04T03:32:13.896 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-04T03:32:13.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.a 2024-05-04T03:32:14.192 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:32:13 smithi039 systemd[1]: Stopping Ceph mds.a for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:14.692 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:32:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a[70069]: 2024-05-04T03:32:14.224+0000 7fcbe052d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:14.692 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:32:14 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a[70069]: 2024-05-04T03:32:14.224+0000 7fcbe052d700 -1 mds.a *** got signal Terminated *** 2024-05-04T03:32:16.917 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:32:16 smithi039 podman[119778]: 2024-05-04 03:32:16.658486479 +0000 UTC m=+2.559247439 container died 2d319958b6f3924d18b066ad8542eec1945502ca820fed1c4bef92263bba7bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-04T03:32:17.175 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:32:16 smithi039 podman[119778]: 2024-05-04 03:32:16.997920586 +0000 UTC m=+2.898681545 container cleanup 2d319958b6f3924d18b066ad8542eec1945502ca820fed1c4bef92263bba7bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-04T03:32:17.176 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:32:16 smithi039 bash[119778]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a 2024-05-04T03:32:17.430 INFO:journalctl@ceph.mds.a.smithi039.stdout:May 04 03:32:17 smithi039 podman[119792]: 2024-05-04 03:32:17.173968255 +0000 UTC m=+0.513765970 container remove 2d319958b6f3924d18b066ad8542eec1945502ca820fed1c4bef92263bba7bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-04T03:32:17.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.a.service' 2024-05-04T03:32:17.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:17.470 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-04T03:32:17.470 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-04T03:32:17.470 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.d 2024-05-04T03:32:17.692 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:32:17 smithi039 systemd[1]: Stopping Ceph mds.d for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:18.192 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:32:17 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d[68780]: 2024-05-04T03:32:17.783+0000 7f5d51e5b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:18.192 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:32:17 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d[68780]: 2024-05-04T03:32:17.783+0000 7f5d51e5b700 -1 mds.d *** got signal Terminated *** 2024-05-04T03:32:19.988 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:32:19 smithi039 podman[119900]: 2024-05-04 03:32:19.693624675 +0000 UTC m=+2.003426691 container died 712b077342d1d79fb3b16319996f78e88e9f49fbf4861fdc675d3353509c0bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-04T03:32:20.313 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:32:20 smithi039 podman[119900]: 2024-05-04 03:32:20.085095205 +0000 UTC m=+2.394897219 container cleanup 712b077342d1d79fb3b16319996f78e88e9f49fbf4861fdc675d3353509c0bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:32:20.313 INFO:journalctl@ceph.mds.d.smithi039.stdout:May 04 03:32:20 smithi039 bash[119900]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-d 2024-05-04T03:32:20.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.d.service' 2024-05-04T03:32:20.483 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:20.483 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-04T03:32:20.483 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-04T03:32:20.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.g 2024-05-04T03:32:20.942 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:32:20 smithi039 systemd[1]: Stopping Ceph mds.g for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:20.942 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:32:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g[69599]: 2024-05-04T03:32:20.804+0000 7ff9f83d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:20.942 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:32:20 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g[69599]: 2024-05-04T03:32:20.804+0000 7ff9f83d5700 -1 mds.g *** got signal Terminated *** 2024-05-04T03:32:24.942 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:32:24 smithi039 podman[120019]: 2024-05-04 03:32:24.677041543 +0000 UTC m=+3.972462340 container died cfacbc7484eee37738833ee0d543b3a37542b460fe83947a00c37e174bb08201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-04T03:32:25.324 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:32:25 smithi039 podman[120019]: 2024-05-04 03:32:25.079843204 +0000 UTC m=+4.375264002 container cleanup cfacbc7484eee37738833ee0d543b3a37542b460fe83947a00c37e174bb08201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-04T03:32:25.324 INFO:journalctl@ceph.mds.g.smithi039.stdout:May 04 03:32:25 smithi039 bash[120019]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-g 2024-05-04T03:32:25.439 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.g.service' 2024-05-04T03:32:25.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:25.471 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-04T03:32:25.472 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-04T03:32:25.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.j 2024-05-04T03:32:25.692 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:32:25 smithi039 systemd[1]: Stopping Ceph mds.j for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:26.192 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:32:25 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j[68330]: 2024-05-04T03:32:25.799+0000 7f8a2ad94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:26.192 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:32:25 smithi039 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j[68330]: 2024-05-04T03:32:25.799+0000 7f8a2ad94700 -1 mds.j *** got signal Terminated *** 2024-05-04T03:32:27.797 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:32:27 smithi039 podman[120140]: 2024-05-04 03:32:27.543635781 +0000 UTC m=+1.841503474 container died a7f4071649bb588c62874ab4745c1d691283391fc06b68a2e79335c8e19176ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=) 2024-05-04T03:32:28.061 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:32:27 smithi039 podman[120140]: 2024-05-04 03:32:27.883393014 +0000 UTC m=+2.181260716 container cleanup a7f4071649bb588c62874ab4745c1d691283391fc06b68a2e79335c8e19176ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:32:28.061 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:32:27 smithi039 bash[120140]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j 2024-05-04T03:32:28.316 INFO:journalctl@ceph.mds.j.smithi039.stdout:May 04 03:32:28 smithi039 podman[120155]: 2024-05-04 03:32:28.059446354 +0000 UTC m=+0.511836154 container remove a7f4071649bb588c62874ab4745c1d691283391fc06b68a2e79335c8e19176ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-04T03:32:28.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.j.service' 2024-05-04T03:32:28.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:28.353 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-04T03:32:28.353 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-04T03:32:28.353 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.b 2024-05-04T03:32:28.673 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:32:28 smithi066 systemd[1]: Stopping Ceph mds.b for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:29.069 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:32:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b[63850]: 2024-05-04T03:32:28.671+0000 7f38ab90f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:29.069 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:32:28 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b[63850]: 2024-05-04T03:32:28.671+0000 7f38ab90f700 -1 mds.b *** got signal Terminated *** 2024-05-04T03:32:32.819 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:32:32 smithi066 podman[71219]: 2024-05-04 03:32:32.553376678 +0000 UTC m=+3.995374075 container died e870791c88c2cac3647aba47ddea47e9d44073b652e1646e99c80bb384b965cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-04T03:32:33.160 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:32:33 smithi066 podman[71219]: 2024-05-04 03:32:33.03775238 +0000 UTC m=+4.479749780 container cleanup e870791c88c2cac3647aba47ddea47e9d44073b652e1646e99c80bb384b965cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430) 2024-05-04T03:32:33.161 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:32:33 smithi066 bash[71219]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b 2024-05-04T03:32:33.430 INFO:journalctl@ceph.mds.b.smithi066.stdout:May 04 03:32:33 smithi066 podman[71234]: 2024-05-04 03:32:33.305627567 +0000 UTC m=+0.746069711 container remove e870791c88c2cac3647aba47ddea47e9d44073b652e1646e99c80bb384b965cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-05-04T03:32:33.447 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.b.service' 2024-05-04T03:32:33.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:33.484 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-04T03:32:33.484 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-04T03:32:33.484 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.e 2024-05-04T03:32:33.819 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:32:33 smithi066 systemd[1]: Stopping Ceph mds.e for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:34.319 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:32:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e[63231]: 2024-05-04T03:32:33.883+0000 7fe67bf39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:34.319 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:32:33 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e[63231]: 2024-05-04T03:32:33.883+0000 7fe67bf39700 -1 mds.e *** got signal Terminated *** 2024-05-04T03:32:36.819 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:32:36 smithi066 podman[71351]: 2024-05-04 03:32:36.557942076 +0000 UTC m=+2.840911460 container died 2ac28a1a403f299934fafdc974d4f80e9ceec66b990a8798b74bd4fa1d0e4159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-04T03:32:37.185 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:32:36 smithi066 podman[71351]: 2024-05-04 03:32:36.916381044 +0000 UTC m=+3.199350444 container cleanup 2ac28a1a403f299934fafdc974d4f80e9ceec66b990a8798b74bd4fa1d0e4159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-04T03:32:37.186 INFO:journalctl@ceph.mds.e.smithi066.stdout:May 04 03:32:36 smithi066 bash[71351]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-e 2024-05-04T03:32:37.298 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.e.service' 2024-05-04T03:32:37.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:37.333 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-04T03:32:37.333 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-04T03:32:37.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.h 2024-05-04T03:32:37.569 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:37 smithi066 systemd[1]: Stopping Ceph mds.h for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:38.069 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h[63541]: 2024-05-04T03:32:37.653+0000 7fb369e76700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:38.069 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:37 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h[63541]: 2024-05-04T03:32:37.653+0000 7fb369e76700 -1 mds.h *** got signal Terminated *** 2024-05-04T03:32:40.822 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:40 smithi066 podman[71473]: 2024-05-04 03:32:40.56897932 +0000 UTC m=+3.017056736 container died 3405fdf73df94a03f86b826c86ec0d44858bcbbaf5dc7c020f14f51146282b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-04T03:32:41.090 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:40 smithi066 podman[71473]: 2024-05-04 03:32:40.89524972 +0000 UTC m=+3.343327126 container cleanup 3405fdf73df94a03f86b826c86ec0d44858bcbbaf5dc7c020f14f51146282b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:32:41.090 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:40 smithi066 bash[71473]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h 2024-05-04T03:32:41.333 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.h.service' 2024-05-04T03:32:41.354 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:41 smithi066 podman[71487]: 2024-05-04 03:32:41.087907066 +0000 UTC m=+0.512131644 container remove 3405fdf73df94a03f86b826c86ec0d44858bcbbaf5dc7c020f14f51146282b49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-04T03:32:41.354 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:41 smithi066 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-04T03:32:41.354 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:41 smithi066 systemd[1]: Stopped Ceph mds.h for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:32:41.354 INFO:journalctl@ceph.mds.h.smithi066.stdout:May 04 03:32:41 smithi066 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.h.service: Consumed 3.048s CPU time. 2024-05-04T03:32:41.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:41.365 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-04T03:32:41.365 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-04T03:32:41.365 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.k 2024-05-04T03:32:41.701 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:32:41 smithi066 systemd[1]: Stopping Ceph mds.k for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:42.069 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:32:41 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k[62910]: 2024-05-04T03:32:41.699+0000 7fbf47fb1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:42.069 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:32:41 smithi066 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k[62910]: 2024-05-04T03:32:41.699+0000 7fbf47fb1700 -1 mds.k *** got signal Terminated *** 2024-05-04T03:32:44.891 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:32:44 smithi066 podman[71592]: 2024-05-04 03:32:44.569015123 +0000 UTC m=+2.975358089 container died ba2bcf5d2559911a365479cdd5769bd0384521601f323188b18317cfaaf0485c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:32:44.892 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:32:44 smithi066 podman[71592]: 2024-05-04 03:32:44.890094513 +0000 UTC m=+3.296437478 container cleanup ba2bcf5d2559911a365479cdd5769bd0384521601f323188b18317cfaaf0485c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , ceph=True) 2024-05-04T03:32:45.318 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:32:44 smithi066 bash[71592]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k 2024-05-04T03:32:45.318 INFO:journalctl@ceph.mds.k.smithi066.stdout:May 04 03:32:45 smithi066 podman[71607]: 2024-05-04 03:32:45.091133301 +0000 UTC m=+0.519775249 container remove ba2bcf5d2559911a365479cdd5769bd0384521601f323188b18317cfaaf0485c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-04T03:32:45.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.k.service' 2024-05-04T03:32:45.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:45.366 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-04T03:32:45.366 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-04T03:32:45.366 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.c 2024-05-04T03:32:45.678 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:45 smithi167 systemd[1]: Stopping Ceph mds.c for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:45.943 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c[64779]: 2024-05-04T03:32:45.763+0000 7f892baa0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:45.943 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:45 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c[64779]: 2024-05-04T03:32:45.763+0000 7f892baa0700 -1 mds.c *** got signal Terminated *** 2024-05-04T03:32:48.815 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:48 smithi167 podman[72282]: 2024-05-04 03:32:48.562683247 +0000 UTC m=+3.004215181 container died c63b4a6b05de4274b4e876e8a7a81b30497dea6cec76f35963f541892e3e8de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-05-04T03:32:49.073 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:48 smithi167 podman[72282]: 2024-05-04 03:32:48.904370077 +0000 UTC m=+3.345901998 container cleanup c63b4a6b05de4274b4e876e8a7a81b30497dea6cec76f35963f541892e3e8de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-04T03:32:49.073 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:48 smithi167 bash[72282]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c 2024-05-04T03:32:49.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:49 smithi167 podman[72308]: 2024-05-04 03:32:49.07221928 +0000 UTC m=+0.503753657 container remove c63b4a6b05de4274b4e876e8a7a81b30497dea6cec76f35963f541892e3e8de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-04T03:32:49.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:49 smithi167 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.c.service: Deactivated successfully. 2024-05-04T03:32:49.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:49 smithi167 systemd[1]: Stopped Ceph mds.c for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:32:49.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:May 04 03:32:49 smithi167 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.c.service: Consumed 54.905s CPU time. 2024-05-04T03:32:49.325 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.c.service' 2024-05-04T03:32:49.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:49.360 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-04T03:32:49.360 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-04T03:32:49.360 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.f 2024-05-04T03:32:49.669 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:49 smithi167 systemd[1]: Stopping Ceph mds.f for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:49.928 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:49 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f[64463]: 2024-05-04T03:32:49.666+0000 7efff598e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:49.928 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:49 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f[64463]: 2024-05-04T03:32:49.666+0000 7efff598e700 -1 mds.f *** got signal Terminated *** 2024-05-04T03:32:52.802 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:52 smithi167 podman[72414]: 2024-05-04 03:32:52.535966022 +0000 UTC m=+2.956218314 container died 4c4e8da1ce11ae0af13d91cc17ebd5a906ade5ce31c9ff4d09db574e8d4e2ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-04T03:32:53.061 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:52 smithi167 podman[72414]: 2024-05-04 03:32:52.88343332 +0000 UTC m=+3.303685619 container cleanup 4c4e8da1ce11ae0af13d91cc17ebd5a906ade5ce31c9ff4d09db574e8d4e2ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-04T03:32:53.062 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:52 smithi167 bash[72414]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f 2024-05-04T03:32:53.303 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.f.service' 2024-05-04T03:32:53.323 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:53 smithi167 podman[72428]: 2024-05-04 03:32:53.059552028 +0000 UTC m=+0.519239009 container remove 4c4e8da1ce11ae0af13d91cc17ebd5a906ade5ce31c9ff4d09db574e8d4e2ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430) 2024-05-04T03:32:53.323 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:53 smithi167 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.f.service: Deactivated successfully. 2024-05-04T03:32:53.323 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:53 smithi167 systemd[1]: Stopped Ceph mds.f for 1e913ae0-09c2-11ef-bc97-c7b262605968. 2024-05-04T03:32:53.323 INFO:journalctl@ceph.mds.f.smithi167.stdout:May 04 03:32:53 smithi167 systemd[1]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.f.service: Consumed 3.484s CPU time. 2024-05-04T03:32:53.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:53.335 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-04T03:32:53.335 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-04T03:32:53.335 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.i 2024-05-04T03:32:53.655 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:32:53 smithi167 systemd[1]: Stopping Ceph mds.i for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:53.928 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:32:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i[65089]: 2024-05-04T03:32:53.653+0000 7f1037aba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:53.928 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:32:53 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i[65089]: 2024-05-04T03:32:53.653+0000 7f1037aba700 -1 mds.i *** got signal Terminated *** 2024-05-04T03:32:56.678 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:32:56 smithi167 podman[72536]: 2024-05-04 03:32:56.544348947 +0000 UTC m=+2.993275295 container died bee2384c7f1af00cb8e3bfc3f8a4bdfd574fdf18cbcda55c953321be34e4ddbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240430) 2024-05-04T03:32:57.560 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:32:57 smithi167 podman[72536]: 2024-05-04 03:32:57.203977524 +0000 UTC m=+3.652903882 container cleanup bee2384c7f1af00cb8e3bfc3f8a4bdfd574fdf18cbcda55c953321be34e4ddbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-04T03:32:57.560 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:32:57 smithi167 bash[72536]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i 2024-05-04T03:32:57.560 INFO:journalctl@ceph.mds.i.smithi167.stdout:May 04 03:32:57 smithi167 podman[72550]: 2024-05-04 03:32:57.438318823 +0000 UTC m=+0.889250215 container remove bee2384c7f1af00cb8e3bfc3f8a4bdfd574fdf18cbcda55c953321be34e4ddbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-i, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-04T03:32:57.582 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.i.service' 2024-05-04T03:32:57.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:32:57.614 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-04T03:32:57.614 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-04T03:32:57.614 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.l 2024-05-04T03:32:57.918 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:32:57 smithi167 systemd[1]: Stopping Ceph mds.l for 1e913ae0-09c2-11ef-bc97-c7b262605968... 2024-05-04T03:32:58.178 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:32:57 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l[64143]: 2024-05-04T03:32:57.915+0000 7f98a9b46700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:32:58.181 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:32:57 smithi167 ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l[64143]: 2024-05-04T03:32:57.915+0000 7f98a9b46700 -1 mds.l *** got signal Terminated *** 2024-05-04T03:33:00.810 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:33:00 smithi167 podman[72656]: 2024-05-04 03:33:00.55180662 +0000 UTC m=+2.730927837 container died 5db8322d980a8103ee51b2ea2d250989854fe42b3bb0549f556e0ad24cbe63eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-04T03:33:01.178 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:33:00 smithi167 podman[72656]: 2024-05-04 03:33:00.908044414 +0000 UTC m=+3.087165639 container cleanup 5db8322d980a8103ee51b2ea2d250989854fe42b3bb0549f556e0ad24cbe63eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-04T03:33:01.178 INFO:journalctl@ceph.mds.l.smithi167.stdout:May 04 03:33:00 smithi167 bash[72656]: ceph-1e913ae0-09c2-11ef-bc97-c7b262605968-mds-l 2024-05-04T03:33:01.324 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1e913ae0-09c2-11ef-bc97-c7b262605968@mds.l.service' 2024-05-04T03:33:01.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:33:01.355 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-04T03:33:01.355 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-04T03:33:01.556 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:33:04.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-04T03:33:04.408 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:33:06.999 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-04T03:33:07.190 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:33:09.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:33:09.851 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:33:09.885 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:33:09.910 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-04T03:33:09.912 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi039/crash 2024-05-04T03:33:09.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash -- . 2024-05-04T03:33:09.951 INFO:teuthology.orchestra.run.smithi039.stderr:tar: /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-04T03:33:09.951 INFO:teuthology.orchestra.run.smithi039.stderr:tar: Error is not recoverable: exiting now 2024-05-04T03:33:09.953 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi066/crash 2024-05-04T03:33:09.954 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash -- . 2024-05-04T03:33:09.994 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-04T03:33:09.994 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2024-05-04T03:33:09.995 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi167/crash 2024-05-04T03:33:09.996 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash -- . 2024-05-04T03:33:10.030 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-04T03:33:10.030 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-05-04T03:33:10.031 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-04T03:33:10.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-04T03:33:10.143 INFO:tasks.cephadm:Compressing logs... 2024-05-04T03:33:10.144 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-04T03:33:10.146 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-04T03:33:10.149 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-04T03:33:10.173 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log 2024-05-04T03:33:10.173 INFO:teuthology.orchestra.run.smithi167.stderr:: No such file or directory 2024-05-04T03:33:10.173 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-04T03:33:10.173 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-04T03:33:10.173 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-04T03:33:10.173 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-04T03:33:10.173 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-04T03:33:10.174 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-04T03:33:10.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-04T03:33:10.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-04T03:33:10.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.z.log: 87.4% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-04T03:33:10.177 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- 90.6% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-04T03:33:10.177 INFO:teuthology.orchestra.run.smithi167.stderr: /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-04T03:33:10.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-04T03:33:10.177 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log: 83.3% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-04T03:33:10.178 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log 2024-05-04T03:33:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:: No such file or directory 2024-05-04T03:33:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-04T03:33:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-04T03:33:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-04T03:33:10.179 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-04T03:33:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-04T03:33:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-04T03:33:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-04T03:33:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-04T03:33:10.182 INFO:teuthology.orchestra.run.smithi066.stderr: 90.5% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log.gz/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.h.log: 2024-05-04T03:33:10.182 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-04T03:33:10.182 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-04T03:33:10.185 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-04T03:33:10.189 INFO:teuthology.orchestra.run.smithi167.stderr: 93.4% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-04T03:33:10.189 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-04T03:33:10.189 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-04T03:33:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-04T03:33:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-04T03:33:10.190 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-04T03:33:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-04T03:33:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-04T03:33:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-04T03:33:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-04T03:33:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-04T03:33:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.10.log: 93.2% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-04T03:33:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-04T03:33:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-04T03:33:10.192 INFO:teuthology.orchestra.run.smithi066.stderr: 83.3% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-04T03:33:10.198 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-04T03:33:10.204 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-04T03:33:10.205 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-04T03:33:10.216 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-04T03:33:10.218 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-04T03:33:10.223 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-04T03:33:10.225 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-04T03:33:10.225 INFO:teuthology.orchestra.run.smithi167.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-04T03:33:10.238 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-04T03:33:10.281 INFO:teuthology.orchestra.run.smithi066.stderr: 94.8% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-04T03:33:10.409 INFO:teuthology.orchestra.run.smithi167.stderr: 94.9% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-04T03:33:10.429 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-04T03:33:10.466 INFO:teuthology.orchestra.run.smithi167.stderr: 94.0% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-04T03:33:10.469 INFO:teuthology.orchestra.run.smithi167.stderr: 94.0% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-04T03:33:10.487 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.9.log: /var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-04T03:33:10.491 INFO:teuthology.orchestra.run.smithi066.stderr: 94.0% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-04T03:33:10.526 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-04T03:33:10.526 INFO:teuthology.orchestra.run.smithi039.stderr: 83.3% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-04T03:33:10.734 INFO:teuthology.orchestra.run.smithi039.stderr: 90.4% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-04T03:33:10.800 INFO:teuthology.orchestra.run.smithi167.stderr: 91.9% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-04T03:33:10.838 INFO:teuthology.orchestra.run.smithi066.stderr: 91.9% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-04T03:33:11.095 INFO:teuthology.orchestra.run.smithi039.stderr: 94.6% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-04T03:33:11.448 INFO:teuthology.orchestra.run.smithi039.stderr: 94.7% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-04T03:33:11.464 INFO:teuthology.orchestra.run.smithi039.stderr: 90.3% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-04T03:33:11.808 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-04T03:33:14.002 INFO:teuthology.orchestra.run.smithi039.stderr: 94.7% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-04T03:33:14.922 INFO:teuthology.orchestra.run.smithi039.stderr: 90.1% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-04T03:33:16.467 INFO:teuthology.orchestra.run.smithi039.stderr: 96.1% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-04T03:33:24.727 INFO:teuthology.orchestra.run.smithi167.stderr: 95.1% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-04T03:33:41.941 INFO:teuthology.orchestra.run.smithi066.stderr: 91.4% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-04T03:33:44.276 INFO:teuthology.orchestra.run.smithi039.stderr: 90.7% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-04T03:33:46.839 INFO:teuthology.orchestra.run.smithi167.stderr: 90.9% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-04T03:33:48.949 INFO:teuthology.orchestra.run.smithi167.stderr: 91.0% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-04T03:33:49.308 INFO:teuthology.orchestra.run.smithi066.stderr: 90.7% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-04T03:33:49.345 INFO:teuthology.orchestra.run.smithi167.stderr: 90.6% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-04T03:33:50.044 INFO:teuthology.orchestra.run.smithi039.stderr: 90.7% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-04T03:33:50.891 INFO:teuthology.orchestra.run.smithi066.stderr: 90.5% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-04T03:33:51.517 INFO:teuthology.orchestra.run.smithi167.stderr: 90.7% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-04T03:33:51.519 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-04T03:33:51.519 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m41.356s 2024-05-04T03:33:51.520 INFO:teuthology.orchestra.run.smithi167.stderr:user 2m46.266s 2024-05-04T03:33:51.520 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m5.143s 2024-05-04T03:33:55.441 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-04T03:33:59.980 INFO:teuthology.orchestra.run.smithi066.stderr: 90.8% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-04T03:33:59.982 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-05-04T03:33:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m49.817s 2024-05-04T03:33:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:user 2m39.316s 2024-05-04T03:33:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m4.551s 2024-05-04T03:34:20.313 INFO:teuthology.orchestra.run.smithi039.stderr: 90.6% -- replaced with /var/log/ceph/1e913ae0-09c2-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-04T03:35:09.955 INFO:teuthology.orchestra.run.smithi039.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-04T03:35:09.957 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-05-04T03:35:09.957 INFO:teuthology.orchestra.run.smithi039.stderr:real 1m59.796s 2024-05-04T03:35:09.957 INFO:teuthology.orchestra.run.smithi039.stderr:user 3m56.627s 2024-05-04T03:35:09.957 INFO:teuthology.orchestra.run.smithi039.stderr:sys 0m7.791s 2024-05-04T03:35:09.958 INFO:tasks.cephadm:Archiving logs... 2024-05-04T03:35:09.958 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi039/log 2024-05-04T03:35:09.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-04T03:35:47.709 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi066/log 2024-05-04T03:35:47.711 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-04T03:36:16.243 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi167/log 2024-05-04T03:36:16.244 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-04T03:36:43.998 INFO:tasks.cephadm:Removing cluster... 2024-05-04T03:36:43.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 --force 2024-05-04T03:36:44.210 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:36:44.712 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 --force 2024-05-04T03:36:44.913 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:36:45.446 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1e913ae0-09c2-11ef-bc97-c7b262605968 --force 2024-05-04T03:36:45.637 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 1e913ae0-09c2-11ef-bc97-c7b262605968 2024-05-04T03:36:46.228 INFO:tasks.cephadm:Removing cephadm ... 2024-05-04T03:36:46.228 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-04T03:36:46.248 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-04T03:36:46.266 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-04T03:36:46.281 INFO:tasks.cephadm:Teardown complete 2024-05-04T03:36:46.282 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-04T03:36:46.292 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-04T03:36:46.292 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-04T03:36:46.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-04T03:36:46.308 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-04T03:36:46.408 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-04T03:36:46.442 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-04T03:36:46.442 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:36:46.446 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:36:46.450 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:36:46.461 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:36:46.462 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:36:46.464 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:36:46.465 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:36:46.465 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-05-04T03:36:46.465 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +531us[ +531us] +/- 53ms 2024-05-04T03:36:46.465 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -1568us[-1568us] +/- 57ms 2024-05-04T03:36:46.465 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +1280us[+1280us] +/- 76ms 2024-05-04T03:36:46.466 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:36:46.467 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:36:46.467 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-05-04T03:36:46.467 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +565us[ +586us] +/- 53ms 2024-05-04T03:36:46.467 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -1537us[-1537us] +/- 57ms 2024-05-04T03:36:46.468 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +1306us[+1306us] +/- 76ms 2024-05-04T03:36:46.468 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:36:46.469 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:36:46.469 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-05-04T03:36:46.469 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +539us[ +560us] +/- 53ms 2024-05-04T03:36:46.469 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -1567us[-1547us] +/- 57ms 2024-05-04T03:36:46.469 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +1272us[+1272us] +/- 76ms 2024-05-04T03:36:46.469 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:36:46.469 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-04T03:36:46.583 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-04T03:36:46.583 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-04T03:36:46.700 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-04T03:36:46.834 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-04T03:36:46.903 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-04T03:36:46.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T03:36:47.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:36:47.003 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T03:36:47.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:36:47.031 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T03:36:47.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:36:47.059 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-04T03:36:47.071 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-04T03:36:47.125 INFO:teuthology.task.internal:Duration was 2674.391183 seconds 2024-05-04T03:36:47.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-04T03:36:47.168 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-04T03:36:47.169 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-04T03:36:47.172 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-04T03:36:47.174 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-04T03:36:47.211 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T03:36:47.212 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T03:36:47.244 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T03:36:47.714 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-04T03:36:47.714 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-05-04T03:36:47.714 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-04T03:36:47.740 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-05-04T03:36:47.740 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-04T03:36:47.767 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-05-04T03:36:47.767 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-04T03:36:47.792 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-04T03:36:47.792 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-04T03:36:47.795 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-04T03:36:47.810 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-04T03:36:47.858 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-04T03:36:47.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-04T03:36:47.867 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-04T03:36:47.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-04T03:36:48.694 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-04T03:36:48.706 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-04T03:36:48.706 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-04T03:36:48.762 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-04T03:36:48.791 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-04T03:36:48.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-04T03:36:48.833 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:36:48.836 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:36:48.840 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:36:48.862 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-05-04T03:36:48.866 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-05-04T03:36:48.878 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-05-04T03:36:48.907 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:36:48.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:36:48.935 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:36:48.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:36:48.964 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:36:48.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:36:48.978 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-04T03:36:48.997 INFO:teuthology.task.internal:Transferring archived files... 2024-05-04T03:36:48.999 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi039 2024-05-04T03:36:48.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-04T03:36:49.070 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi066 2024-05-04T03:36:49.070 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-04T03:36:49.117 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688612/remote/smithi167 2024-05-04T03:36:49.117 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-04T03:36:49.158 INFO:teuthology.task.internal:Removing archive directory... 2024-05-04T03:36:49.158 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-04T03:36:49.161 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-04T03:36:49.164 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-04T03:36:49.204 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-04T03:36:49.217 INFO:teuthology.task.internal:Not uploading archives. 2024-05-04T03:36:49.218 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-04T03:36:49.255 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-04T03:36:49.256 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-04T03:36:49.258 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-04T03:36:49.261 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-04T03:36:49.271 INFO:teuthology.orchestra.run.smithi039.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 4 03:36 /home/ubuntu/cephtest 2024-05-04T03:36:49.276 INFO:teuthology.orchestra.run.smithi167.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 4 03:36 /home/ubuntu/cephtest 2024-05-04T03:36:49.276 INFO:teuthology.orchestra.run.smithi066.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 4 03:36 /home/ubuntu/cephtest 2024-05-04T03:36:49.277 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-04T03:36:49.292 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-04T03:36:49.425 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 2674.3911833763123 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-04T03:36:49.425 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-04T03:36:49.524 INFO:teuthology.run:pass